This paper develops an extremely robust solution for absolute pose estimation with known prior gravity direction by motion decoupling. Absolute pose estimation is a fundamental problem in computer vision, and recently the prior known vertical direction is commonly applied to help solve the pose estimation problem. In this paper, we explore the geometrical constraints of the absolute pose estimation with a known direction. We find that the rigid pose can be decoupled with the help of the known direction. Thereby, absolute pose estimation algorithms, which decouple rigid motion, are proposed. Notably, in real applications, there may be imperfect inputs, i.e., outliers, due to incorrect 2D-3D matches. Unfortunately, these outliers may lead to unacceptable results. To suppress the outliers, the decoupled absolute pose estimation problem is solved by branch-and-bound algorithm and globally voting, which can provide the optimal solution with provable guarantees. Moreover, in extreme case, the proposed method can solve absolute pose estimation problem without knowing the 2D-3D correspondences, which is also known as simultaneous camera pose correspondence estimation. To demonstrate the feasibility and the superiority of the proposed methods, comprehensive comparison experiment are conduced. The source code is available at https://github.com/Liu-Yinlong/algorithm-for-PnP-with-known-vertical-direction.