Effect of pnp_error_multiplier

[ayehninnkhine]

Hi,

I'd like to share my findings on the "pnp_error_multiplier" in pose estimation. I'm running an experiment with the CAB dataset using a fusion image retrieval (Netvlad+APGEM), DISK feature extraction, and lightglue matcher. Using Python open3D library, I attempted to visualize the pose estimation results of CAB phone query data (validation data). The value of "pnp_error_multiplier" has a substantial effect on the pose estimation outcome according to my findings. Pose estimation results vary depending on the value of “pnp_error_multiplier”. This scenario also happens when I run the experiments with a custom dataset.

I'm not familiar with this parameter or its impact on pose estimation. Could you kindly explain why this is happening and provide any resources that can help me learn more about it? Thank you very much.

Visualization Result - https://docs.google.com/document/d/1d9DjTCQMIn7Sf36a-cpOcsu8q-ZHK_nqcodjRF0EnYM/edit?usp=sharing

pnp_error_multiplier value (0.0005)
{'Rt_thresholds': [(1, 0.1), (5, 1.0)], 'recall': [0.020202020202020204, 0.08838383838383838]}

pnp_error_multiplier value (3.0)
{'Rt_thresholds': [(1, 0.1), (5, 1.0)], 'recall': [0.3484848484848485, 0.5]}

[mihaidusmanu]

Hello. You can imagine the pnp_error_multiplier as a resolution / feature agnostic RANSAC inlier threshold (it is unit-less; it is multiplied by the keypoint noise in px before being piped as threshold to the estimators).

Example: Let's say our feature extractor only runs at 640 max edge resolution and has a keypoint noise of 1px (these are defined and populated in HLoc).

• For HoloLens data (640x480), that means feature extraction will run at full res so the keypoint noise will stay 1px. Now with a pnp_error_multiplier of 4, the RANSAC inlier threshold will be 4px.
• Now, let's take phone data (1920x1080), that means that we will have to downsample the image by a factor of 3 to have largest edge smaller than 640, leading to a keypoint noise of 3px. With the same pnp_error_multiplier as before, the RANSAC inlier threshold will be 12px now.