Supplying partial poses for conditioning

[ReneWilbers-Basicfit]

Thank you for the great work!

I am about to start some experiments with Pi3X conditioning.
I have a set of camera positions and yaws for my building, but I do not have data on pitch, roll and intrinsics.

Is it true that you always need to supply all of these in multimodal mode? Or is there a way to let the model fit in missing values?

[yyfz]

Hi, ReneWilbers-Basicfi!

Pi3X only allows full rotation matrix to condition.

[ReneWilbers-Basicfit]

Allright, thank you. And as I understand from Pi3X.encode, you need not only full rotation matrix but also full intrinsic mats when we supply pose mats (if poses is None: ... else: assert rays is not None )

I am now thinking of solving this by inferencing first single images, fit floor planes, retrieve pitch & roll relative to floor plane, fit camera intrinsics, and then do a second conditioned inference. Quite a complex setup, but seems the only option then.

Do you think the first inference would give realiable enough results for this?

[yyfz]

You can try this: Predict the initial camera poses first, then manually override the yaw and position with known values. Finally, run the model again conditioned on these modified poses.