OAK-D Pro POE

[stephanschulz]

Your current example uses cam = pipeline.create(dai.node.ColorCamera)
I wonder if you can help me take advantage of the IR laser projector to improve the depth readings.

I know Blazepose used 2D images to infer the depth. I am hoping to use the OAK-D Pro POE at light. This means I should at least use the IR LED to illuminate the scene. And even better use the IR laser dot projector to sample the depths.

Thanks,
Stephan.

[geaxgx]

You may get better depth reading thanks to the IR laser projector, but I think you will still have the potential drawbacks explained there: https://github.com/geaxgx/depthai_blazepose#inferred-3d-vs-measured-3d :
First, a inferred keypoint may stand "outside" of its counterpart in the image and therefore in the aligned depth frame. It happens probably more frequently with extremities, and it can be explained by the inaccuracy of the model which is never 100% perfect. Secondly, we can't get depth for hidden keypoints.

[stephanschulz]

If I understand your code correctly you are currently using the RGB stereo camera to:

• use blaze pose on a 2d image to get 3d points
• then use the stereo camera to get a reference point?
• if so can I replace the stereo camera with the IR projector to get a reference point?

An alternative solution, implemented here, is to combine the inferred 3D world landmarks with the measured 3D location of one reference point, the center between hips. Compared to extremities, this reference point can be more robustly measured.

[geaxgx]

If I understand your code correctly you are currently using the RGB stereo camera to:

use blaze pose on a 2d image to get 3d points
then use the stereo camera to get a reference point?

Yes, that's correct.

if so can I replace the stereo camera with the IR projector to get a reference point?

No, you still need the stereo camera. The IR projector is just helping the stereo matching process. You can see the IR projector as an additional source of light.