[ros-users] Correlating a point from a stereo cloud with a pixel clicked on the stereo camera's screen.

[zizi poil]

Dear all,

I am trying to build a node that would reproduce the basics of the camera
pluggin of rviz.
The difference is that I'd like to be able to correlate a point clicked by
the user on the camera screen (in pixels)  with a point from a 3D point
cloud (in meters).

In other words, I have my stereo camera that generates a 3D point cloud and
I would like to allow the user to select on of these 3D points using the 2D
image of the left or right eye.
If the user clicks a point were no 3d info is available, then I'd like to
choose the closest 3d point available.

So, I need a way to correlate x,y coordinates in pixels with x,y,z
coordinates in meters.

I am thinking of 2 main alternatives:

A) project the 3d-points back to a stereo_sensor frame that would represent
the sensor of the camera, and from there convert my meters in pixels (I know
1 pixel is 6 um on my sensor)
B) send a ray from my pixel point and see if I collide into a 3d point (I
have no idea how to do this, maybe bullet apparently?)

I have the feeling tf can help me for the first alternative.
Still, would anyone know precisely how to achieve this and which packages
would help in this way (ROS is really wide and I am new to it).
Also, am I taking the good approach or is there a far more simple way do
this?

Thanks for your help

Raphael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/ros-users/attachments/20100609/8a18ec0a/attachment-0002.html>