Hello all,

As far as I understand, the image_proc pipeline uses the initUndistortRectifyMap [1] function to rectify incoming images, and then it passes them along. It subscribes to an incoming camera info containing the intrinsic distortion matrices of the distorted image.

From the OpenCV documentation: "The undistorted image looks like original, as if it is captured with a camera using the camera matrix =newCameraMatrix. In case of a monocular camera, newCameraMatrix is usually equal to cameraMatrix , or it can be computed by getOptimalNewCameraMatrix() [1]

I compared the original vs optimal matrices for the default calibration of the Xtion depth camera:

Original:

--------------------------------------

[578.8, 0, 316.0;
0, 574.6, 246.0;
0, 0, 1]

newCameraMatrix (from getOptimalNewCameraMatrix)

--------------------------------------
[573.4, 0, 315.5;
0, 570.8, 245.2;
0, 0, 1]

It seems that there is a small, yet probably significant difference between the two. I expect this difference will be bigger for cameras with higher distortion.

Wouldn't it make sense for image_proc to publish the rectified (optimal) matrix, in the same way the original (distorted) camera info is published?

Here's what the topics look like right now [2]:

Subscibed:

 * image_raw (Raw image stream from the camera driver)
 * camera_info (Camera metadata)
Published:
 * image_mono (Monochrome unrectified image)

 * image_rect (Monochrome rectified image)
 * image_color (Color unrectified image)
 * image_rect_color (Color rectified image)

Without breaking the API, we could add a topic:

Published:

 * camera_rect_info

It would be even nicer (but API-breaking) to clean up the namespaces. For example

Subscribed

 * image/raw (raw camera stream)

 * image/info (metadata)

[1] http://docs.opencv.org/modules/imgproc/doc/geometric_transformations.html#initundistortrectifymap

[2] http://www.ros.org/wiki/image_proc