Hello all,

As far as I understand, the image_proc pipeline uses the
initUndistortRectifyMap [1] function to rectify incoming images, and then
it passes them along. It subscribes to an incoming camera info containing
the intrinsic distortion matrices of the distorted image.

>From the OpenCV documentation: "The undistorted image looks like original,
as if it is captured with a camera using the camera matrix
=newCameraMatrix. In case of a monocular camera, newCameraMatrix is usually
equal to cameraMatrix , or it can be computed by
getOptimalNewCameraMatrix() [1]

I compared the original vs optimal matrices for the default calibration of
the Xtion depth camera:

Original:
--------------------------------------
[578.8, 0, 316.0;
0, 574.6, 246.0;
0, 0, 1]

newCameraMatrix (from getOptimalNewCameraMatrix)
--------------------------------------
[573.4, 0, 315.5;
0, 570.8, 245.2;
0, 0, 1]

It seems that there is a small, yet probably significant difference between
the two. I expect this difference will be bigger for cameras with higher
distortion.

Wouldn't it make sense for image_proc to publish the rectified (optimal)
matrix, in the same way the original (distorted) camera info is published?

Here's what the topics look like right now [2]:

Subscibed:
 * image_raw (Raw image stream from the camera driver)
 * camera_info (Camera metadata)
Published:
 * image_mono (Monochrome unrectified image)
 * image_rect (Monochrome rectified image)
 * image_color (Color unrectified image)
 * image_rect_color (Color rectified image)

Without breaking the API, we could add a topic:

Published:
 * camera_rect_info

It would be even nicer (but API-breaking) to clean up the namespaces. For
example

Subscribed
 * image/raw (raw camera stream)
 * image/info (metadata)
Published:
 * image/mono (Monochrome unrectified image)
 * image/color (Color unrectified image)
 * image/rect/mono (Monochrome rectified image)
 * image/rect/color (Color rectified image)
 * image/rect/info

I was hoping to hear other people's opinions on this subject.

Cheers,

Ivan Dryanovski

[1]
http://docs.opencv.org/modules/imgproc/doc/geometric_transformations.html#initundistortrectifymap
[2] http://www.ros.org/wiki/image_proc