On Wed, Mar 17, 2010 at 8:27 PM, Jack O'Quin <jack.oquin@gmail.com> wrote:
I'm having trouble figuring out how to set the camera matrix (K). The
CameraInfo.msg says:

# Intrinsic camera matrix for the raw (distorted) images
# Projects 3D points in the camera coordinate frame to 2D pixel
# coordinates using the focal lengths (fx, fy) and principal point
# (cx, cy):
#     [fx  0 cx]
# K = [ 0 fy cy]
#     [ 0  0  1]

float64[9]  K # 3x3 row-major matrix

I don't understand how to obtain this information from the dc1394 interface.

Probably you can't.

If it's not available, then I suppose fx, fy, cx and cy would have to
be parameters defined for each camera. How does the user determine
them?

Every camera needs to be calibrated separately. As Blaise mentioned we have the camera_calibration package for doing that with a checkerboard target.

Are there reasonable defaults to use when these data are not
available?

You should leave the calibration matrices as all zeros, which is what the CameraInfo constructor sets them to. image_proc and stereo_image_proc understand that to mean that the camera is uncalibrated. In that case they will only publish on the debayered image_mono and image_color topics; all the other topics (rectified images, stereo outputs) require a calibrated camera.

If you have very high-quality components (precisely manufactured camera sensor, distortion-free lens) you might get reasonable results just setting the values from the spec sheets, though I'd still recommend doing a calibration. In that case you would set fx = fy = focal length in meters, and (cx, cy) = the central pixel coordinates. E.g. for 640x480 resolution, cx = 319.5 and cy = 239.5. Distortion (D) you would leave set to zeros. R and P you fill as I described earlier.

It would be useful to display the images in rviz (which
actually does display, but with an error and with the camera window
covering the upper left of the rviz window).

I think rviz is correct to complain, as its image viewer wants to display the image in the context of whatever else rviz knows about the environment. That requires knowing the frame and calibration of the camera. Probably rviz's behavior and error message could be improved for this case though.

image_view doesn't care about the camera info.

How do the PR2 cameras solve this problem? Are the lenses just known
for each model number?

We calibrate all of them individually. For the WGE100 and Prosilica cameras we have, we store the calibration on the camera itself in non-volatile memory, but that won't be possible for every camera. As Blaise described you'll need to load the calibration from a file, and have set_camera_info save to a file. We use the YAML format in camera_calibration_parsers for archiving CameraInfo.

Cheers,
Patrick