[ros-users] Digital Camera 1394 in ROS?

[Jack O'Quin]

On Thu, Mar 18, 2010 at 12:12 PM, Blaise Gassend
<blaise at willowgarage.com> wrote:
>> I'm having trouble figuring out how to set the camera matrix (K). The
>> CameraInfo.msg says:
>>
>> # Intrinsic camera matrix for the raw (distorted) images
>> # Projects 3D points in the camera coordinate frame to 2D pixel
>> # coordinates using the focal lengths (fx, fy) and principal point
>> # (cx, cy):
>> #     [fx  0 cx]
>> # K = [ 0 fy cy]
>> #     [ 0  0  1]
>>
>> float64[9]  K # 3x3 row-major matrix
>>
>> I don't understand how to obtain this information from the dc1394 interface.
>
> It is probably not available in a generic way.

I came to realize that. High-end 1394 cameras can attach different
lenses, so there's no way for the camera electronics to know their
characteristics.

>> If it's not available, then I suppose fx, fy, cx and cy would have to
>> be parameters defined for each camera. How does the user determine
>> them? Are there reasonable defaults to use when these data are not
>> available? It would be useful to display the images in rviz (which
>> actually does display, but with an error and with the camera window
>> covering the upper left of the rviz window).
>>
>> How do the PR2 cameras solve this problem? Are the lenses just known
>> for each model number?
>
> We use the camera_calibration package. It looks at a stream of images
> containing a checker board with known dimensions, and produces
> calibration information. It then uses SetCameraInfo to store this
> information on the camera.

IIDC standard 1394 cameras are not going to provide flash memory to
store calibration data. So, I was thinking about this issue already.

> We have had some discussions on the fact that not all cameras will be
> able to store their camera_info. Probably the easiest way to deal with
> this is to have the camera driver take a file as a parameter. At startup
> it will load its camera_info from that file (if there is an error it
> would use the all-zero camera_info). If SetCameraInfo is called, it
> overwrites that file with the new camera_info, and starts publishing
> that instead.

I had not considered that approach.

A recent discussion on ros-users had convinced me that roslaunch is
essentially hostile to the whole idea of writing files from nodes. It
goes out of its way to change the current working directory to
something useless rather than allow nodes to conveniently write test
output to the current directory.

Apparently, the thinking is that files should generally only be read
from or written to package directories. Reading from packages makes
sense in many cases, but I really dislike the idea of storing
per-device calibration data in a package directory. Packages are
source code. They should be sharable read-only.

There are reasons for roslaunch to take this approach, of course.
Launch files should be nest-able, so the system can be composed of
modular subcomponents. And, nodes may be launched on multiple
machines. Cameras are especially good candidates for that, providing
parallel processing for compute-intensive operations.

> We'll get to it eventually, but if you write something like this up
> before then, I can take care of pushing it into the right place.

My idea was to use the parameter server for this. Persistent
calibration then becomes a matter of using "rosparam dump" and
"rosparam load" to save and restore the data. This is useful for much
more than just cameras. Many servo devices require calibration. It
could also be helpful for parameters tuned via machine learning.

This approach works cleanly and transparently in a distributed system
environment.

Nothing in CameraInfo is too high-bandwidth for the parameter server
to become a bottleneck. The camera driver would simply read its
parameters as usual. If the CameraInfo is not available it publishes a
default setting. (Rviz and other tools should handle that correctly --
it's not an error, just a new, uncalibrated device.) When calibration
runs, it calls SetCameraInfo. The current instance of the driver
stores the data, also writing them back to the parameter server.

I like this approach better than the file manager you suggested. But,
perhaps there are issues I have not considered. Further suggestions
and comments are welcome.
-- 
 joq