> Camera subscribers are strongly encouraged to use the 
> PinholeCameraModel class from image_geometry to wrap up the details of 
> using the CameraInfo for tasks like rectification and projecting 3d 
> points to image coordinates (and vice versa). In that case their code 
> will behave correctly in the presence of binning/ROI without any extra 
> effort. ROI in particular requires tricky tweaks to the camera matrix, 
> which PinholeCameraModel does for you.
Of course. Probably a comment about it on the CameraInfo page will be nice.

> Basically, ROI makes rectification a nasty business, as the REP 
> describes. You define the ROI as a rectangle in raw image coordinates 
> (since that's what the camera driver understands), but when you 
> rectify, that rectangle gets stretched into some wavy shape in the 
> rectified image plane. So you have to choose some new rectangle (the 
> rectified ROI) to slap over it, and call that your rectified image.
That is clear from the REP.
> Without ROI, this is no big deal, as you actually choose that new 
> rectangle at calibration time. For convenience we make it the same 
> size as the raw image, and the alpha parameter controls how much of 
> that wavy shape the rectangle covers.
That is clear too.
> do_rectify selects between these two behaviors. If do_rectify is
>  * True: compute the rectified ROI from the raw ROI as described in 
> REP 104.
>  * False: just reuse the raw ROI, because we've chosen our calibration 
> so that they match up nicely.
>
Here it comes my question. If do_rectify is true, when is this rectified 
ROI computed?
Is it done inside the PinholeCameraModel class? Or is it SUPPOSED to be 
done there as suggested by the to TODO's in the API documentation:
http://www.ros.org/doc/api/image_geometry/html/c++/classimage__geometry_1_1PinholeCameraModel.html#adcf644e58628954d28a89473beb4a03c

> To be honest, nobody's been breathing down my neck wanting to make 
> serious use of ROI. I needed it a couple years ago for one specialized 
> application, later added binning and standardized the behavior in REP 
> 104, and since then it's mostly gathered cobwebs. Is this a feature 
> that's important to you? What sort of use cases do you have in mind? 
> Having actual users would do a lot to bump this up on my priority list :).

Our immediate use case is with a 1024x768 wide angle stereo camera. 
1024x768 x 2 images x 10 frames/sec is too much data for a 100Mbit/s 
network over which we need to share the images. The vision system where 
the camera is attached is used for navigation, so to allow a maximum 
overlap between consecutive frames we want the full size images (no 
ROI). However, they need to be send also to another machine that 
controls an arm for manipulation. Since they do not fit with the given 
bandwith, scaling and/or ROI are the options that we manage 
(manipulation only done when the object reaches the central region of 
the image). Compression is not an option since it consumes too much CPU 
and the compressed image frame rate falls down to 3 frames/sec. This is 
even less than what the bandwith allows for uncompressed images.

-- 
Joan Pau Beltran
Grup de Sistemes, Robòtica i Visió - DMI
Universitat de les Illes Balears
Ctra. Valldemossa, km 7.5 07122 Palma
Campus Universitari, edifici Anselm Turmeda
Telf (+34) 971 17 28 13
Fax (+34) 971 17 30 03