Hi Joan,I was not aware of that list. Thanks!
Thanks for the contribution, and sorry for the delayed response. I've only now found time to digest everything. If you're interested in the image_pipeline, you should consider joining the Perception Pipelines SIG, or at least its mailing list.
I created a ticket to track your code and suggestions: https://code.ros.org/trac/ros-pkg/ticket/5201. Further comments below.If it is better to discuss it there, let me know.
I am aware of that and agree on the lost of information, but that it is what decimate means. I mean that from my point of view it is an issue of keeping concepts separated. A 2x2 rude subsampling of a Bayer image into an RGB image is a very particular case of image processing, not a decimate process. And that no interpolation is needed whatsoever is arguable.
If the input image encoding is bayer, this nodelet performs crop/binning and debayering.
We would like to discuss with the ROS community if it would be more convenient perform only the crop/binning, and do NOT perform the debayering. That debayering process, with possibly other image_proc tasks, could be performed by another image_proc node in the subsampled image topic namespace. From our point of view, it will have two main advantages:
- It will allow more complex debayering methods to be aplied to the scaled images by image_proc, with zero mantainment cost of the crop/binning nodelet.
I have problems with this. If you're doing 2x2 decimation of a Bayer image, you have all the information to transform a 2x2 Bayer cell into one RGB pixel right there. The "debayering" is trivial, no interpolation needed whatsoever, and you lose very little information (just from averaging the 2 green pixels in each cell).
With your approach, you throw away 3/4ths of the information up front, and hope that a complex (hence expensive) debayering algorithm can fill some of that back in. You burn a lot more CPU and, no matter how sophisticated your algorithm, you won't get results as good as the trivial downsample-to-RGB approach.
The scaling question in our case arose from the fact that we could not use compression to send images through the network. Our camera was 10 frames/sec bayer, the bandwidth allowed only 7 frames/sec at full resolution without compression, and only 3 frames/sec with compression. We supposed that this was due to the limited power of the CPU compared to the demands of the compression method.
- It will allow subsampling bayer images to reduce its size and send them through the network without losing the color information (if sended in mono) nor triplicating the required bandwith (if sended in color)
True, full RGB takes 3x the bandwidth of Bayer. Then again, imposing a Bayer pattern is a pretty simplistic compression method. We have image_transport for applications where bandwidth is limited.
I agree completely. The discussion on that follows from the original post mail in the next lines:The subsampling is quite easy to implement. It just changes depending on the encoding. Subsampling of mono and color encoded images is quite intuitive. In Bayer encodings, the idea is to consider each set of 2x2 pixels as a color cell, and these color cells are the subsampling units. The resulting image has the same encoding than the original one.
As implemented using decimation, you'll get bad aliasing artifacts when downsampling Bayer images, even in the 2x2 case where you have the information to do it perfectly. For each set of 4 rows/columns you throw out 2 adjacent ones.
Cameras that support "color binning" in hardware do something like this, but using binning/subsampling (averaging blocks of Bayer cells) instead of decimation (throwing away all but one). In that case the aliasing is mitigated.
It should be said that this subsampling is the crudest one, and it can introduce artifacts in the resulting image. Another completely different approach will be to use OpenCV resizing functions to do the job. In that case, as in the debayering process, a set of methods could be supported with an extra dynamic_reconfigure parameter.
That's true. The current decimation approach is basically equivalent to cv::resize using the INTER_NEAREST (nearest-neighbor) method, which is fast but indeed crude. We could get much better output from INTER_AREA, which gives Moire-free results. A parameter to switch between at least those two options would be useful.
I think that last OpenCV versions only understand images as bgr8 encoded cv::Mat's (even mono images), so the approach will be:
And it won't be possible to scale a bayer image, debayer it first will be mandatory. Is there any OpenCV expert that can confirm or refutate these statements? Any information about the resizing functions will be appreciated, since the documentation is quite poor.
- subscribe to the color image
- subsample it to a color scaled image
- from this color scaled image build the mono scaled image if necessary
The very latest documentation for cv::resize is here and seems adequate to me. I believe it handles one-channel and three-channel 8- or 16-bit images just fine.
For Bayer, I would still do the 2x2 decimation to RGB (almost lossless), and then use cv::resize for any further decimation.
-- Joan Pau Beltran Grup de Sistemes, Robòtica i Visió - DMI Universitat de les Illes Balears Ctra. Valldemossa, km 7.5 07122 Palma Campus Universitari, edifici Anselm Turmeda Telf (+34) 971 17 28 13 Fax (+34) 971 17 30 03