[ros-users] Reinventing the wheel: training and evaluation of algorithms for objects detection in images

Sun Nov 21 22:10:01 UTC 2010

> In parallel, we have started other initiatives such as the series of "Solved Problems in Perception" workshops, and our colleague Gary
> Bradski will soon announce an even bigger initiative that includes proper challenges, with money prizes, etc.

The "Solutions In Perception Challenge" seems indeed as an interesting
initiative. Let us see how that goes.

> We completely agree with your assessment, and have thus already started the investigation of an infrastructure for object detection
> evaluation. The infrastructure started this past summer and is called ReIn (Recognition Infrastructure - http://www.ros.org/wiki/rein).
> ReIn mixes and matches several detectors, filters, and evaluators. ReIn is based on OpenCV and PCL, and is tightly integrated with
> ROS. The developing efforts are led by Marius Muja, the author of FLANN, and a student in David Lowe's group at UBC.
>  We currently have many people contributing to the project, though we need to accelerate and come up with a formal review process for > including ReIn in Diamondback.

Yes indeed I have seen Rein and the other modules in the
object_recognition stack. These seem all to be "online oriented". Rein
defines some training methods, but they seem oriented towards online
training.

( included it in the "Similar initiatives and project" of my initial
draft https://wave.google.com/wave/waveref/googlewave.com/w+yH-HOCb6H
)

What I am aiming for is to have some kind on standard architecture for
the offline training stage (and evaluation). This part seems not to be
covered by any ROS package I am aware of.

It seems that Caroline Pantofaru is somewhat working on the
"exchanging datasets and providing easy evaluation" part of the
project with the "Moving People, Moving Platform Dataset" initiative,
but very little of it is visible right now.

> Together with your help and the help of the community, we might be able to provide a solution for the evaluation of detectors for object > recognition.

I'm looking for the best way of creating this tool to make most
useful. ROS integration would clearly be a nice to have, but as
mentioned, I am worried it may be "too much" for non-robotics users. I
guess this will depend on the feedback I get.

Inputs on the questionnaire are welcome ;P
https://spreadsheets.google.com/viewform?formkey=dFFzaDlLM1liVGNOS2FENnhrc1VWckE6MQ

Thanks Radu for your comments and let us hope we get to a "good
enough" solution.
Regards,
rodrigob.

On Sun, Nov 21, 2010 at 10:39 PM, Radu Bogdan Rusu
<rusu at willowgarage.com> wrote:
>
> Rodrigo,
>
> Thanks for sharing your ideas with the community. We completely agree with your assessment, and have thus already started the investigation of an infrastructure for object detection evaluation. The infrastructure started this past summer and is called ReIn (Recognition Infrastructure - http://www.ros.org/wiki/rein). ReIn mixes and matches several detectors, filters, and evaluators. ReIn is based on OpenCV and PCL, and is tightly integrated with ROS. The developing efforts are led by Marius Muja, the author of FLANN, and a student in David Lowe's group at UBC. We currently have many people contributing to the project, though we need to accelerate and come up with a formal review process for including ReIn in Diamondback.
>
> Together with your help and the help of the community, we might be able to provide a solution for the evaluation of detectors for object recognition.
>
> In parallel, we have started other initiatives such as the series of "Solved Problems in Perception" workshops, and our colleague Gary Bradski will soon announce an even bigger initiative that includes proper challenges, with money prizes, etc. More to follow, so let's stay in touch. :)
>
> Cheers,
> Radu.
>
>
> On 11/21/2010 01:09 PM, Rodrigo Benenson wrote:
>>
>> Hello all.
>>
>>
>>      I'm contacting you because I am considering starting a new open source project to solve a specific problem:
>>      training and evaluating objects detection algorithms.
>>
>>      Hundreds of students have been there before: "I want to create a program that detects objects in images".
>>      They choose a dataset for training (e.g. INRIA pedestrians), a feature descriptor (e.g. HOG), a machine learning
>>      method (e.g. linear SVM), and then, they write the code to get it all together.
>>
>>      In the best case they will take bits and pieces from multiple places and spend a few weeks on the glue code. In
>>      the worst case they will spend months reimplementing existing methods.
>>
>>      It is time to stop the madness.
>>      Training detectors for objects detection in images is a specific and well defined problem.
>>      It is time to share our effort and build a reference open source tool to solve this common problem.
>>      We should have an open source tool that provides all the common bits, the glue and allows us to focus on what we
>>      really care: the algorithms.
>>
>>      In some sense OpenCv <http://www.google.com/url?sa=D&q=http%3A%2F%2Fopencv.willowgarage.com%2F> 2.2 helps a lot to
>>      the task, however OpenCv is aimed to be a generic library not a specific application framework. In that sense it
>>      will never provide the desired "install, run, see the graphs coming out" experience.
>>
>>      Also ROS.org <http://www.google.com/url?sa=D&q=http%3A%2F%2FROS.org> helps a lot the task, by providing a generic
>>      framework to create and exchange software modules, along with standard tools for messages passing, data storage
>>      and exploration. However this framework by itself has a non negligible learning curve and it is unfamiliar to
>>      anyone outside the robotics community.
>>
>>
>>      I currently have my own idea of how things could be. However before creating "yet one more framework" I would like
>>      to have /your/ input on the topic.
>>
>>      I have created a short form to collect your opinions. I would be very glad if you could help me go in the right
>>      direction by giving your input.
>>
>>      https://spreadsheets.google.com/viewform?formkey=dFFzaDlLM1liVGNOS2FENnhrc1VWckE6MQ
>>      <http://www.google.com/url?sa=D&q=https%3A%2F%2Fspreadsheets.google.com%2Fviewform%3Fformkey%3DdFFzaDlLM1liVGNOS2FENnhrc1VWckE6MQ>
>>
>>      The form is anonymous and the results are public.
>>
>>      Based your opinions and ideas I will do my best to move forward an usable open source solution.
>>      Further information will be posted at
>>      https://wave.google.com/wave/waveref/googlewave.com/w+yH-HOCb6H
>>      <http://www.google.com/url?sa=D&q=https%3A%2F%2Fwave.google.com%2Fwave%2Fwaveref%2Fgooglewave.com%2Fw%2ByH-HOCb6H>
>>
>>      Best regards,
>>      rodrigo benenson phd.
>>
>>      ps: If you are interested do not hesitate to send me a message. You can contact me via github as "rodrigob".
>>
>>
>>
>>
>> _______________________________________________
>> ros-users mailing list
>> ros-users at code.ros.org
>> https://code.ros.org/mailman/listinfo/ros-users