[ros-users] Sync problem with stereo_image_proc
Radu Bogdan Rusu
rusu at willowgarage.com
Wed May 26 18:03:38 UTC 2010
Dear Sacha,
Since this seems to be a pure software calibration issue, I will let James answer, as he is the author of that package.
One thing I didn't understand is the cameras zooming part :) What do you mean by that?
The 7cm baseline is fine.
Cheers,
Radu.
On 05/25/2010 09:15 AM, Sacha Aury wrote:
> Dear Radu,
>
> I tried again today to calibrate my cameras, and I can bring a few more
> information about what happens :
>
> I updated to the latest (unstable) ROS version, to check the python
> error during the calibration : no change, it seems impossible to
> calibrate with a 320*240 resolution.
>
> I tried to calibrate with a 640*480 resolution, which implies 15 fps.
> There is no crash, just a lot of small freezes (turning grey on ubuntu,
> meaning the process does not answer for a while). But all the values are
> just terrible, with an epi of 5-8 and a size of 0.110 (should be ~0.02).
> Moreover, after clicking on "calibrate", the camera are zooming, which I
> think should not happen (looking to the tutorial).
>
> I am getting lost with all these issues. Could the offset between my
> cameras, which is about 7cm, be too much / not enough?
>
> Thanks
>
> Cheers,
> Sacha
>
> Le lundi 24 mai 2010 à 08:36 -0700, Radu Bogdan Rusu a écrit :
>> Dear Sacha,
>>
>> We need to wait for James (the author of that calibration package) to reply. I see the point about getting some better
>> links to calibration from the stereo_image_proc page. We'll try to add that later today (James, do you see any problem
>> with that?).
>>
>> Cheers,
>> Radu.
>>
>> On 05/24/2010 07:42 AM, Sacha Aury wrote:
>>> Dear Radu,
>>>
>>> I am trying to calibrate my cameras right now. I didn't do it before,
>>> because I didn't find the way to do it on the step-by-step tutorial. It
>>> could be good to add a link somewhere in
>>> http://www.ros.org/wiki/stereo_image_proc to tell it has to be done.
>>> I try to do the calibration using 320*340 resolution and 30 fps.
>>> Everything is fine until I click on "calibrate", then the python script
>>> fails :
>>>
>>> epipolar error: 0.857935751957
>>> OpenCV Error: Assertion failed (src.type() == dst.type()&& dst.size()
>>> == mapx.size()) in cvRemap,
>>> file /opt/ros/boxturtle/stacks/vision_opencv/opencv2/build/opencv-svn/src/cv/cvimgwarp.cpp, line 3083
>>> Exception in thread Thread-9:
>>> Traceback (most recent call last):
>>> File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
>>> self.run()
>>> File
>>> "/opt/ros/boxturtle/stacks/image_pipeline_copy/camera_calibration/nodes/cameracalibrator.py", line 59, in run
>>> self.function(m)
>>> File
>>> "/opt/ros/boxturtle/stacks/image_pipeline_copy/camera_calibration/nodes/cameracalibrator.py", line 233, in handle_stereo
>>> lscrib = self.c.lremap(lrgb)
>>> File
>>> "/opt/ros/boxturtle/stacks/image_pipeline_copy/camera_calibration/src/camera_calibration/calibrator.py", line 407, in lremap
>>> cv.Remap(src, r, self.lmapx, self.lmapy)
>>> error: src.type() == dst.type()&& dst.size() == mapx.size()
>>>
>>> I tried to do it using 15fps/640*480, but the cams are not very usable
>>> at this rate. The calibration seems to work (at least, not crashing)
>>> like this, but the params K and P are still infinite...
>>>
>>> Cheers,
>>> Sacha
>>> Le vendredi 21 mai 2010 à 09:20 -0700, Radu Bogdan Rusu a écrit :
>>>> Dear Sacha,
>>>>
>>>> Thanks for the uploading the data. I had a look at the bag file that you recorded, and as far as I can tell, it looks
>>>> like the camera_info is screwed up:
>>>>
>>>> header:
>>>> seq: 2145
>>>> stamp: 2144000000000
>>>> frame_id: fixed_frame
>>>> height: 240
>>>> width: 320
>>>> roi:
>>>> x_offset: 100
>>>> y_offset: 0
>>>> height: 240
>>>> width: 320
>>>> D: [0.0, 0.0, 0.0, 0.0, 0.0]
>>>> K: [inf, 0.0, 159.5, 0.0, inf, 119.5, 0.0, 0.0, 1.0]
>>>> R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
>>>> P: [inf, 0.0, 159.5, 0.0, 0.0, inf, 119.5, 0.0, 0.0, 0.0, 1.0, 0.0]
>>>>
>>>> Did you calibrate the cameras using the instructions on the wiki? Having inf values in K and P is bad :)
>>>>
>>>> Cheers,
>>>> Radu.
>>>>
>>>> On 05/21/2010 09:15 AM, Sacha Aury wrote:
>>>>> Dear Radu
>>>>>
>>>>> Thanks for your help. Since my connection at work is not really fast, I
>>>>> will send the bag on http://fly.yozora-irc.net/shadow tonight. You can
>>>>> already see my disparity map and both left / right images at this place.
>>>>>
>>>>> Cheers,
>>>>> Sacha
>>>>>
>>>>> Le vendredi 21 mai 2010 à 08:55 -0700, Radu Bogdan Rusu a écrit :
>>>>>> Dear Sacha,
>>>>>>
>>>>>> Can you assemble a bag file with the following topics? Please replace "mystereo" with the name of your stereo camera:
>>>>>> /mystereo/left/image_raw
>>>>>> /mystereo/left/camera_info
>>>>>> /mystereo/right/image_raw
>>>>>> /mystereo/right/camera_info
>>>>>>
>>>>>> and upload it somewhere where I can access it?
>>>>>>
>>>>>> In the meantime, can you make a screenshot of the disparity image? Just visualize it using image_view, and save it to a
>>>>>> jpg/png.
>>>>>>
>>>>>> Thanks,
>>>>>> Radu.
>>>>>>
>>>>>> On 05/21/2010 02:41 AM, Sacha Aury wrote:
>>>>>>> Dear Radu,
>>>>>>>
>>>>>>> Even if I understand that having some not valid disparity values is
>>>>>>> possible, I actually do not have any good ones. Moreover, all my points
>>>>>>> are quite the same (0.0, 0.0, nan) which is a bit strange I think. What
>>>>>>> could I do to fix this ? I already tried to change parameters with
>>>>>>> dynamic_reconfigure, but it did not change anything.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Sacha
>>>>>>>
>>>>>>> Le jeudi 20 mai 2010 à 10:45 -0700, Radu Bogdan Rusu a écrit :
>>>>>>>> Dear Sacha,
>>>>>>>>
>>>>>>>> What you're seeing makes sense. Though your disparity "looks good", I am sure you don't have valid disparity values at
>>>>>>>> each pixel in the disparity image. None of us do :). What you are seeing as the output of rostopic echo is the list of
>>>>>>>> 3D points that do not have valid disparity values (we mark those with nan).
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Radu.
>>>>>>>>
>>>>>>>> On 05/20/2010 09:37 AM, Sacha Aury wrote:
>>>>>>>>> I do not really understand why, but I looked to the TF tutorial, added a
>>>>>>>>> few things to my program and rviz do not complain anymore. To recap, I
>>>>>>>>> now have a disparity map, rviz do not complain about a fixed_frame, but
>>>>>>>>> my goal is to get the points cloud. Rviz warns :
>>>>>>>>> [ WARN] 1274372115.122808000: TF_OLD_DATA ignoring data from the past
>>>>>>>>> for frame /test at time 16566 according to authority /camera_sync
>>>>>>>>> Possible reasons are listed at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> $rostopic echo /stereo/points | less :
>>>>>>>>> header:
>>>>>>>>> seq: 937
>>>>>>>>> stamp: 18596000000000
>>>>>>>>> frame_id: fixed_frame
>>>>>>>>> points: [
>>>>>>>>> x: 0.0
>>>>>>>>> y: 0.0
>>>>>>>>> z: nan,
>>>>>>>>> x: 0.0
>>>>>>>>> y: 0.0
>>>>>>>>> z: nan,
>>>>>>>>> x: 0.0
>>>>>>>>> y: 0.0
>>>>>>>>> z: nan,
>>>>>>>>> x: 0.0
>>>>>>>>> y: 0.0
>>>>>>>>> z: nan,
>>>>>>>>> x: 0.0
>>>>>>>>> y: 0.0
>>>>>>>>> z: nan,
>>>>>>>>> x: 0.0
>>>>>>>>>
>>>>>>>>> I don't get it, since my disparity map looks good. I can send you a
>>>>>>>>> screenshot of the map if you need it.
>>>>>>>>> So, why is the processing not sending the good coords ? I did not
>>>>>>>>> calibrate my cameras since I did not find a way to do it for stereo
>>>>>>>>> vision, is there a problem with that ?
>>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> Le mercredi 19 mai 2010 à 10:38 -0700, Josh Faust a écrit :
>>>>>>>>>> If you don't have a TF transform tree (http://ros.org/wiki/tf), the
>>>>>>>>>> fixed frame needs to match the frame you've set. You also need to set
>>>>>>>>>> the timestamp in the header.
>>>>>>>>>>
>>>>>>>>>> msg.header.stamp = ros::Time::now();
>>>>>>>>>>
>>>>>>>>>> Josh
>>>>>>>>>>
>>>>>>>>>> On Wed, May 19, 2010 at 7:53 AM, Sacha Aury<sacha at shadowrobot.com>
>>>>>>>>>> wrote:
>>>>>>>>>> Thanks for your help. I now have a single node synchronizing
>>>>>>>>>> my two
>>>>>>>>>> cameras (image and infos). It works with the stereo_view and I
>>>>>>>>>> get the
>>>>>>>>>> disparity map, which I am configuring. My question is now
>>>>>>>>>> about rviz,
>>>>>>>>>> which I've never used. It seems to need a fixed frame to
>>>>>>>>>> display my 3D
>>>>>>>>>> cloud points. I tried to set a frame_id in my synchronization
>>>>>>>>>> node
>>>>>>>>>> (before broadcasting it to stereo_image_proc), but it does not
>>>>>>>>>> work,
>>>>>>>>>> rviz still can't get any fixed frame.
>>>>>>>>>>
>>>>>>>>>> How can I do that ? I think I am close to the result, but I
>>>>>>>>>> can't see
>>>>>>>>>> it !
>>>>>>>>>>
>>>>>>>>>> Le mardi 18 mai 2010 à 09:07 -0700, Blaise Gassend a écrit :
>>>>>>>>>>
>>>>>>>>>> > Hi Sacha,
>>>>>>>>>> >
>>>>>>>>>> > stereo_image_proc doesn't care about when your message was
>>>>>>>>>> published, it
>>>>>>>>>> > just cares about the timestamp in the message. So your
>>>>>>>>>> republishing node
>>>>>>>>>> > needs to set the timestamp, both in the image topic and in
>>>>>>>>>> the
>>>>>>>>>> > camera_info topic.
>>>>>>>>>> >
>>>>>>>>>> > Blaise
>>>>>>>>>> >
>>>>>>>>>> > On Tue, 2010-05-18 at 16:29 +0100, Sacha Aury wrote:
>>>>>>>>>> > > I think that having a hardware solution is a bit overkill
>>>>>>>>>> for my
>>>>>>>>>> > > problem. So I tried to write a node which collected image
>>>>>>>>>> messages from
>>>>>>>>>> > > my two cameras and which published both message every x
>>>>>>>>>> seconds,
>>>>>>>>>> > > typically 1/2. And when I am using
>>>>>>>>>> image_pipeline/stereo_image_proc, I
>>>>>>>>>> > > got messages of that kind :
>>>>>>>>>> > >
>>>>>>>>>> > > [ WARN] 1274196325.866264000: [stereo_image_proc] Low
>>>>>>>>>> number of
>>>>>>>>>> > > synchronized left/right image pairs received.
>>>>>>>>>> > > Left images received: 831
>>>>>>>>>> > > Right images received: 831
>>>>>>>>>> > > Synchronized pairs: 0
>>>>>>>>>> > > Possible issues:
>>>>>>>>>> > > * The cameras are not synchronized.
>>>>>>>>>> > > * The network is too slow. For each synchronized image
>>>>>>>>>> pair, at most 1
>>>>>>>>>> > > is getting through.
>>>>>>>>>> > >
>>>>>>>>>> > > I don't know what I could do more, since the
>>>>>>>>>> synchronisation should be
>>>>>>>>>> > > done by a TimeSynchronizer in the stereo_image_proc class.
>>>>>>>>>> Could it be
>>>>>>>>>> > > posible to cheat on the timestamps in any way ?
>>>>>>>>>> > >
>>>>>>>>>> > > Thanks, and sorry for the double post, I made a mistake at
>>>>>>>>>> first.
>>>>>>>>>> > >
>>>>>>>>>> > > Le lundi 17 mai 2010 à 07:53 -0700, Gary Bradski a écrit :
>>>>>>>>>> > > > What level do you want synchronization on? Within
>>>>>>>>>> framerate (both
>>>>>>>>>> > > > frames happen randomly but within 1/30 of a second of
>>>>>>>>>> each other) or
>>>>>>>>>> > > > exact frame synchronization? The first is achievable,
>>>>>>>>>> the second needs
>>>>>>>>>> > > > extra hardware.
>>>>>>>>>> > > >
>>>>>>>>>> > > > At most, ROS can get you timestamps to align to, but to
>>>>>>>>>> get actual
>>>>>>>>>> > > > simultaneous frames, you'll need a triggering signal in
>>>>>>>>>> hardware such
>>>>>>>>>> > > > as a wire between 2 cameras (that allow for this). Some
>>>>>>>>>> 1394 cameras
>>>>>>>>>> > > > may allow triggering over the 1394 bus and then you'll
>>>>>>>>>> only have the
>>>>>>>>>> > > > problem of getting both buses to trigger at the same
>>>>>>>>>> time ... which
>>>>>>>>>> > > > may be approximately possible, most of the time ... on
>>>>>>>>>> average.
>>>>>>>>>> > > > Depends on the driver and the 1394 port.
>>>>>>>>>> > > >
>>>>>>>>>> > > > I've seen an automotive 1394 article (not ROS related)
>>>>>>>>>> where they have
>>>>>>>>>> > > > about 10 cameras. The cameras run the cameras at very
>>>>>>>>>> high framerates
>>>>>>>>>> > > > and then they do timestamp synchronization which allows
>>>>>>>>>> fairly
>>>>>>>>>> > > > synchronous alignment of multiple with no extra
>>>>>>>>>> hardware.
>>>>>>>>>> > > >
>>>>>>>>>> > > > Gary
>>>>>>>>>> > > >
>>>>>>>>>> > > > On Mon, May 17, 2010 at 5:23 AM, Sacha Aury
>>>>>>>>>> <sacha at shadowrobot.com>
>>>>>>>>>> > > > wrote:
>>>>>>>>>> > > > Hi,
>>>>>>>>>> > > >
>>>>>>>>>> > > > I am trying to make a stereo camera acquisition
>>>>>>>>>> using ROS with
>>>>>>>>>> > > > the
>>>>>>>>>> > > > cameradc1394 driver and the stereo_proc package.
>>>>>>>>>> I've got a
>>>>>>>>>> > > > separate
>>>>>>>>>> > > > computer to stream the image_raw from my two
>>>>>>>>>> cameras. The
>>>>>>>>>> > > > stream works,
>>>>>>>>>> > > > but when I launch stereo_image_proc, it seems
>>>>>>>>>> that my two
>>>>>>>>>> > > > cameras are
>>>>>>>>>> > > > not synchronized :
>>>>>>>>>> > > >
>>>>>>>>>> > > > [ WARN] 1274098675.074074000:
>>>>>>>>>> [stereo_image_proc] Low number
>>>>>>>>>> > > > of
>>>>>>>>>> > > > synchronized left/right image pairs received.
>>>>>>>>>> > > > Left images received: 3079
>>>>>>>>>> > > > Right images received: 3076
>>>>>>>>>> > > > Synchronized pairs: 0
>>>>>>>>>> > > > Possible issues:
>>>>>>>>>> > > > * The cameras are not synchronized.
>>>>>>>>>> > > > * The network is too slow. For each
>>>>>>>>>> synchronized image
>>>>>>>>>> > > > pair, at most 1
>>>>>>>>>> > > > is getting through.
>>>>>>>>>> > > >
>>>>>>>>>> > > > Here is my launch file :
>>>>>>>>>> > > >
>>>>>>>>>> > > > <!-- startstream.launch -->
>>>>>>>>>> > > > <launch>
>>>>>>>>>> > > > <param name="display" type="int"
>>>>>>>>>> value="0"/>
>>>>>>>>>> > > > <param name="framerate" type="double"
>>>>>>>>>> value="15"/>
>>>>>>>>>> > > > <param name="mode"
>>>>>>>>>> value="MODE_320x240_YUV422"/>
>>>>>>>>>> > > >
>>>>>>>>>> > > > <node name="camera_left" pkg="cameradc1394"
>>>>>>>>>> > > > type="cameradc1394"
>>>>>>>>>> > > > respawn="true">
>>>>>>>>>> > > > <param name="prefix" type="string"
>>>>>>>>>> > > > value="/stereo/left/"/>
>>>>>>>>>> > > > <param name="cameraindex" type="int"
>>>>>>>>>> value="0"/>
>>>>>>>>>> > > > </node>
>>>>>>>>>> > > >
>>>>>>>>>> > > >
>>>>>>>>>> > > > <node name="camera_right" pkg="cameradc1394"
>>>>>>>>>> > > > type="cameradc1394"
>>>>>>>>>> > > > respawn="true">
>>>>>>>>>> > > > <param name="prefix" type="string"
>>>>>>>>>> > > > value="/stereo/right/"/>
>>>>>>>>>> > > > <param name="cameraindex" type="int"
>>>>>>>>>> value="1"/>
>>>>>>>>>> > > > </node>
>>>>>>>>>> > > >
>>>>>>>>>> > > > </launch>
>>>>>>>>>> > > >
>>>>>>>>>> > > > Is there any specific way to make the two
>>>>>>>>>> cameras
>>>>>>>>>> > > > synchronize ?
>>>>>>>>>> > > >
>>>>>>>>>> > > > I am following that tutorial :
>>>>>>>>>> > > >
>>>>>>>>>> > > > http://www.ros.org/wiki/stereo_image_proc
>>>>>>>>>> > > >
>>>>>>>>>> > > > Thank you for any help.
>>>>>>>>>> > > >
>>>>>>>>>> > > > Sacha
>>>>>>>>>> > > >
>>>>>>>>>> > > >
>>>>>>>>>> > > > _______________________________________________
>>>>>>>>>> > > > ros-users mailing list
>>>>>>>>>> > > > ros-users at code.ros.org
>>>>>>>>>> > > > https://code.ros.org/mailman/listinfo/ros-users
>>>>>>>>>> > > >
>>>>>>>>>> > > > _______________________________________________
>>>>>>>>>> > > > ros-users mailing list
>>>>>>>>>> > > > ros-users at code.ros.org
>>>>>>>>>> > > > https://code.ros.org/mailman/listinfo/ros-users
>>>>>>>>>> > >
>>>>>>>>>> > >
>>>>>>>>>> > > _______________________________________________
>>>>>>>>>> > > ros-users mailing list
>>>>>>>>>> > > ros-users at code.ros.org
>>>>>>>>>> > > https://code.ros.org/mailman/listinfo/ros-users
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > _______________________________________________
>>>>>>>>>> > ros-users mailing list
>>>>>>>>>> > ros-users at code.ros.org
>>>>>>>>>> > https://code.ros.org/mailman/listinfo/ros-users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> ros-users mailing list
>>>>>>>>>> ros-users at code.ros.org
>>>>>>>>>> https://code.ros.org/mailman/listinfo/ros-users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> ros-users mailing list
>>>>>>>>>> ros-users at code.ros.org
>>>>>>>>>> https://code.ros.org/mailman/listinfo/ros-users
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> ros-users mailing list
>>>>>>>>> ros-users at code.ros.org
>>>>>>>>> https://code.ros.org/mailman/listinfo/ros-users
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
--
| Radu Bogdan Rusu | http://rbrusu.com/
More information about the ros-users
mailing list