[ros-users] ROS & DDS

Brian Gerkey gerkey at osrfoundation.org
Tue Feb 18 03:14:49 UTC 2014

hi Matt,

Thanks for the detailed report of your experience with DDS and ROS.

I have to say that I'm a little surprised by your experience, as it
seems to run contrary to what I know about ROS combined with what I've
read about DDS.   As we experiment, we'll pay close attention to the
issues you raised (discovery responsiveness, large message
transmission, and reliability over wifi).


On Mon, Feb 17, 2014 at 5:21 PM, Matthew Thompson <matthewbot at gmail.com> wrote:
> I'm a little late to this too! I have some experience using DDS on a
> robotics project, which had grown its own framework of sorts embodying some
> of the ideas previously discussed. The DDS API is verbose and complex, and
> binding to even the portion we used to Python resulted in some pretty
> complex code. We combated API complexity to some extent by abstracting over
> it, but when things went wrong and all messages were silently being dropped
> we were exposed to DDS proper. Once thoroughly debugged, the abstraction
> layer was helpful, but we never did completely resolve every issue before we
> gave up and ported all of our code from our custom framework to ROS. This
> took less time than fully developing the DDS abstraction layer, which needed
> a large second revision after lessons learned from the first.
> The issues we had remaining may have a been a problem with our specific DDS
> implementation, RTI DDS, or more likely bugs that persisted in our
> abstraction layer or even our own DDS understanding after almost two years
> of using it. I can only remember a few, but a large one was that messages
> above a certain size (10s of kilobytes IIRC) were never received. Another
> was that peer finding was sporadically inconsistent in the face of changes
> to network topology, or at least occasionally did not become consistent
> quickly enough for our uses. Restarting one side would cause things to fully
> reconnect. We also found DDS to be unreliable over even a strong WiFi
> signal; switching to ROS instantly improved this use case, though it is
> still far from ideal. We were also not impressed with the debugging tools
> available on Linux. I'm listing these because I don't mean to spread FUD
> about DDS. My experience is that after spending a reasonable amount of time
> using it, somewhere in our stack we still had fairly serious problems that
> we did not encounter using ROS.
> I would be pretty disappointed if ROS began using DDS underneath. While I
> think it is likely to work given sufficient effort, I'm don't see how it
> could would any better than simply improving the existing ROS
> communications. All ROS really needs is to incorporate a simple reliable UDP
> layer. It definitely does not need to adopt DDS's vast capabilities, which
> include a bunch of subtle variations on pub-sub crossed with a database, but
> no RPC whatsoever. Even if performance is inferior to DDS, it will be much
> simpler, and IMO that simplicity will lead to a higher reliability than DDS.
> This is not because DDS is unreliable, but because it is complex, and humans
> become unreliable in the face of complexity.
> If ROS was looking to use a new message format, I'd suggest Cap'n Proto, a
> descendant of protocol buffers with an eye for performance and built-in RPC,
> and already has a substantial OSS community with bindings to a ton of
> languages. IMO, a simple reliable UDP layer and Cap'n Proto would likely
> outperform DDS, in addition to being simpler and better suited for ROS.
> Matt Thompson
> On Mon, Feb 17, 2014 at 6:19 PM, <rdean at gdrs.com> wrote:
>> Hello all,
>> Late to the conversation, I was integrating robots in linux land last week
>> and only get email in Windows land after I reboot.
>> Couple of points to make here:
>> 0) I do not use DDS, but have studied it.
>> 1) DDS is PubSub. It has a default set of configuration options, and uses
>> discovery to find topics. No rosmaster required.
>>     Therefore, unless you need to optimize your comms (capability which
>> ROS does not have), it should not be necessary to configure QoS or use the
>> XML configuration system.
>> 2) It should be easy to write a script to convert .msg into DDS IDL as
>> part of the build process.
>> 3) Putting DDS underneath the node_handle interface should be
>> straightforward. I put a shared memory transport under node_handle once,
>> which was fine up until I discovered that...
>> 4) there are currently additional layers on top of node_handle which have
>> questionable design, and are implemented by reaching inside the node_handle,
>> breaking various design pattern concepts.
>> 5) DDS itself and other standards such as JAUS are designed around wire
>> level interoperability. This is a great design decision. After running into
>> the hidden required layers, I dropped the compatibility interface and
>> implemented my own comm plugin to our code. Took about a week. ROS
>> interoperability without the dependencies.
>> I get the impression from this thread that people seem to have forgotten
>> that Robotics is HARD. A good tool simplifies the issue, but only so far.
>> Use of ROS, for example, does not negate the need to understand TCP/UDP. At
>> some point you will need to be able to diagnose why component X is failing
>> and dig out wireshark.
>> What will give you real headaches moving forward is this: there are parts
>> of the ROS comm stack which should be broken and/or replaced. Some of this
>> is due to the natural evolution of systems, others due to implementers with
>> a lot of heart, but less experience with comm architectures. This happens
>> all the time in software development.
>> There is also the question of development cost. It would probably be much
>> cheaper to move to a solution such as DDS, break/fix the ROS apis, and fix
>> the various packages than it would be to add a custom QoS to ROS.
>> Thank you,
>> Robert Dean
>> RCTA Program, Intelligence Task Coordinator
>> (http://www.arl.army.mil/www/default.cfm?page=392)
>> Open Architecture Technical Lead
>> General Dynamics Robotic Systems
>> (410) 876-9200 x1768
>> (240) 418-3634 (cell)
>> -----Original Message-----
>> From: ros-users-bounces at lists.ros.org
>> [mailto:ros-users-bounces at lists.ros.org] On Behalf Of Brian Gerkey
>> Sent: Sunday, February 16, 2014 1:36 PM
>> To: User discussions
>> Subject: Re: [ros-users] ROS & DDS
>> This is a great thread!  I've missed these passionate discussions.
>> And all it took was asking a seemingly simple question.
>> Here's my view of the consensus so far (I'm trying not to draw any
>> conclusions; just restating what's been said):
>> * Good: Improving the capabilities of the ROS middleware, such as adding
>> QoS features.  The ROS middleware is nice, but QoS and reliability in
>> general are notably absent.
>> * Good: Relying on a high-quality existing middleware, perhaps DDS.
>> ZeroMQ also came up, but with the caveat that it only does the
>> socket-handling, so you have to add your own discovery and marshaling, as
>> well as QoS, if you care about that.
>> * Bad: Making ROS code harder to write (e.g., by forcing everyone to use
>> the full DDS API, which is quite verbose).  For the common case, the code
>> should be as simple as it is now in ROS.
>> * Bad: Making ROS systems harder to configure (e.g., by forcing everyone
>> to become an expert in the DDS XML configuration system).
>> For the common case, everything should just work out of the box.
>> * Bad: Forcing everyone to make a lot of changes to their existing ROS
>> code to adapt to updates in ROS, even when they don't care about the new
>> features brought about by those updates (e.g., roslisp code, which doesn't
>> care about QoS, should just keep working, no matter what).
>> Whatever changes are made, the transition path must be easy, including the
>> option to not make the transition but keep using ROS.
>> Other proposals have been made, including:
>> * simplify ROS middleware to speak HTTP; and
>> * extend ROS middleware by adding transports, as has been done for images
>> and by Cedric in the ethzasl_message_transport package.
>> It has been suggested that DDS, like CORBA before it, is overly (perhaps
>> pointlessly) complex, and the fact that we would need to hide its complexity
>> from most users should give us pause.  On the other hand, it's also been
>> suggested that it's good practice to hide the complexity and details of any
>> middleware, to allow for a separation of concerns.
>> brian.
>> On Sun, Feb 16, 2014 at 8:02 AM, Ingo Lütkebohle <iluetkeb at gmail.com>
>> wrote:
>> > Hi Christian,
>> >
>> > I general agree with your suggestions. In particular, I think it is
>> > definitely possible to present a user interface that the ROS community
>> > will like, or even the same as currently, based on an established
>> > middleware package. I am *also* positive that doing so won't be easy,
>> > but it can be done.
>> >
>> > That said, I think there is one other concern that people have: That
>> > the underlying platform, even if well encapsulated, might still cause
>> > issues. This is, in my opinion, a serious and well-founded concern
>> > supported by lots of evidence in that past! If there is a lot of
>> > complexity beneath that implements all sorts of features, this will
>> > typically also affect the somple code-paths. If nothing else, more
>> > code usually means more bugs, and if there is a bug, encapsulation
>> > usually helps not at all.
>> >
>> > Moreover, I would like to mention one thing that is routinely
>> > forgotten by people working with high-end middleware: Installation is
>> > often not trivial at all! Many of the DDS packages mentioned here are
>> > not available as part of a standard Linux distribution, and supporting
>> > them through packages might have licensing issues. For example, the
>> > OpenSplice Community Edition is LGPL, not BSD.
>> >
>> > I'm am sure all of these concerns can be addressed, but only if we
>> > look at them, instead of focusing on API issues only!
>> >
>> > Cheers,
>> >
>> > On Sun, Feb 16, 2014 at 9:16 AM, Christian Schlegel <schlegel at hs-ulm.de>
>> > wrote:
>> >> Hi,
>> >>
>> >> "freedom of choice" is what general purpose tools like CORBA etc.
>> >> provide. They give no guidance which parts to use in what situation. They
>> >> require far too much knowledge from standard users but provide all you need
>> >> for experts (framework builder, all kinds of domains, etc., all kinds of
>> >> interoperable implementations of the standards).  You never ever should
>> >> expose these to the domain experts that is the robotics experts.
>> >>
>> >> "freedom from choice" is what we provide by a domain-specific
>> >> presentation of those artefacts along with their proper configuration. We
>> >> present communication patterns with an agreed semantics and all the mapping
>> >> onto standard middleware is hidden (including which CORBA, DDS, ACE, zeroMQ
>> >> etc. concepts are selected and how they are configured). Thus, the
>> >> abstraction is NOT presenting everything of DDS and/or CORBA in a simplified
>> >> way but in a domain-specific way: such how robotics people want to have
>> >> their presentation.
>> >>
>> >> Exactly this helped us to seamlessly migrate from CORBA to ACE without
>> >> exposing anything of that to the robotics user!
>> >>
>> >> Christian
>> >>
>> >> ---
>> >> Prof. Dr. Christian Schlegel
>> >> Prodekan, Studiendekan Master IS
>> >> Fakultät Informatik
>> >> Hochschule Ulm
>> >>
>> >> Tel.: 0731 / 50-28242
>> >>
>> >> http://www.hs-ulm.de/schlegel
>> >> http://www.zafh-servicerobotik.de/
>> >> http://www.youtube.com/user/roboticsathsulm
>> >> http://smart-robotics.sourceforge.net/
>> >> http://www.joser.org/
>> >>
>> >>> Am 16.02.2014 um 00:19 schrieb "Michael Zillich"
>> >>> <zillich at acin.tuwien.ac.at>:
>> >>>
>> >>> Hi,
>> >>>
>> >>> my two cents regarding this discussion come from experiences with
>> >>> another "industry-standard" middleware, that was somewhat ...
>> >>> complex internally and thus typically was hidden under layers of
>> >>> convenience
>> >>> code: the abominable CORBA.
>> >>> (I ended up writing my own middleware after CORBA nearly killed a
>> >>> project)
>> >>>
>> >>> I have to admit I know nothing about DDS, so forgive me if I
>> >>> unjustly critizise it. But reading in some of the ongoing
>> >>> discussions that its complexity can easily be handled by some
>> >>> abstraction layer .. that rang an alarm bell.
>> >>>
>> >>> If something is too complex to use for the intended target audience
>> >>> of developers, and thus has to be hidden behind another layer
>> >>> (which, trust me, inevitably leads to the most hilarious bugs you
>> >>> can possibly
>> >>> imagine) then it is probably the wrong technology for that
>> >>> application area. (But there are certainly other appliction areas
>> >>> where this middleware is the perfect choice).
>> >>>
>> >>> So my personal choice is always a clear and simple solution targeted
>> >>> at a specific application area (with specific requirements,
>> >>> problems, and development procedures and cycles) rather than a
>> >>> solves-all-and-everyones-problems solution developled by large
>> >>> committees.
>> >>>
>> >>> cheers,
>> >>> Michael
>> >>>
>> >>> p.s. For those interested in the CORBA story read the enlightening
>> >>> article by Michi Henning
>> >>>   https://queue.acm.org/detail.cfm?id=1142044
>> >>> who as one of the authors of "Advanced CORBA Programming with C++"
>> >>> is probably one of the only two persons on the planet, who actually
>> >>> understood CORBA (the other person being Steve Vinoski, the other
>> >>> author of the book).
>> >>>
>> >>> --
>> >>> Dr. Michael Zillich
>> >>> ACIN Institute of Automation and Control Vienna University of
>> >>> Technology (DVR-Number 0005886) Gusshausstr 27-29/E376, 1040 Vienna,
>> >>> Austria
>> >>> zillich at acin.tuwien.ac.at    http://users.acin.tuwien.ac.at/mzillich
>> >>> Tel: +43 1 58801 376648        Fax: +43 1 58801 37698
>> >>> _______________________________________________
>> >>> ros-users mailing list
>> >>> ros-users at lists.ros.org
>> >>> http://lists.ros.org/mailman/listinfo/ros-users
>> >>>
>> >> _______________________________________________
>> >> ros-users mailing list
>> >> ros-users at lists.ros.org
>> >> http://lists.ros.org/mailman/listinfo/ros-users
>> >
>> >
>> >
>> > --
>> > Ingo Lütkebohle, Dr.-Ing.
>> > Machine Learning and Robotics Lab, IPVS, Universität Stuttgart
>> > http://www.ipvs.uni-stuttgart.de/abteilungen/mlr/abteilung/mitarbeiter
>> > /Ingo.Luetkebohle
>> > +49-711-685-88350
>> >
>> > PGP Fingerprint 3187 4DEC 47E6 1B1E 6F4F  57D4 CD90 C164 34AD CE5B
>> > _______________________________________________
>> > ros-users mailing list
>> > ros-users at lists.ros.org
>> > http://lists.ros.org/mailman/listinfo/ros-users
>> _______________________________________________
>> ros-users mailing list
>> ros-users at lists.ros.org
>> http://lists.ros.org/mailman/listinfo/ros-users
>> -------------------------------------------------------
>> This is an e-mail from General Dynamics Robotic Systems. It is for the
>> intended recipient only and may contain confidential and privileged
>> information. No one else may read, print, store, copy, forward or act in
>> reliance on it or its attachments. If you are not the intended recipient,
>> please return this message to the sender and delete the message and any
>> attachments from your computer. Your cooperation is appreciated.
>> _______________________________________________
>> ros-users mailing list
>> ros-users at lists.ros.org
>> http://lists.ros.org/mailman/listinfo/ros-users
> _______________________________________________
> ros-users mailing list
> ros-users at lists.ros.org
> http://lists.ros.org/mailman/listinfo/ros-users

More information about the ros-users mailing list