On Thursday 18 March 2010 04:24:30 Ken Conley wrote: > ROS 1.1 has been released. This is an **unstable** development release > to test and develop features for ROS 1.2. > > Please see the release announcement here: > > http://www.ros.org/news/2010/03/ros-11-released-unstable.html > > Change list: > > http://www.ros.org/wiki/ROS/ChangeList/1.1 > Dear roscpp developers, I apologize to mere mortal users for posting this technical mail on a user's list, but it's the only list I can post to. I was planning to write this email somewhere in april, with a patch attached, but it seems the fast pace of roscpp development has overtaken me already. The main idea of this email is this: To propose an improved message type and serialization infrastructure such that ROS, Orocos and any other robotics framework or *library* could share the same data types without hand-coding conversions. In addition, it would allow to use each other's (visualisation) tools and avoid duplication. In order to accomplish this goal, we came to the conclusion that 2 major changes were required to roscpp: 1. Remove the dependency on ROS from the data types, ie allow 3rd party data types (like KDL::Frame) be transported over ROS. 2. Provide ROS msg/srv generators to non-ROS transports or robotics frameworks. Fortunately, ROS 1.1 already implements 70% of the changes I wanted to propose, ie templated Messages and use of native types in callbacks. I fully support this change and applaud it. Great job. However, there remains some subtleties to be solved to accomplish our interoperability-goals: * ros::Time mapping The ros::Time class serves three purposes: representing a specific moment in time, reading system/simulation time and sleeping until a given time. This makes ros::Time not a header-only class, in contrast with all the other message types. This is the only message type that is treated specially. In order to use messages that contain a time field (or a Header field) in non- ROS applications, we would need a time object that only represents the specific moment in time. Call this for example std_msgs::TimeSpec, which has only two fields: uint32_t sec and uint32_t nsec, just like ros::Time. The latter can allow (implicit) conversions to and from TimeSpec in order to accomodate current practice, but std_msgs::TimeSpec would not know nor include ros::Time. This change would make all messages in common_msg available to non-ROS libraries or frameworks, since there would no longer be a library dependency on ROS. * serialization functions This has been tackled in the 1.1 release, but still some issues may remain. I liked to have ROS types to be serializable with the boost::serialization library[1] too. In principle, that library can (de)serialize and introspect any type 'Type' for which this *free* function is present: template operator&(Archive& a, Type& value) { /*...*/ } For example, serializing a struct: struct Foo { uint32_t bar, baz; }; Would only require this function present in a header file: template operator&(Archive& a, Foo& value) { uint32_t &bar = value.bar, &baz = value.baz; // line below expands to: a & nvp("bar", bar); a & BOOST_SERIALIZATION_NVP( bar ); a & BOOST_SERIALIZATION_NVP( baz ); } Without going into details, the above function allows real-time serialization/deserialization of Foo and introspection of the member names. The Archive class decides if the name ("bar") is serialized or not. An XML- like serializer would take it into account, a performant network serializer wouldn't. boost::serialization has implemented this function for the stl containers. In case of more complex types (sequences) two separate save()/load() functions can be implemented instead of operator&(). I wanted to propose to additionally support this serialization interface such that other frameworks can read and write the ROS generated types. For example, the Orocos 2.0 rt-message queues transport uses this interface to marshal/demarshal structs in real-time[2,3]. Also the Orocos scripting language will use the same facilities to introspect and modify user types. Clearly, the types would also be boost::serializable. The major advantage of this API is: - Archives have access to the *name* of the struct's element, but can ignore it for performance reasons - Since it's a templated free function, it is non intrusive and zero-overhead if not used. - It can be used next to another serialization method, like ROS's own If such additions are a no-no, it's time for plan B: * Separate to a different file the struct/type definition from the serialization code No longer inheriting from Message is a big thing, but it would even be better if the generated type definition files did not depend on any ros header. This can only be done when the serialization code is in a different file. This would allow other robotics software libraries to include and use the generated headers, without depending on roscpp at link time. I'm not proposing (yet) to split of the msgs + generators into a separate package, that's another discussion. I'm just aiming now for binary independence - header only types. An Orocos specific serialization generator would then parse the .msg files again and generate the required serialization functions for the Orocos type system, ie using the boost::serialization API. Maybe ROS could benefit too from this separation when different serialization methods of the same struct become desired. In case replacing ros::Time is not an option either, the non-ROS generator would also need to generate the structs to a ROS-independent format, duplicating more of the effort and maintenance. Similar cases can be made for the srvs API, since also that API can be accessed cross-framework. Since now's the time to give feedback, I had not the luxury of preparing a patch, do comparison tests etc. I hope nevertheless that you see the advantages of this 'shared data types' goal and consider it when redesigning the messages API. A 'patches welcome for X or Y' would already be greatly appreciated. Regards, Peter [1] http://www.boost.org/doc/libs/1_42_0/libs/serialization/doc/index.html [2] http://github.com/psoetens/orocos-rtt/blob/rtt-2.0- mainline/rtt/marsh/binary_data_archive.hpp [3] http://github.com/psoetens/orocos-rtt/blob/rtt-2.0- mainline/rtt/transports/mqueue/MQSerializationProtocol.hpp