<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue-Light, Helvetica Neue Light, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px"><div id="yui_3_16_0_1_1420666569516_10896"><span id="yui_3_16_0_1_1420666569516_16178">Hi Tim,</span></div><div id="yui_3_16_0_1_1420666569516_10898"><span><br></span></div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362"><span id="yui_3_16_0_1_1420666569516_9796">Those performance metrics are great! Are they based on an actual connection to a ROSNode? Or was the performance tested on a high performance mach ROSNode? The clients will only perform as fast as the weakest link, therefore if the ROSNode is performing poorly, you may never know the potential of your ROS Client.</span></div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362"><span><br></span></div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362">My custom loggers aren't actually custom at all, they are an implementation of rosbag. The Java based Autonomous Robotics Framework I've been working on contains a SensorProcessorManager, that incoming messages are sent to. The framework architecture includes the requirement to proxy specific Sensors (or in ROS Speak - Topics, Services, Actions) from incoming RCSM (<span style="color: rgb(34, 34, 34); font-family: 'Open Sans', 'Helvetica Neue', Helvetica, Helvetica, Arial, sans-serif; line-height: 25.600000381469727px;" class="" id="yui_3_16_0_1_1420666569516_14970">Robot Control System Messenger</span>) Plugins (of which ROS Client is an RCSM plugin). The Perception Engine contains the SensorProcessorManager, and by default a Hardware Event Control Manager, that calls an InstructionManager to obtain Instruction Objects that are reused via Object Pools for each Hardware Sensor, if one does not exist, one will be autogenerated on the fly </div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362"><span id="yui_3_16_0_1_1420666569516_16195">Input/Output Handler classes, and a Native Instruction Object</span>. Android can use a web service to compile new Instruction Objects remotely via our web service, and then download the new Instruction Objects. Incoming data messages are migrated from binary to Instruction Objects, and passed on for handling via perception engine in the first Sensor Processor, and ROSBag recording of specified topics services is via the second synchronously called Sensor Processor. The same process is implemented on outgoing messages via my Frameworks Movement Processor Manager.</div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362"><br></div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362">ROSBag made sense for recording because ROSBag Playback tools can be used in simulation on any implementation, and It can be used without any modifications to record all incoming messages regardless of whether they are from ROS, or another RCSM Provider.</div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362"><br></div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362">As for where to record ROSBag files to, it doesn't matter in the framework I am using, because its flexible enough to write a ROSBag to a local file, a web service like Dropbox, Google Drive, any JDBC compliant SQL DB, or MongoDB with a minor amount of code. Currently, the dev plugin implementation is writing to file, but its so huge, it may be a network location, bandwidth permitting.</div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362"><br></div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362">The reason to use custom logging? Flexibility is necessary, when you are implementing a robotic system that can plugin to any robot for AI out of the box, with a long term goal of self replicating, repairing, mining, farming, manufacturing complex humanoid robots. Sounds crazy, huh?</div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362"><br></div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362">Thanks,<br><br>Aaron</div><div dir="ltr" id="yui_3_16_0_1_1420666569516_9362"><br></div> <div class="qtdSeparateBR"><br><br></div><div class="yahoo_quoted" style="display: block;"> <div style="font-family: HelveticaNeue-Light, Helvetica Neue Light, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div dir="ltr"> <font size="2" face="Arial"> On Wednesday, January 7, 2015 1:52 AM, Tim Niemueller <niemueller@kbsg.rwth-aachen.de> wrote:<br> </font> </div> <br><br> <div class="y_msg_container">Am 06.01.2015 um 22:37 schrieb Aaron Schiffman:<br clear="none">> Hi Tim,<br clear="none">> <br clear="none">> We are doing<br clear="none">> data logging with a custom rosbag recorder implementation in Java, that writes to an outputstream.<br clear="none"><br clear="none">Hi Aaron.<br clear="none"><br clear="none">Can you give some hints why you want to have a custom logger? If it is<br clear="none">about making the data structures available and query-able, you might<br clear="none">want to check out mongodb_log [1, 2], maybe it already does what you<br clear="none">intend to do (it's fast and provides powerful query capabilities). If it<br clear="none">is really just about recording the binary (encoded) message flow, what<br clear="none">would your method provide over rosbag?<br clear="none"><br clear="none">Back in the days I did some simple benchmarks in particular regarding<br clear="none">decoding performance of messages in C++, Lua, and Python. The results<br clear="none">are on <a shape="rect" href="http://timn.github.io/roslua/" target="_blank">http://timn.github.io/roslua/ </a>and the code used at<br clear="none"><a shape="rect" href="https://github.com/timn/ros-actionlib_benchmark." target="_blank">https://github.com/timn/ros-actionlib_benchmark. </a>Maybe you want to<br clear="none">compare these to Java decoding performance. It turns out that this very<br clear="none">step is often what limits you the most for large-scale logging,<br clear="none">especially when you want to store the data in a query-able format<br clear="none">(because you often have many small messages which are hierarchically<br clear="none">structured and thus expensive to decode CPU-wise, rather than blobs you<br clear="none">could decode quicker which would actually be limited by available I/O<br clear="none">bandwidth).<br clear="none"><br clear="none">The data was recorded back in 2010 with Diamondback or Electric I guess,<br clear="none">so things might have changed. Back then rosbag also had the bug of<br clear="none">storing the message definition verbatim for each and every message<br clear="none">(resulting in the enormous compression rates one could achieve on bag<br clear="none">files and bad I/O performance, don't know if that has been fixed in the<br clear="none">meantime, presented my findings at OSRF in 2013).<div class="yqt9213304795" id="yqtfd92803"><br clear="none"><br clear="none"> Tim</div><br clear="none"><br clear="none">[1] <a shape="rect" href="http://www.fawkesrobotics.org/projects/mongodb-log/" target="_blank">http://www.fawkesrobotics.org/projects/mongodb-log/</a><br clear="none">[2] <a shape="rect" href="http://wiki.ros.org/mongodb_log" target="_blank">http://wiki.ros.org/mongodb_log</a><br clear="none"><br clear="none">-- <br clear="none">KBSG - Knowledge-Based Systems Group AllemaniACs RoboCup Team<br clear="none">========================================================================<br clear="none"><a shape="rect" href="http://robocup.rwth-aachen.de/" target="_blank">http://robocup.rwth-aachen.de </a> RWTH Aachen University<br clear="none"><a shape="rect" href="http://kbsg.rwth-aachen.de/" target="_blank">http://kbsg.rwth-aachen.de </a> Ahornstrasse 55<br clear="none"><a shape="rect" href="http://www.fawkesrobotics.org/" target="_blank">http://www.fawkesrobotics.org </a> D-52056 Aachen<div class="yqt9213304795" id="yqtfd84441"><br clear="none"></div><br><br></div> </div> </div> </div> </div></body></html>