Does the RPC latency include the time going to the master to lookup the service, or is it just measuring the service throughput (i.e. with persistent enabled)? I would expect relatively high latency is this was establishing new connections for each service call. - Ken On Wed, Aug 4, 2010 at 11:27 PM, Daniel Stonier wrote: > > > On 14 July 2010 15:51, Daniel Stonier wrote: >> >> On 14 July 2010 03:52, Josh Faust wrote: >> >> == NUM_PTS 1000 == >> >> >> >> board               avg_ser     avg_deser >> >> intel i5             0.000011    0.000025 >> >> arm1176jzf-s   0.001328    0.003354 >> > >> > Ok, so serialization is definitely much slower, but with the latency >> > test >> > you're using it shouldn't affect anything.  Good to know though. >> > >> >> >> >> == Latency Results == >> >> >> >>                          IPC        NoTCP      Bypass >> >> intel i5           :   0.4ms     0.10ms     0.05ms >> >> arm1176jzf-s :   25.0ms   4.50ms     3.50ms >> >> >> >> ====== Raw Socket Tests ====== >> >> >> >> Tests the latencies of sending a single char from server to client, >> >> written using simple posix socket code. >> >> >> >> intel i5           : 0.1ms >> >> arm1176jzf-s : 1.4ms >> > >> > Do you have any way of profiling exactly what's taking time, either with >> > gprof, google perf tools, or something else? >> > Josh >> >> Since the slowdowns were across the board I started doubting the >> platform underneath the ros - ran some test programs I have back on >> our very simple non-ros platform and noticed a significant difference. >> So I'll rebuild the platform, check some of the flags used on the old >> build and make sure I match them on the new one and see how that goes. >> Will try gprof in parallel too I think, but its going to be a couple >> of days recompiling. Will let you know how it goes and thanks for the >> advice. >> > > Got waylaid by another project for a while - sorry to drag out old cobwebs! > > Optimised everything else, made sure I'm using the correct flags for the > platform and the tcp/ip msg latencies have dropped from 25ms -> 9-10ms, > which is starting to perhaps seem reasonable. The rpc latencies are still > horrendous though - ~100ms for one process to get in contact with another > (and about 5ms for the return trip). I managed to get gprof working on it > and compared it to the same processes running on the intel i5, but its > showing up nothing of considerable note. Just some boost spinlocks > contributing an extra 10% worth of cpu time (which probably dont show up on > the i5 simply because they trigger on and off so fast). But certainly > nothing obvious. > > So, I'm guessing that the process (or roscore) is idling/hanging somewhere. > I'm going to have to manually hack some of the ros code and pull some > timings to figure out just where I think (don't know of a better way). > > The results are at http://snorriheim.dnsdojo.com/tmp/rpc_latency if > interested. > > _______________________________________________ > ros-users mailing list > ros-users@code.ros.org > https://code.ros.org/mailman/listinfo/ros-users > >