[ros-users] Announcing the Bag Database

Thu May 5 22:18:58 UTC 2016

Thanks!

- Description is currently only entered via users; location will be
automatically populated if you've configured a Google API key and it can
find at least one GPS coordinate; and the vehicle field will be
automatically populated by the first string it finds on the
"/vms/vehicle_name" topic.

- Regarding the vehicle names and GPS topics, I realize it's bad to have
the topic names hard-coded. :-)  Making those configurable is on my
issue list.  The list of GPS topics was originally done because many of
our bag files have multiple topics with GPS messages, and some of them
are "better" than others (due to corrections or filtering being
applied), so they were prioritized in order to only extract the best
results.
I think that it doesn't look for /any/ NavSatFix out of the fear that
that would be slow... but after having used it for a while and observed
its speed, I think that fear is unfounded, so it would make sense to
fall back to that if none of the prioritized topics are found.

- Multi-editing is not currently possible, but something I've thought
about before, so I'll add an issue for that, too.  I've written support
in for inline grid editing, but that's broken in the current version of
ExtJS...

- Currently there's no way to export or import the metadata (barring
logging into the database and running raw SQL), but I agree that would
be nice to do; I'll take a look at your tool and see if there's some way
to integrate that or make them compatible with each other.

On 05/05/2016 04:50 PM, Hordur Heidarsson wrote:
> Nice! Writing something like this has been on my todo list for a wile,
> and now I don't have to! :)
> I've set it up using docker, super easy, and it's indexing right now,
> working nicely so far.
>
> Few questions:
> - The description, location & vehicle fields, can those be put in the
> bag file somehow before, like at recording time?
> - If I fill those fields out, is there any way of syncing the info
> back to the bag files or to some sidecar file?
>   (I actually wrote a tool to add metadata to bag files some time
> back https://github.com/hordurk/rosbag_metadata and was going to build
> some server side support for it later on)
> - Possible to edit those fields for many files at once (put same info
> in them for many files)?
> - I'm not seeing GPS data even though my bags have it. Is it looking
> for something else than sensor_msgs/NavSatFix ?
>   ....Never mind, just noticed that you only search at certain topics.
> Is this configurable? Why not just search for any topic of type
> sensor_msgs/NavSatFix?
>
> Awesome work, thanks for sharing!
>
> - hordur
>
> On Thu, May 5, 2016 at 1:14 PM, P. J. Reed via ros-users
> <ros-users at lists.ros.org <mailto:ros-users at lists.ros.org>> wrote:
>
>     As hinted yesterday, the real project I have to announce is the Bag
>     Database.  This is a server that will scan and monitor an arbitrary
>     directory of bag files, index them, and provide a web-based interface
>     that can be used to quickly search through, analyze, and download
>     them.
>     Have you ever wondered, "Do we have any bags that have a
>     TexturedMarker
>     message in them?", or "What did the path this vehicle followed
>     look like
>     on a satellite terrain map?"  This will help you answer both of those
>     questions.
>
>     It was designed primarily for internal use; our team has a NAS on
>     which
>     we store thousands of bag files, many of which are several GB in size,
>     and searching through them by hand was difficult and time-consuming.
>     With the Bag Database, everybody can still use tools such as Samba or
>     SFTP to put their bags on the NAS, and the Bag DB will automatically
>     analyze them and make them available in its UI.  We have about 15
>     TB of
>     bags, and it takes about half a second to search through all of
>     them for
>     arbitrary message types or topics.
>
>     Features:
>     - Display any of the information about a bag normally obtained through
>     "rosbag info"
>     - Quickly search for bags based on their filename, location, contained
>     message types, or published topics
>     - Filter the visible list of bags based on start and end times,
>     latitude/longitude, size, and more
>     - Store user-entered metadata such as the vehicle name or description
>     - Display the bag's path of GPS coordinates on a MapQuest or Bing map
>     - Use Google's reverse-geocoding API to get a string describing a
>     bag's
>     location from its lat/lon coordinates
>     - Identify duplicate bag files and tell you about them (although
>     this UI
>     could be better...)
>
>     The Bag Database is a Java servlet that only needs a PostgreSQL
>     database
>     to be useful, and it's easiest to deploy it as a Docker container.
>     Source code, documentation, and installation instructions are all
>     available on GitHub:
>     https://github.com/swri-robotics/bag-database
>
>     I know it still has a few rough spots I can work out, but I
>     thought now
>     was a good time to go ahead and release it and see how much interest
>     there is.
>
>     Feel free to submit any issues or feature requests on GitHub, and
>     let me
>     know if you have any other questions about it.
>
>     --
>     P. J. Reed, Senior Research Analyst
>     (210) 522-6948 <tel:%28210%29%20522-6948>
>     Intelligent Vehicle Systems (http://ivs.swri.org)
>     Applied Sensing Department (http://appliedsensing.swri.org)
>     Southwest Research Institute (http://www.swri.org)
>
>
>
>     _______________________________________________
>     ros-users mailing list
>     ros-users at lists.ros.org <mailto:ros-users at lists.ros.org>
>     http://lists.ros.org/mailman/listinfo/ros-users
>
>
>
>
> -- 
> Hörður Kristinn Heiðarsson / Hordur Kristinn Heidarsson
> hordur at hordur.us <mailto:hordur at hordur.us> | http://hordur.us/

-- 
P. J. Reed, Senior Research Analyst
(210) 522-6948
Intelligent Vehicle Systems (http://ivs.swri.org)
Applied Sensing Department (http://appliedsensing.swri.org)
Southwest Research Institute (http://www.swri.org)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ros.org/pipermail/ros-users/attachments/20160505/1c06172e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://lists.ros.org/pipermail/ros-users/attachments/20160505/1c06172e/attachment.asc>