Nice! Writing something like this has been on my todo list for a wile, and now I don't have to! :)I've set it up using docker, super easy, and it's indexing right now, working nicely so far.
Few questions:- The description, location & vehicle fields, can those be put in the bag file somehow before, like at recording time?- If I fill those fields out, is there any way of syncing the info back to the bag files or to some sidecar file?(I actually wrote a tool to add metadata to bag files some time back https://github.com/hordurk/rosbag_metadata and was going to build some server side support for it later on)- Possible to edit those fields for many files at once (put same info in them for many files)?- I'm not seeing GPS data even though my bags have it. Is it looking for something else than sensor_msgs/NavSatFix ?....Never mind, just noticed that you only search at certain topics. Is this configurable? Why not just search for any topic of type sensor_msgs/NavSatFix?
Awesome work, thanks for sharing!
- hordur
On Thu, May 5, 2016 at 1:14 PM, P. J. Reed via ros-users <ros-users@lists.ros.org> wrote:
As hinted yesterday, the real project I have to announce is the Bag
Database. This is a server that will scan and monitor an arbitrary
directory of bag files, index them, and provide a web-based interface
that can be used to quickly search through, analyze, and download them.
Have you ever wondered, "Do we have any bags that have a TexturedMarker
message in them?", or "What did the path this vehicle followed look like
on a satellite terrain map?" This will help you answer both of those
questions.
It was designed primarily for internal use; our team has a NAS on which
we store thousands of bag files, many of which are several GB in size,
and searching through them by hand was difficult and time-consuming.
With the Bag Database, everybody can still use tools such as Samba or
SFTP to put their bags on the NAS, and the Bag DB will automatically
analyze them and make them available in its UI. We have about 15 TB of
bags, and it takes about half a second to search through all of them for
arbitrary message types or topics.
Features:
- Display any of the information about a bag normally obtained through
"rosbag info"
- Quickly search for bags based on their filename, location, contained
message types, or published topics
- Filter the visible list of bags based on start and end times,
latitude/longitude, size, and more
- Store user-entered metadata such as the vehicle name or description
- Display the bag's path of GPS coordinates on a MapQuest or Bing map
- Use Google's reverse-geocoding API to get a string describing a bag's
location from its lat/lon coordinates
- Identify duplicate bag files and tell you about them (although this UI
could be better...)
The Bag Database is a Java servlet that only needs a PostgreSQL database
to be useful, and it's easiest to deploy it as a Docker container.
Source code, documentation, and installation instructions are all
available on GitHub:
https://github.com/swri-robotics/bag-database
I know it still has a few rough spots I can work out, but I thought now
was a good time to go ahead and release it and see how much interest
there is.
Feel free to submit any issues or feature requests on GitHub, and let me
know if you have any other questions about it.
--
P. J. Reed, Senior Research Analyst
(210) 522-6948
Intelligent Vehicle Systems (http://ivs.swri.org)
Applied Sensing Department (http://appliedsensing.swri.org)
Southwest Research Institute (http://www.swri.org)
_______________________________________________
ros-users mailing list
ros-users@lists.ros.org
http://lists.ros.org/mailman/listinfo/ros-users
--
Hörður Kristinn Heiðarsson / Hordur Kristinn Heidarsson
-- P. J. Reed, Senior Research Analyst (210) 522-6948 Intelligent Vehicle Systems (http://ivs.swri.org) Applied Sensing Department (http://appliedsensing.swri.org) Southwest Research Institute (http://www.swri.org)