[ros-users] Meta-data for bag files

[Tim Field]

Yes - storing meta-data in a relational database is a good solution to
recommend as well.

You do have to be careful to maintain referential integrity with the
filesystem, but you gain the ability to make arbitrary queries on your
meta-data, and to store both custom indexes of your data and relations
between bag files.

Tim

On Mon, Jun 21, 2010 at 9:42 PM, Hai Nguyen <haidai at gmail.com> wrote:

> I've been using python's sqlite bindings for my datasets (
> http://docs.python.org/library/sqlite3.html
> ).  Using the format is just (bag_filename, field1, field2, field3,
> etc).  It's simple, short, sweet, and supports complicated queries.  ex:
> conn = sqlite3.connect('/tmp/example')
> c = conn.cursor()
>
> c.execute('''create table stocks
>  (date text, trans text, symbol text,
>  qty real, price real)''')
>
> c.execute("""insert into stocks
>           values ('2006-01-05','BUY','RHAT',100,35.14)""")
>
> conn.commit()
> c.close()
>
> -Hai
>
> On Jun 21, 2010, at 10:05 PM, Tim Field wrote:
>
> > Hi Ivan,
> >
> > I'm not sure it's possible to devise a meta-data schema flexible
> > enough to satisfy all use cases.  The ROS toolchain is pretty
> > flexible, though, and it should be easy to engineer a solution for
> > your particular needs.  The following techniques[*] might help you:
> >
> > To query the topics and message types in a bag, use:
> >
> >     rosbag info --yaml mybag.bag
> >
> > to return information about the bag in a YAML document.
> >
> > For arbitrary meta-data, it's easy to add any data you want to a
> > bag.  For example, here's a Python script that will take a bag
> > filename passed on the command-line, and add as the first message in
> > the bag a message on the /metadata topic:
> >
> >     import roslib; roslib.load_manifest('rosbag')
> >     import sys, rosbag, rospy, std_msgs.msg
> >     b = rosbag.Bag(sys.argv[1], 'a')
> >
> >     # @todo: generate your meta-data message here
> >     metadata = std_msgs.msg.String()
> >     metadata.data = 'my metadata'
> >
> >     for _, _, t in b.read_messages():
> >         break
> >     b.write('/metadata', metadata, t - rospy.Duration(0, 1))
> >     b.close()
> >
> > For example, if you wanted to annotate the bag with a description
> > and keywords, the meta-data message could be populated from the
> > command-line arguments the user passes in.
> >
> > Your meta-data can then easily be read from the bag, either using
> > the rosbag API, or via the command-line, e.g.
> >
> >     rostopic echo -b metadata.bag -n 1 /metadata
> >
> > Tim
> >
> > [*] assuming a C-turtle installation
> >
> > On Mon, Jun 21, 2010 at 5:46 PM, Ivan Dryanovski <
> ivan.dryanovski at gmail.com
> > > wrote:
> > Hi,
> >
> > We've been running into a somewhat logistical problem of having too
> > many bag files, and no good way to organize them. Has anyone given
> > thought to the idea of adding meta-data to the .bag files? It would be
> > useful if we can attach a description and keywords so we can quickly
> > find relevant bags. For example, you would keep all your bags in a
> > specific directory, and have a ros tool that is able to search through
> > them based on keywords, topics, or message types.
> >
> > Since adding explicit meta-data to the bag format might cause
> > compatibility problems with current bag tools, perhaps a round-about
> > solution would be to attach a "metadata" topic to each bag.
> >
> > Ivan Dryanovski
> > _______________________________________________
> > ros-users mailing list
> > ros-users at code.ros.org
> > https://code.ros.org/mailman/listinfo/ros-users
> >
> > _______________________________________________
> > ros-users mailing list
> > ros-users at code.ros.org
> > https://code.ros.org/mailman/listinfo/ros-users
>
> _______________________________________________
> ros-users mailing list
> ros-users at code.ros.org
> https://code.ros.org/mailman/listinfo/ros-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ros.org/pipermail/ros-users/attachments/20100621/9d373aed/attachment-0003.html>