Well, right now I have the following

each text file has 6 columns, I represent each line using a ros msg of six entities
then represent the whole file as array of the line msg which looks ok to me.

then I was thinking of creating a master msg(just a thought) which I call database which will contain arrays of the file msgs, this way I can iterate using the Header entity which I have in every file msg when I want to locate a particular file and then load the contents from there.

The catch here is that I have a bunch of services which I want to stuff like add file(so request is a file msg), then I should be able to sort of append the new msg to the database msg and this should be storable to a file in the state I leave it in.

of course if I could store msgs(directly) it would be much easier to just store the file msgs. I am really not sure what to do, I think I am going to look into the folowing for now:

- try to implement a simple conversion from text file->msg then msg-> text file
- try to implement some sort of file system to directly store msg files
- since bags require me to have topics, and I have mainly services, i dont think there is much I can do about them right now
- alternatively I can just use sqlite to have a small database and make interfaces for accessing it, but then will mess with my nice communication of msgs containing the info.

On Fri, Jun 11, 2010 at 6:30 PM, Jeremy Leibs <leibs@willowgarage.com> wrote:
So, if you are just looking to go from memory to disk and back to memory again, bag files are possibly an entirely reasonable choice starting with cturtle.  To save, you could write all the messages into a bag.  And then when you start your program you could give it a bag to load all the messages from again.

However, the caveat is that the only structure of messages you can store at the moment is an ordered (or unordered) set.  You can abuse the time parameter to create an arbitrary ordering.  However, if you have some other data structure of messages, e.g., a dictionary mapping strings to messages, there is no way to map that into a bag file.

Clearly there would be great value in allowing bags to be created and indexed using arbitrary key-types such as strings or ints, but it will still be quite some time before that gets implemented.

So, if your database state can be captured completely by a suitable ordered set of messages, go ahead and try using bags.  Otherwise you might have to implement something on your own.  If you do have to implement your own file format, you should take a look at the ROS serialization API.  All ROS messages can fairly easily be serialized into an appropriate binary block for writing to file and that same binary block, when loaded back from the file can be deserialized into a message.

--Jeremy



On Fri, Jun 11, 2010 at 2:43 PM, Billy Okal <b.okal@jacobs-university.de> wrote:
Yes I wanted a more ROS approach to the storage, I could of course just convert them back to text files and store them, but I I was looking for something smarter than that and I though bag files could do some magic here, if not at least this should be ticketed or something like that( a way to store msg data directly into mem and be able to load it and do stuff on it)

thanks


On Fri, Jun 11, 2010 at 5:23 PM, Jeremy Leibs <leibs@willowgarage.com> wrote:
Billy,

I'm sorry I'm not entirely sure what it is that you are looking for.  If you want things to be fast, it sounds like you can load all of these files as messages into memory and work with them there without too much trouble.  1000 files of 4000 floats is only about 30MB, even if you store them as doubles you should be fine.

Are you just looking for a way to write the messages out to disk and load them in again at a later point?  It sounds like you already have a way of reading your text files and converting them to messages.  Is it that you haven't implemented a way of writing them back out to your text format, or are you wanting to use a different, ROS-native, format to store them?

If you could elaborate a little bit on what it is that you are looking for a solution to I can maybe point you in the right direction.

--Jeremy


On Fri, Jun 11, 2010 at 1:38 PM, Billy Okal <b.okal@jacobs-university.de> wrote:
Dear Ros Users,

I am writting a ros package to emulate a database in which I want to store many(1000+) text files(each containing about 4000 float values). Originally I though of using bag files to store this info but now it looks like thats not what I want. Basically I want to store these files and be able to perform Addition, Deletion, and Querying(compare one text file against all the rest to infer some properties of grasps). Right now I have made Ros msg representing the text files and can move them around ros, but the problem is storing these messages in some lasting memory so that I just load the database(which I can also make a ros message) in the beginning and to stuff then write it back.

Does anybody have experience in this or something related or just some ideas I could use to go about this? Also I need to querying to be really fast(the goal is real time control)

--
Best Regards,

Billy Okal

sure vi is user friendly, its just particular about who to be friends with

_______________________________________________
ros-users mailing list
ros-users@code.ros.org
https://code.ros.org/mailman/listinfo/ros-users



_______________________________________________
ros-users mailing list
ros-users@code.ros.org
https://code.ros.org/mailman/listinfo/ros-users




--
Best Regards,

Billy Okal

sure vi is user friendly, its just particular about who to be friends with

_______________________________________________
ros-users mailing list
ros-users@code.ros.org
https://code.ros.org/mailman/listinfo/ros-users



_______________________________________________
ros-users mailing list
ros-users@code.ros.org
https://code.ros.org/mailman/listinfo/ros-users




--
Best Regards,

Billy Okal

sure vi is user friendly, its just particular about who to be friends with