Thanks for the great suggestions. I'm now testing a new version of the indexer that downloads N rosinstall files into the combined index. If this new version works, repo maintainers are welcome to take over hosting of their rosinstall files. You can see a list of the ones I'm currently testing with here: https://code.ros.org/svn/ros/stacks/rosorg/trunk/rosdoc_rosorg/repos/ These are then indexed here: https://code.ros.org/svn/ros/stacks/rosorg/trunk/rosdoc_rosorg/repos.list Another nice aspect of this approach is that the rosinstall file is multi-purpose. Not only does it feed the indexer, but it can generally be used with rosinstall to download all stacks in your repositories. - Ken On Tue, Aug 16, 2011 at 9:30 AM, Ken Conley wrote: > 2011/8/16 Lorenz Mösenlechner : >> Hi, >> >> I'm not sure if this approach is really practical. The problem with >> git submodules is that they reference a specific revision in another >> repository. That means whenever I add a new package to a stack and I >> want to have it indexed, I need to make sure that I update the >> repository with the submodule, too. The same holds whenever I make an >> API change or something that I want to have indexed. > > Thanks for the explanation. > >> What about using a magic rosinstall file in a repository. For >> instance, if the git repository tum-ros-pkg.git contains a file >> tum-ros-pkg.rosinstall on toplevel, could the crawler maybe use it to >> find the actual repositories with the stacks? > > I like this implementation approach, though I would make it less > magical.  Instead of pointing to tum-ros-pkg.git, we could have a > meta-indexer that listed these rosinstall files and produce a combined > index. > > Overall, it doesn't suffer from the checkout naming issues and works > for any git/hg/svn/bzr-based repository.  In the future, it > potentially also allows repo maintainers to produce rosinstall index > files for different ROS distributions. > >  - Ken > >> Lorenz >> >>> On Tue, Aug 16, 2011 at 9:04 AM, Lorenzo Riano wrote: >>> > Ok, a more practical example: >>> > >>> > We at Ulster have a shared account on GitHub called  "uu-isrc-robotics". We >>> > started with one repository, called uu-isrc-robotics-pr2-pkgs that contained >>> > one stack and some packages in the stack. When we released the rubik's cube >>> > solver code, I decided to keep it separate from the other stacks (to avoid >>> > people having to pull a lot of code they don't need). I guess this will >>> > become common practice in the future, say Berkely stack becomes a collection >>> > of several stacks hosted by the same user (but not in the same repository). >>> > >>> > Now the problem is that every time we release a new stack, we'll have to >>> > notify ros-users, get a confirmation, and wait for the the crawler to find >>> > the new stack. So my idea would be to have a crawler that monitors a user >>> > account (for example "uu-isrc-robotics") and automatically finds and indexes >>> > which repositories contain stacks. >>> > >>> > I hope it makes sense now >>> >>> The above implementation that is a bit trickier, as it involves the >>> crawler having explicit capabilities with github.  As the crawler is >>> just a rosinstall file, this requires some big feature additions. >>> >>> Also, the major issue I see is that the implicit mapping of the >>> repository name to a checkout name is not correct.  This is important >>> as ROS sets the stack name based on the directory name. >>> >>> As an alternative, you could just create a repository called >>> 'uu-isrc-robotics'.  When you create a new stack repo on git, you >>> could add it as a submodule.  Our indexer would only have to know >>> about uu-isrc-robotics and would get updates whenever you update that >>> repo.   Still no need to notify ros-users, and it has an explicit >>> opt-in for each repository -- for example, for most github accounts, >>> there are several non-ROS-related repositories as well. >>> >>> So my question was basically, why the github-centric approach, instead >>> of just git submodules?  I don't use git, so I don't know if there are >>> some deficiencies with submodules that could be corrected with the >>> github approach. >>> >>>  - Ken >>> >>> >>> > >>> > On 16 August 2011 16:52, Ken Conley wrote: >>> >> >>> >> On Tue, Aug 16, 2011 at 7:24 AM, Lorenzo Riano >>> >> wrote: >>> >> > I have received some emails of people who wanted to run the rubik's cube >>> >> > stack, but they didn't want to pull the whole UU stack just because the >>> >> > rubik's stack depends on a single package in UU (OK it's a bit >>> >> > complicated...) >>> >> > >>> >> > Anyway, what I think would be interesting is to have the ROS indexer >>> >> > scan >>> >> > the packages of a (for example) GitHub user and retrieve all the stacks. >>> >> > This way Berkley, TUM, UU and so on will have an account on GitHub (or >>> >> > google code or whatever) with several repositories, each of them being a >>> >> > stack. >>> >> >>> >> What is the argument for this over, say, git submodules?  There are >>> >> many issues on the implementation side of doing this, so I'm wondering >>> >> what the advantage would be. >>> >> >>> >>  - Ken >>> >> >>> >> > >>> >> > I hope this makes sense... >>> >> > >>> >> > Lorenzo >>> >> > >>> >> > On 15 August 2011 21:05, Ibrahim Awwal >>> >> > wrote: >>> >> >> >>> >> >> Hi ROS users, >>> >> >> >>> >> >> We're considering/planning on switching to git/github for the Berkeley >>> >> >> ROS package. What are the conventions on structure and such that people >>> >> >> are using? With svn we have a url like this >>> >> >> http://ros.berkeley.edu/svn/berkeley-ros-pkg/ with all our stuff, with >>> >> >> git would it be preferrable to have one repo called berkeley-ros-pkg or >>> >> >> separate repos for each stack and maybe link them together with >>> >> >> submodules? What are other people doing that are using git? It seems >>> >> >> like at least TUM is using separate repos per project on their git repo >>> >> >> (which seems to be the much more sane route) but I was wondering if >>> >> >> there was any requirement to have something called foo-ros-pkg >>> >> >> somewhere. Thanks, >>> >> >> >>> >> >> -Ibrahim >>> >> >> _______________________________________________ >>> >> >> ros-users mailing list >>> >> >> ros-users@code.ros.org >>> >> >> https://code.ros.org/mailman/listinfo/ros-users >>> >> > >>> >> > >>> >> > >>> >> > >>> >> > phone: +44 (0)28 71375187 >>> >> > email: l.riano@ulster.ac.uk, lorenzo.riano@gmail.com >>> >> > skype: lorenzo.riano >>> >> > >>> >> > Webpage: http://isrc.ulster.ac.uk/Staff/LRiano/Contact.html >>> >> > >>> >> > _______________________________________________ >>> >> > ros-users mailing list >>> >> > ros-users@code.ros.org >>> >> > https://code.ros.org/mailman/listinfo/ros-users >>> >> > >>> >> > >>> >> _______________________________________________ >>> >> ros-users mailing list >>> >> ros-users@code.ros.org >>> >> https://code.ros.org/mailman/listinfo/ros-users >>> > >>> > >>> > >>> > >>> > phone: +44 (0)28 71375187 >>> > email: l.riano@ulster.ac.uk, lorenzo.riano@gmail.com >>> > skype: lorenzo.riano >>> > >>> > Webpage: http://isrc.ulster.ac.uk/Staff/LRiano/Contact.html >>> > >>> > _______________________________________________ >>> > ros-users mailing list >>> > ros-users@code.ros.org >>> > https://code.ros.org/mailman/listinfo/ros-users >>> > >>> > >>> _______________________________________________ >>> ros-users mailing list >>> ros-users@code.ros.org >>> https://code.ros.org/mailman/listinfo/ros-users >>> >> >> -- >> Lorenz Mösenlechner            | moesenle@in.tum.de >> Technische Universität München | Boltzmannstr. 3 >> 85748 Garching bei München     | Germany >> http://ias.cs.tum.edu/         | Tel: +49 (89) 289-26910 >> _______________________________________________ >> ros-users mailing list >> ros-users@code.ros.org >> https://code.ros.org/mailman/listinfo/ros-users >> >