On Mon, Oct 14, 2013 at 4:24 PM, Isaac Isao Saito <
iisaito@opensource-robotics.tokyo.jp> wrote:

> Thanks for a clear answer Tully.
> I now understand the reason why this wiki is calling itself
> `regression` test. Seems pretty useful once I understand what it does.
> http://wiki.ros.org/regression_tests
>
> Let me ask one more -- I see 2 wiki page sections that describe ways
> to run local pre-release test.
>
> - http://wiki.ros.org/jenkins_tools#run_chroot_local
> - http://wiki.ros.org/bloom/Tutorials/PrereleaseTest#Locally
>
> How are they different in terms of test features? Since jenkins_tools
> looks like maintained at OSRF should we favor it?
>

Both are maintained by OSRF. The jenkins_tools one is what the buildfarm
actually runs.  The one in the PrereleaseTest#Locally section simply will
verify that the package builds.  I added a comment to that effect to the
wiki page.

Tully


>
> Thanks again.
>
> Isaac
>
>
> On Sun, Oct 13, 2013 at 10:39 PM, Tully Foote <tfoote@osrfoundation.org>
> wrote:
> > HI Isaac
> >
> >
> > On Sat, Oct 12, 2013 at 2:29 PM, Isaac Isao Saito <130s@lateeye.net>
> wrote:
> >>
> >> Hi ROS-Releasers,
> >>
> >> I'm a slow learner and still need to ask a few questions about
> >> pre-release test (let me abbreviate as PT) on ROS buildfarm even
> >> though I'm already a big fan of it.
> >>
> >> Q1) A PT failed [1] due to a missing package with the following error.
> >>
> >>     [rospack] Error: stack/package openrtm_aist not found
> >>
> >> From regression_test wiki [2], I understand that PT fetches
> >> dependencies from shadow-fixed. But I see the package that's claimed
> >> to be missing `openrtm_aist` do exist in the shadow-fixed.
> >>
> >> What's happening here? Am I misunderstanding something?
> >
> >
> > Being available from APT does not mean that it will be available to
> rospack.
> > You must have the dependency declared for it to be installed. That's
> exactly
> > the sort of errors that the prerelease job is designed to catch.
> >
> >>
> >>
> >>
> >> Q2) Is there a case when a package can be still released even its PT
> >> fails? If so how can I tell that?
> >>
> >> I see some PTs fail while production build of the same packages don't.
> >>
> >> See Groovy actionlib PT for example [3]. It's failing because building
> >> of some downloaded packages that depend on actionlib fail. And I see
> >> that all of its unit tests passed beforehand.
> >>
> >> So can it be said "as long as the unit tests of the tested package
> >> passed"?
> >
> >
> > It is quite possible for lower level packages that not all downstream
> > packages fail.  If the maintainer is confident that the failures
> previously
> > existed or are unrelated to their current release they can make a
> judgement
> > call to release anyway.  That quote says that you should at a minimum not
> > release if your own unit tests fail.
> >
> >>
> >>
> >>
> >> Q3) Related to Q2, what's the purpose of this passage from the same
> >> regression_test wiki?
> >>
> >>     "Additionally, it downloads all repos/stacks that depend on the
> >> tested repos/stacks, and builds/tests them."?
> >
> >
> > This is saying that it not just tests the packages requested but also
> > released packages which depend on the requested packages/stacks.
> >
> > Tully
> >
> >>
> >>
> >> [1]
> >>
> http://jenkins.ros.org/job/prerelease-groovy-openhrp3/ARCH_PARAM=amd64,UBUNTU_PARAM=precise,label=prerelease/2/consoleFull
> >> [2] http://wiki.ros.org/regression_tests#Prerelease_Tests
> >> [3]
> >>
> http://jenkins.ros.org/job/prerelease-groovy-actionlib/ARCH_PARAM=amd64,UBUNTU_PARAM=precise,label=prerelease/7/console
> >>
> >> Thanks a lot!
> >>
> >> --
> >> Isao Isaac Saito
> >> Software Engineer
> >> TORK (Tokyo Opensource Robotics Kyokai Association) / 東京オープンソースロボティクス協会
> >> http://opensource-robotics.tokyo.jp
> >> _______________________________________________
> >> ros-release mailing list
> >> ros-release@code.ros.org
> >> http://lists.ros.org/mailman/listinfo/ros-release
> >
> >
> _______________________________________________
> ros-release mailing list
> ros-release@code.ros.org
> http://lists.ros.org/mailman/listinfo/ros-release
>