Thanks for a clear answer Tully. I now understand the reason why this wiki is calling itself `regression` test. Seems pretty useful once I understand what it does. http://wiki.ros.org/regression_tests Let me ask one more -- I see 2 wiki page sections that describe ways to run local pre-release test. - http://wiki.ros.org/jenkins_tools#run_chroot_local - http://wiki.ros.org/bloom/Tutorials/PrereleaseTest#Locally How are they different in terms of test features? Since jenkins_tools looks like maintained at OSRF should we favor it? Thanks again. Isaac On Sun, Oct 13, 2013 at 10:39 PM, Tully Foote wrote: > HI Isaac > > > On Sat, Oct 12, 2013 at 2:29 PM, Isaac Isao Saito <130s@lateeye.net> wrote: >> >> Hi ROS-Releasers, >> >> I'm a slow learner and still need to ask a few questions about >> pre-release test (let me abbreviate as PT) on ROS buildfarm even >> though I'm already a big fan of it. >> >> Q1) A PT failed [1] due to a missing package with the following error. >> >> [rospack] Error: stack/package openrtm_aist not found >> >> From regression_test wiki [2], I understand that PT fetches >> dependencies from shadow-fixed. But I see the package that's claimed >> to be missing `openrtm_aist` do exist in the shadow-fixed. >> >> What's happening here? Am I misunderstanding something? > > > Being available from APT does not mean that it will be available to rospack. > You must have the dependency declared for it to be installed. That's exactly > the sort of errors that the prerelease job is designed to catch. > >> >> >> >> Q2) Is there a case when a package can be still released even its PT >> fails? If so how can I tell that? >> >> I see some PTs fail while production build of the same packages don't. >> >> See Groovy actionlib PT for example [3]. It's failing because building >> of some downloaded packages that depend on actionlib fail. And I see >> that all of its unit tests passed beforehand. >> >> So can it be said "as long as the unit tests of the tested package >> passed"? > > > It is quite possible for lower level packages that not all downstream > packages fail. If the maintainer is confident that the failures previously > existed or are unrelated to their current release they can make a judgement > call to release anyway. That quote says that you should at a minimum not > release if your own unit tests fail. > >> >> >> >> Q3) Related to Q2, what's the purpose of this passage from the same >> regression_test wiki? >> >> "Additionally, it downloads all repos/stacks that depend on the >> tested repos/stacks, and builds/tests them."? > > > This is saying that it not just tests the packages requested but also > released packages which depend on the requested packages/stacks. > > Tully > >> >> >> [1] >> http://jenkins.ros.org/job/prerelease-groovy-openhrp3/ARCH_PARAM=amd64,UBUNTU_PARAM=precise,label=prerelease/2/consoleFull >> [2] http://wiki.ros.org/regression_tests#Prerelease_Tests >> [3] >> http://jenkins.ros.org/job/prerelease-groovy-actionlib/ARCH_PARAM=amd64,UBUNTU_PARAM=precise,label=prerelease/7/console >> >> Thanks a lot! >> >> -- >> Isao Isaac Saito >> Software Engineer >> TORK (Tokyo Opensource Robotics Kyokai Association) / 東京オープンソースロボティクス協会 >> http://opensource-robotics.tokyo.jp >> _______________________________________________ >> ros-release mailing list >> ros-release@code.ros.org >> http://lists.ros.org/mailman/listinfo/ros-release > >