<div dir="ltr">Another point about testing vs public or shadow-fixed is that if you haven't released yet, then your new version of your package will never deployed on top of the exact debs in public, because you'll release your new version into shadow-fixed (with all other staged releases) and then they are all moved to public all at once. Therefore, a prerelease which tests on top of public is a configuration which never exists in the wild (unless at the time of your prerelease the binaries in shadow-fixed exactly match what is in public, i.e. right after a sync). So I'd argue testing on top of public doesn't really make sense and testing on top of shadow-fixed better reflects what is actually going to happen after you release.<div><br></div><div>Also as to the purpose of a prerelease is to answer the question is my release going to work when we sync to pubic next time, which I can break down into three questions:</div><div><br></div><div>1. Did I break my thing?</div><div>2. Did I break someone that uses me?</div><div>3. Did someone else break me?</div><div><br></div><div>In my opinion, I think 3 is just as valid to be testing for as 1 or 2 is.</div><div><br></div><div>Cheers,</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Nov 16, 2015 at 12:01 PM, Austin Hendrix via ros-release <span dir="ltr"><<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I see the PR build as separate from the prerelease - prerelease tests compatibility with the rest of the ecosystem, whereas the PR build would be a deb build to validate that the package has been bloom'ed correctly, and that it builds debs.<br>
<br>
Paired releases is definitely worth considering; esp since each repo that is released is usually built as many separate debs. If we wanted to test that all debs within a released unit build correctly, we'd have to set up some kind of temporary deb repo for use during the test process. This is probably the same regardless of whether a unit is a single repo, or several linked repos that are released simultaneously.<br>
<br>
-Austin<br>
<br>
On 11/16/2015 11:37 AM, Tully Foote wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
<br>
<br>
On Mon, Nov 16, 2015 at 11:32 AM, Austin Hendrix via ros-release <<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a> <mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>>> wrote:<br>
<br>
For test builds for PRs, why not do the test (deb)build against<br>
the stable repository, instead of shadow-fixed or building?<br>
<br>
It won't catch issues caused by changes in upstream packages, but<br>
by testing against stable you should always have dependencies<br>
available (and it's stable, so the dependencies are less likely to<br>
be broken).<br>
<br>
<br>
The core will be stable. But if we only test against the public release repository. If you want to do a paired release of two packages you would have to wait a full release cycle before the first one would be available to attempt to build against the previously released one.<br>
<br>
<br>
-Austin<br>
<br>
On 11/16/2015 10:39 AM, Tully Foote via ros-release wrote:<br>
<br>
Hi,<br>
<br>
Indeed we agree that getting more prereleases run is<br>
important. That's why we've worked very hard to make sure that<br>
it as well as all the jobs are reproducible.<br>
<br>
The web hosted solution has elements which are nice for users.<br>
However, it was an expensive pain to maintain. It also<br>
provided an inconsistent user experience, especially if<br>
there's a large delay due to waiting for yours or others jobs<br>
to run. As long as it's reproducible it's better to users<br>
leverage their own resources and know reliably when things<br>
will run.<br>
<br>
For preventing broken releases I do also want to integrate<br>
test builds into the PR validation. It will more likely be a<br>
test debbuild rather than a prerelease. And it has trouble if<br>
the buildfarm is in the middle of a large rebuild and all the<br>
dependencies are not immediately available.<br>
<br>
Tully<br>
<br>
On Sat, Oct 31, 2015 at 5:16 AM, Daniel Stonier<br>
<<a href="mailto:d.stonier@gmail.com" target="_blank">d.stonier@gmail.com</a> <mailto:<a href="mailto:d.stonier@gmail.com" target="_blank">d.stonier@gmail.com</a>><br></div></div><div><div class="h5">
<mailto:<a href="mailto:d.stonier@gmail.com" target="_blank">d.stonier@gmail.com</a> <mailto:<a href="mailto:d.stonier@gmail.com" target="_blank">d.stonier@gmail.com</a>>>> wrote:<br>
<br>
<br>
Aye, thanks from me too.<br>
<br>
I'd like to +1 Jack's comments about preferring the web<br>
service<br>
that was previously available.<br>
<br>
1. Getting things right on one web server somewhere is always<br>
going to be far easier than getting it right on thousands<br>
of users<br>
systems. Even if docker does make this proposition easier,<br>
what we<br>
have seen above is it still gets awkward when the dependencies<br>
shift (e.g. needing a custom version of docker). I also<br>
ran into<br>
problems because of python3 interfering with my environment.<br>
<br>
2. Getting users to pre-release is a desirable thing. Less<br>
rosdistro PR's to approve, less red blips on the build<br>
chart, less<br>
latency for Tully to wrap up an official release into<br>
public. The<br>
less barriers there are for them to do this, I feel the<br>
better and<br>
easier the maintainence will be.<br>
<br>
Daniel.<br>
<br>
<br>
On 27 October 2015 at 06:12, Mani Monajjemi via ros-release<br>
<<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><br></div></div>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><span class=""><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>>>> wrote:<br>
<br>
Thanks Tully. pre-release script now works fine<br>
without any issue.<br>
<br>
- Mani<br>
<br>
Mani Monajjemi<br>
<br>
On Mon, Oct 26, 2015 at 11:08 AM, Tully Foote via<br>
ros-release<br>
<<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><br></span>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><span class=""><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>>>><br>
wrote:<br>
<br>
I've pushed ros_buildfarm 0.2.1 it should now be<br>
usable<br>
from the debian packages.<br>
<br>
Tully<br>
<br>
On Mon, Oct 26, 2015 at 10:42 AM, William Woodall via<br>
ros-release <<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><br></span>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><span class=""><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>>>> wrote:<br>
<br>
On Mon, Oct 26, 2015 at 10:09 AM, Jack O'Quin via<br>
ros-release <<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><br></span>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><span class=""><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>>>> wrote:<br>
<br>
<br>
<br>
On Mon, Oct 26, 2015 at 11:24 AM, Daniel<br>
Stonier<br>
via ros-release <<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><br></span>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><div><div class="h5"><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>>>> wrote:<br>
<br>
<br>
It has been a while since I've built open<br>
source debs, but this is above and<br>
beyond the<br>
effort required to prerelease for me<br>
though.<br>
It used to be such a fundamental part<br>
of the<br>
process - what is the current<br>
thinking? Given<br>
that its been out for at least three<br>
months,<br>
are most people just guessing,<br>
rebuilding on<br>
the farm, guessing again? Is there a<br>
planned<br>
remedy on the horizon?<br>
<br>
<br>
Basically, yes.<br>
<br>
Presently, the pre-release tests take more<br>
effort<br>
than just hoping for the best and then fixing<br>
things that break.<br>
<br>
<br>
Why? I've been using them for rviz and it seems to<br>
work fairly well. What's holding up making them<br>
useful? It is just the need to install it into a<br>
virtualenv first?<br>
<br>
My 2 cents - I'd really love to see this<br>
working again ;) ;) ;) ;)<br>
<br>
<br>
Again, what's not working? Is there an issue<br>
on Github<br>
tracking the problem?<br>
<br>
<br>
Saves alot of time for me not having<br>
to ping<br>
pong back and forth trying to get my<br>
dependencies right and I'm sure it<br>
makes the<br>
job easier on the other end avoiding<br>
having so<br>
many red blips on the radar so often.<br>
<br>
<br>
+1 I would find it helpful, too. I much prefer<br>
running the tests.<br>
<br>
The pre-docker web interface was very<br>
convenient.<br>
I think this could be, too, although it's<br>
annoying<br>
that the Trusty version of docker is too<br>
old to use.<br>
<br>
<br>
There's nothing to be done about that<br>
unfortunately.<br>
<br>
-- joq<br>
<br>
_______________________________________________<br>
ros-release mailing list<br>
<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a> <mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><br></div></div>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><span class=""><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>>><br>
<a href="http://lists.ros.org/mailman/listinfo/ros-release" rel="noreferrer" target="_blank">http://lists.ros.org/mailman/listinfo/ros-release</a><br>
<br>
<br>
<br>
<br>
-- William Woodall<br>
ROS Development Team<br>
<a href="mailto:william@osrfoundation.org" target="_blank">william@osrfoundation.org</a> <mailto:<a href="mailto:william@osrfoundation.org" target="_blank">william@osrfoundation.org</a>><br></span>
<mailto:<a href="mailto:william@osrfoundation.org" target="_blank">william@osrfoundation.org</a><span class=""><br>
<mailto:<a href="mailto:william@osrfoundation.org" target="_blank">william@osrfoundation.org</a>>><br>
<a href="http://wjwwood.io/" rel="noreferrer" target="_blank">http://wjwwood.io/</a><br>
<br>
_______________________________________________<br>
ros-release mailing list<br>
<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a> <mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><br></span>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><span class=""><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>>><br>
<a href="http://lists.ros.org/mailman/listinfo/ros-release" rel="noreferrer" target="_blank">http://lists.ros.org/mailman/listinfo/ros-release</a><br>
<br>
<br>
<br>
_______________________________________________<br>
ros-release mailing list<br>
<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a> <mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><br></span>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><span class=""><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>>><br>
<a href="http://lists.ros.org/mailman/listinfo/ros-release" rel="noreferrer" target="_blank">http://lists.ros.org/mailman/listinfo/ros-release</a><br>
<br>
<br>
<br>
_______________________________________________<br>
ros-release mailing list<br>
<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a> <mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><br></span>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><span class=""><br>
<mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>>><br>
<a href="http://lists.ros.org/mailman/listinfo/ros-release" rel="noreferrer" target="_blank">http://lists.ros.org/mailman/listinfo/ros-release</a><br>
<br>
<br>
<br>
<br>
-- Phone : <a href="tel:%2B82-10-5400-3296" value="+821054003296" target="_blank">+82-10-5400-3296</a> <tel:%2B82-10-5400-3296><br>
<tel:%2B82-10-5400-3296> (010-5400-3296)<br>
Home: <a href="http://snorriheim.dnsdojo.com/" rel="noreferrer" target="_blank">http://snorriheim.dnsdojo.com/</a><br>
Yujin Inno: <a href="http://inno.yujinrobot.com/" rel="noreferrer" target="_blank">http://inno.yujinrobot.com/</a><br>
<<a href="http://rnd.yujinrobot.com/" rel="noreferrer" target="_blank">http://rnd.yujinrobot.com/</a>><br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
ros-release mailing list<br></span>
<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a> <mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><span class=""><br>
<a href="http://lists.ros.org/mailman/listinfo/ros-release" rel="noreferrer" target="_blank">http://lists.ros.org/mailman/listinfo/ros-release</a><br>
<br>
<br>
_______________________________________________<br>
ros-release mailing list<br>
<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a> <mailto:<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a>><br>
<a href="http://lists.ros.org/mailman/listinfo/ros-release" rel="noreferrer" target="_blank">http://lists.ros.org/mailman/listinfo/ros-release</a><br>
<br>
<br>
</span></blockquote><span class="">
<br>
_______________________________________________<br>
ros-release mailing list<br>
<a href="mailto:ros-release@lists.ros.org" target="_blank">ros-release@lists.ros.org</a><br>
</span><a href="http://lists.ros.org/mailman/listinfo/ros-release" rel="noreferrer" target="_blank">http://lists.ros.org/mailman/listinfo/ros-release</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr">William Woodall<div>ROS Development Team</div><div><a href="mailto:william@osrfoundation.org" target="_blank">william@osrfoundation.org</a></div><div><a href="http://wjwwood.io/" target="_blank">http://wjwwood.io/</a></div></div></div>
</div>