[ros-release] Build Farm Spam and Debugging

Thu Apr 7 18:57:23 UTC 2016

Jackie and I came up with a fix for the Debian jobs:
https://github.com/ros-infrastructure/ros_buildfarm/pull/281 It fixes the
locale on Debian and afterwards is able to parse and decode the manifest
correctly. The job in question has passed now.

The emails are sent for a reason and not really "spam". They are notifying
you about a problem with a package you. Please feel free to ask questions
on this mailing list if a job is "bugging" you with emails and you are not
sure how to address it. That is always better then to ignore the
notifications.

Thanks,
- Dirk

On Thu, Apr 7, 2016 at 10:55 AM, Dirk Thomas <dthomas at osrfoundation.org>
wrote:

> Before drawing any conclusions please take a close look to the actual
> error message:
>
> ```
> Traceback (most recent call last):
>   File "/tmp/ros_buildfarm/scripts/release/get_sources.py", line 34, in
> <module>
>     sys.exit(main())
>   File "/tmp/ros_buildfarm/scripts/release/get_sources.py", line 30, in
> main
>     args.os_name, args.os_code_name, args.source_dir)
>   File "/tmp/ros_buildfarm/ros_buildfarm/sourcedeb_job.py", line 52, in
> get_sources
>     pkg = parse_package(sources_dir)
>   File "/usr/lib/python3/dist-packages/catkin_pkg/package.py", line 370,
> in parse_package
>     return parse_package_string(f.read(), filename, warnings=warnings)
>   File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
>     return codecs.ascii_decode(input, self.errors)[0]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 183:
> ordinal not in range(128)
> ```
>
> Decoding a non-ASCII character when then codec is set to `ascii` can't
> work. Since `!` is a valid ASCII character that can't be the reason. So it
> is likely the `é`.
>
> What needs to change here? Well, the first important fact to note is that
> the sourcedeb jobs for the same package pass on all Ubuntu platforms. So
> obviously the buildfarm as well as catkin_pkg are able to handle this.
>
> Why is it failing for this Debian job then? Debian has just recently been
> added and there must be something different. Looking at the console output
> again the following warnings just a few lines above should be enough:
>
> ```
> perl: warning: Setting locale failed.
> perl: warning: Please check that your locale settings:
> LANGUAGE = (unset),
> LC_ALL = (unset),
> LANG = "en_US.UTF-8"
>     are supported and installed on your system.
> perl: warning: Falling back to the standard locale ("C").
> perl: warning: Setting locale failed.
> perl: warning: Please check that your locale settings:
> LANGUAGE = (unset),
> LC_ALL = (unset),
> LANG = "en_US.UTF-8"
>     are supported and installed on your system.
> perl: warning: Falling back to the standard locale ("C").
> ```
>
> While the job tries to use UTF-8 it fails to do so on Debian and falls
> back to the standard locale "C". And with that locale it is simply
> impossible to decode the special character.
>
> As a conclusion the recent modification to enable Debian jobs (
> https://github.com/ros-infrastructure/ros_buildfarm/commit/5106e704ecdb2d55ac513da66ec8d699731bb859#diff-8859c2a0d6adc3dc403698f904632818)
> needs fixing since it is not enabling the UTF-8 locale as expected.
>
> Cheers,
> - Dirk
>
>
> On Thu, Apr 7, 2016 at 10:07 AM, Jackie Kay via ros-release <
> ros-release at lists.ros.org> wrote:
>
>> I also find the build farm spam quite annoying. I get the same
>> notification failure whenever anything on Kinetic breaks. :)
>>
>> Glancing at the package.xml for map_msgs, could it be that the accented
>> "é" in the author field (Stéphane Magnenat) is breaking unicode parsing,
>> not the exclamation points?
>>
>>
>> https://github.com/ros-planning/navigation_msgs/blob/jade-devel/map_msgs/package.xml
>>
>> I will look into sanitizing the characters so that we can support package
>> maintainers and authors with unicode characters in their names.
>>
>>
>> On Thu, Apr 7, 2016 at 9:32 AM, Tully Foote via ros-release <
>> ros-release at lists.ros.org> wrote:
>>
>>> I think the exclamation points in your name are breaking the unicode
>>> parsing on the debian target. Looking at the documentation the maintainer
>>> field only supports ASCII characters.
>>>
>>> The debian maintainer field format is defined by RFC822
>>> https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Maintainer
>>> There's a note about problems with full stop.
>>>
>>> And RFC822 can be seen here
>>> https://www.w3.org/Protocols/rfc822/3_Lexical.html
>>>
>>> I think it's likely that debian is more strict in it's interpretation of
>>> the formatting. And the quickest solution will be to remove the characters
>>> from the maintainer name. It could either be patched in the jessie specific
>>> branch in the release repo. The simplest solution is a rerelease without
>>> the characters.
>>>
>>> You could open an issue on
>>> https://github.com/ros-infrastructure/ros_buildfarm and we could look
>>> at sanitizing the characters in that field, but developing the correct
>>> replacement/substitituion policy may be a challenge.
>>>
>>> B) Sourcedebs are triggered on a 15 minute cycle since people want them
>>> to come out quickly. We typically will roll back any releases which fail
>>> their sourcedeb jobs since it's rarely a platform specific issue. And if
>>> the sourcedeb is not working we might as well pull it from the farm.
>>>
>>> Tully
>>>
>>> On Thu, Apr 7, 2016 at 6:27 AM, David Lu!! via ros-release <
>>> ros-release at lists.ros.org> wrote:
>>>
>>>> I am but a humble navigation code monkey, and am not trained in the
>>>> ways of the build farm. Can someone help me interpret what's going on
>>>> that's causing
>>>> A) A specific build to fail e.g.
>>>>
>>>> http://build.ros.org/job/Ksrc_dJ__map_msgs__debian_jessie__source/87/console
>>>> B) So many emails (see attached)
>>>>
>>>> _______________________________________________
>>>> ros-release mailing list
>>>> ros-release at lists.ros.org
>>>> http://lists.ros.org/mailman/listinfo/ros-release
>>>>
>>>>
>>>
>>> _______________________________________________
>>> ros-release mailing list
>>> ros-release at lists.ros.org
>>> http://lists.ros.org/mailman/listinfo/ros-release
>>>
>>>
>>
>> _______________________________________________
>> ros-release mailing list
>> ros-release at lists.ros.org
>> http://lists.ros.org/mailman/listinfo/ros-release
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ros.org/pipermail/ros-release/attachments/20160407/7dcac662/attachment-0003.html>