[ros-release] Build Farm Spam and Debugging

Thu Apr 7 17:55:25 UTC 2016

Before drawing any conclusions please take a close look to the actual error
message:

```
Traceback (most recent call last):
  File "/tmp/ros_buildfarm/scripts/release/get_sources.py", line 34, in
<module>
    sys.exit(main())
  File "/tmp/ros_buildfarm/scripts/release/get_sources.py", line 30, in main
    args.os_name, args.os_code_name, args.source_dir)
  File "/tmp/ros_buildfarm/ros_buildfarm/sourcedeb_job.py", line 52, in
get_sources
    pkg = parse_package(sources_dir)
  File "/usr/lib/python3/dist-packages/catkin_pkg/package.py", line 370, in
parse_package
    return parse_package_string(f.read(), filename, warnings=warnings)
  File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 183:
ordinal not in range(128)
```

Decoding a non-ASCII character when then codec is set to `ascii` can't
work. Since `!` is a valid ASCII character that can't be the reason. So it
is likely the `é`.

What needs to change here? Well, the first important fact to note is that
the sourcedeb jobs for the same package pass on all Ubuntu platforms. So
obviously the buildfarm as well as catkin_pkg are able to handle this.

Why is it failing for this Debian job then? Debian has just recently been
added and there must be something different. Looking at the console output
again the following warnings just a few lines above should be enough:

```
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
```

While the job tries to use UTF-8 it fails to do so on Debian and falls back
to the standard locale "C". And with that locale it is simply impossible to
decode the special character.

As a conclusion the recent modification to enable Debian jobs (
https://github.com/ros-infrastructure/ros_buildfarm/commit/5106e704ecdb2d55ac513da66ec8d699731bb859#diff-8859c2a0d6adc3dc403698f904632818)
needs fixing since it is not enabling the UTF-8 locale as expected.

Cheers,
- Dirk

On Thu, Apr 7, 2016 at 10:07 AM, Jackie Kay via ros-release <
ros-release at lists.ros.org> wrote:

> I also find the build farm spam quite annoying. I get the same
> notification failure whenever anything on Kinetic breaks. :)
>
> Glancing at the package.xml for map_msgs, could it be that the accented
> "é" in the author field (Stéphane Magnenat) is breaking unicode parsing,
> not the exclamation points?
>
>
> https://github.com/ros-planning/navigation_msgs/blob/jade-devel/map_msgs/package.xml
>
> I will look into sanitizing the characters so that we can support package
> maintainers and authors with unicode characters in their names.
>
>
> On Thu, Apr 7, 2016 at 9:32 AM, Tully Foote via ros-release <
> ros-release at lists.ros.org> wrote:
>
>> I think the exclamation points in your name are breaking the unicode
>> parsing on the debian target. Looking at the documentation the maintainer
>> field only supports ASCII characters.
>>
>> The debian maintainer field format is defined by RFC822
>> https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Maintainer
>> There's a note about problems with full stop.
>>
>> And RFC822 can be seen here
>> https://www.w3.org/Protocols/rfc822/3_Lexical.html
>>
>> I think it's likely that debian is more strict in it's interpretation of
>> the formatting. And the quickest solution will be to remove the characters
>> from the maintainer name. It could either be patched in the jessie specific
>> branch in the release repo. The simplest solution is a rerelease without
>> the characters.
>>
>> You could open an issue on
>> https://github.com/ros-infrastructure/ros_buildfarm and we could look at
>> sanitizing the characters in that field, but developing the correct
>> replacement/substitituion policy may be a challenge.
>>
>> B) Sourcedebs are triggered on a 15 minute cycle since people want them
>> to come out quickly. We typically will roll back any releases which fail
>> their sourcedeb jobs since it's rarely a platform specific issue. And if
>> the sourcedeb is not working we might as well pull it from the farm.
>>
>> Tully
>>
>> On Thu, Apr 7, 2016 at 6:27 AM, David Lu!! via ros-release <
>> ros-release at lists.ros.org> wrote:
>>
>>> I am but a humble navigation code monkey, and am not trained in the
>>> ways of the build farm. Can someone help me interpret what's going on
>>> that's causing
>>> A) A specific build to fail e.g.
>>>
>>> http://build.ros.org/job/Ksrc_dJ__map_msgs__debian_jessie__source/87/console
>>> B) So many emails (see attached)
>>>
>>> _______________________________________________
>>> ros-release mailing list
>>> ros-release at lists.ros.org
>>> http://lists.ros.org/mailman/listinfo/ros-release
>>>
>>>
>>
>> _______________________________________________
>> ros-release mailing list
>> ros-release at lists.ros.org
>> http://lists.ros.org/mailman/listinfo/ros-release
>>
>>
>
> _______________________________________________
> ros-release mailing list
> ros-release at lists.ros.org
> http://lists.ros.org/mailman/listinfo/ros-release
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ros.org/pipermail/ros-release/attachments/20160407/b7dc80e0/attachment-0003.html>