<div dir="ltr"><br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="white" link="blue" vlink="purple" lang="EN-US">
<div><div><div class="h5"><div><p class="">
<br>
<br>
<br>
On <a href="tel:31.08.2013%2004" value="+13108201304" target="_blank">31.08.2013 04</a>:39, Edwards, Shaun M. wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<p class="">All,<u></u><u></u></p>
<p class=""> <u></u><u></u></p>
<p class="">We have received feedback from the users of ROS-Industrial on two issues that I think are important to the larger community. We have taken preliminary steps to address these issues, but in no way would we consider these the correct or
permanent solutions. It is for this reason, I am address the ROS user’s group to solicit feedback and discussion about these two issues:<u></u><u></u></p>
<p class=""> <u></u><u></u></p>
<p><u></u><span>1.<span style="font:7pt "Times New Roman"">
</span></span><u></u>Stack/Meta-Package/Package Status – Many people have commented that it is hard to know the true status of a package (whether the code is complete or in development). The existence of a wiki is not an indicator, as several packages in
ROS (including some of our own) are released early (i.e. agile development). For this reason we have started identifying the status of a package on our wiki pages (see:
<a href="http://ros.org/wiki/Industrial/Software_Status" target="_blank">http://ros.org/wiki/Industrial/Software_Status</a> ). Here is an example of a stack/meta-package that has been marked with its current status:
<a href="http://ros.org/wiki/industrial_core" target="_blank">http://ros.org/wiki/industrial_core</a> . This is only a start to what I think needs to be done, but it solves and urgent need for us and all the developers that are using our software. I would like to see this
status or something similar used by the larger community (what to you guys think).<u></u><u></u></p>
<p><u></u><span>2.<span style="font:7pt "Times New Roman"">
</span></span><u></u>Code quality/testing/metrics – Several users have asked for this type of information about our packages. This is one of the (not the only) reasons we set up a Jenkins server specifically for ROS-Industrial (see
<a href="http://rosindustrial.org/news/2013/8/13/jenkins-system-for-ros-industrial-repositories" target="_blank">
http://rosindustrial.org/news/2013/8/13/jenkins-system-for-ros-industrial-repositories</a> ). The ROS community already utilizes Jenkins servers for continuous integration and debain builds, but code metrics are missing (even some as simple as how many compiler
warnings are generated). We would like to see this kind of data rolled into official ROS Jenkins servers. Is this a need for those in the large community?<u></u><u></u></p>
<p class=""></p></blockquote></div></div></div></div></blockquote><div><br></div><div>My 2 cents,<br><br></div><div>From a user perspective, I'm mostly interested in these qualities:<br>- Good and stable API<br>- Well tested<br>
</div><div></div><div>- Well documented<br>- Actively maintained. Not necessarily actively developed.<br><br></div><div>Except testing metrics like coverage, most of the above qualities are already accessible through the ROS wiki.<br>
</div><div><br></div><div>I usually also check how many downstream projects/packages depend on the project of interest, as this correlates well to some of the above points.<br></div><div><br></div><div>From a developer perspective, code quality metrics like the work of Johannes Kuehn make more sense.<br>
<br>One might argue that good code metrics should also bias user choices, as a project with poor code metrics might be difficult to move forwards. However, projects with good APIs, documentation, testing and a sizable user base will have enough traction to overcome parts of it with a poor implementation (and are probably less likely to have bad code metrics to start with).<br>
<br></div><div>Concerning Shaun's proposal, I'd move "The software has some level of unit-testing or has been used on production level code in the past" from production to development, and make the production statement much more strict, reading something like "The software has a comprehensive test suite and documentation, and is being used in production-level code."<br>
<br></div><div>Best,<br><br></div><div>Adolfo.<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="white" link="blue" vlink="purple" lang="EN-US">
<div><div><div class="h5"><blockquote style="margin-top:5pt;margin-bottom:5pt"><p class=""> <u></u><u></u></p>
<p class="">Thanks for you time,<u></u><u></u></p>
<p class=""> <u></u><u></u></p>
<p class=""> <u></u><u></u></p>
<p class=""> <u></u><u></u></p>
<p class=""><span style="color:rgb(31,73,125)">Shaun Edwards</span><u></u><u></u></p>
<p class=""><span style="color:rgb(31,73,125)">Senior Research Engineer</span><u></u><u></u></p>
<p class=""><span style="color:rgb(31,73,125)">Manufacturing System Department</span><u></u><u></u></p><br></blockquote></div></div></div></div></blockquote></div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Sun, Sep 1, 2013 at 7:29 PM, Edwards, Shaun M. <span dir="ltr"><<a href="mailto:sedwards@swri.org" target="_blank">sedwards@swri.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="white" link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="color:#1f497d">Thibault,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d">Thanks for the feedback. <u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d">I agree a status label does not communicate everything, but it does communicate the first bit of information which is what the developer/maintainer thinks the code quality is. If the developer/maintainer has
labeled their own code <production quality, then any user would quickly decide not to use it for their production system. Of course if they have labeled it “production quality”, then the user still has a lot of work to do (as you point out).<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d">As for metrics I can tell you after a year and a half of trying to sell ROS the industrial community, these will help. If for no other reasons, metrics will help potential users weed out the obvious bad packages.
A key metric that I haven’t seen anywhere (even in Tully’s example) is unit test coverage. This will soon be added to the ROS-Industrial Jenkins toolkit and in my opinion is very helpful in understanding the quality of a particular piece of software. I do
agree that we should follow the lead of other open source projects, like the ones you reference.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d">Finally, I would like to point out that we need a solution to at least some of these issues sooner rather than later. My goal in sending out this email was to gage community interest. My hope is that we can
come up with an incremental plan that solves problems sooner rather than later. I would not support an approach that attempts to create an entire solution up front. Such a large project would not be easily supported by volunteers and even if it had funding
would take too long to complete.<u></u><u></u></span></p><div class="im">
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<div>
<p class="MsoNormal"><span style="color:#1f497d">Shaun Edwards<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d">Senior Research Engineer<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d">Manufacturing System Department<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><a href="http://robotics.swri.org" target="_blank">http://robotics.swri.org</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><a href="http://rosindustrial.swri.org/" target="_blank">http://rosindustrial.swri.org/</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><a href="http://ros.swri.org/" target="_blank">http://ros.swri.org</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><a href="https://groups.google.com/group/swri-ros-pkg-dev/boxsubscribe" target="_blank">Join the ROS-Industrial Developers List</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d">Southwest Research Institute<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><a href="tel:210-522-3277" value="+12105223277" target="_blank">210-522-3277</a><u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
</div><div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> <a href="mailto:ros-users-bounces@code.ros.org" target="_blank">ros-users-bounces@code.ros.org</a> [mailto:<a href="mailto:ros-users-bounces@code.ros.org" target="_blank">ros-users-bounces@code.ros.org</a>]
<b>On Behalf Of </b>Thibault Kruse<br>
<b>Sent:</b> Sunday, September 01, 2013 6:20 AM<br>
<b>To:</b> <a href="mailto:ros-users@code.ros.org" target="_blank">ros-users@code.ros.org</a><br>
<b>Subject:</b> Re: [ros-users] Software Status Reporting and Custom Builds<u></u><u></u></span></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hi,<br>
<br>
I believe it would be helpful to state more clearly in what way any such data would be used, and for what purpose, and by what kind of users.
<br>
<br>
Knowing the true status of a package is a hard problem in general, it is IMO not solvable by simply sticking a label to a package. Any label is only as good and accountable as the governing authority producing those labels, IMO. And deciding whether to trust
the guy having set the label can be just as hard.<br>
<br>
As a different example, knowing a project is a project of the Apache Foundation tells you a lot because to become a project at Apache, you need to go through an incubation phase, proving that there is a maintainer, a community, and plentiful other of metadata
to indicate project health.<br>
<br>
Since we do not have that, checking the indicators of package quality remains a lot of hard work. One needs to check the documentation (complete, well-written, up-to-date), the project history, the issue tracker state, the amount of regression tests, which
company is using a package, and maybe more.<br>
<br>
There cannot be a mere "status" label to replace all that activity to figure out the status of a project, IMO.<br>
<br>
<br>
<br>
Regarding Code quality/testing/metrics:<br>
Again it would be more useful if any person having a specific interest in any such data would speak up for themselves. I believe this kind of data can only ever be valuable for the active maintainers/developers of a package. I cannot imagine any person saying:
"I was thinking about using package XYZ, but since I could not see the number of compiler warnings/style warnings/complexity measures in the wiki, i decided against investigating further".<br>
<br>
I can believe a person saying: "I'd love to introduce package XYZ to my company, but I struggle with persuading my boss that the quality of such an Open-Source package can be relied upon." But then, simple code metrics would probably not help with that.<br>
<br>
Selling ROS (even for free) to a company is a marketing problem, it is about maintaining a community, a pretty website and advertising, finding reference customers speaking up for the product, and so on.<br>
<br>
<br>
<br>
Instead of imagining a future world with useful metrics, it might help to point at real-world projects in the wild already using metrics, and copy that for ROS if enough people prefer investing time in that than in, say, ROS2.0.
<br>
E.g. compare the list of projects and the stats they offer at <br>
<a href="http://freecode.com/" target="_blank">http://freecode.com/</a><br>
<a href="https://pypi.python.org/pypi" target="_blank">https://pypi.python.org/pypi</a><br>
<a href="https://analysis.apache.org/dashboard/index" target="_blank">https://analysis.apache.org/dashboard/index</a><br>
<br>
<br>
regards,<br>
Thibault<br>
<br>
<br>
<br>
<br>
On <a href="tel:31.08.2013%2004" value="+13108201304" target="_blank">31.08.2013 04</a>:39, Edwards, Shaun M. wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">All,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">We have received feedback from the users of ROS-Industrial on two issues that I think are important to the larger community. We have taken preliminary steps to address these issues, but in no way would we consider these the correct or
permanent solutions. It is for this reason, I am address the ROS user’s group to solicit feedback and discussion about these two issues:<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p><u></u><span>1.<span style="font:7.0pt "Times New Roman"">
</span></span><u></u>Stack/Meta-Package/Package Status – Many people have commented that it is hard to know the true status of a package (whether the code is complete or in development). The existence of a wiki is not an indicator, as several packages in
ROS (including some of our own) are released early (i.e. agile development). For this reason we have started identifying the status of a package on our wiki pages (see:
<a href="http://ros.org/wiki/Industrial/Software_Status" target="_blank">http://ros.org/wiki/Industrial/Software_Status</a> ). Here is an example of a stack/meta-package that has been marked with its current status:
<a href="http://ros.org/wiki/industrial_core" target="_blank">http://ros.org/wiki/industrial_core</a> . This is only a start to what I think needs to be done, but it solves and urgent need for us and all the developers that are using our software. I would like to see this
status or something similar used by the larger community (what to you guys think).<u></u><u></u></p>
<p><u></u><span>2.<span style="font:7.0pt "Times New Roman"">
</span></span><u></u>Code quality/testing/metrics – Several users have asked for this type of information about our packages. This is one of the (not the only) reasons we set up a Jenkins server specifically for ROS-Industrial (see
<a href="http://rosindustrial.org/news/2013/8/13/jenkins-system-for-ros-industrial-repositories" target="_blank">
http://rosindustrial.org/news/2013/8/13/jenkins-system-for-ros-industrial-repositories</a> ). The ROS community already utilizes Jenkins servers for continuous integration and debain builds, but code metrics are missing (even some as simple as how many compiler
warnings are generated). We would like to see this kind of data rolled into official ROS Jenkins servers. Is this a need for those in the large community?<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Thanks for you time,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d">Shaun Edwards</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d">Senior Research Engineer</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d">Manufacturing System Department</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d"><a href="http://robotics.swri.org" target="_blank">http://robotics.swri.org</a></span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d"><a href="http://rosindustrial.swri.org/" target="_blank">http://rosindustrial.swri.org/</a></span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d"><a href="http://ros.swri.org/" target="_blank">http://ros.swri.org</a></span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d"><a href="https://groups.google.com/group/swri-ros-pkg-dev/boxsubscribe" target="_blank">Join the ROS-Industrial Developers List</a></span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d">Southwest Research Institute</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#1f497d"><a href="tel:210-522-3277" value="+12105223277" target="_blank">210-522-3277</a></span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif""><br>
<br>
<br>
<u></u><u></u></span></p>
<pre>_______________________________________________<u></u><u></u></pre>
<pre>ros-users mailing list<u></u><u></u></pre>
<pre><a href="mailto:ros-users@code.ros.org" target="_blank">ros-users@code.ros.org</a><u></u><u></u></pre>
<pre><a href="http://lists.ros.org/mailman/listinfo/ros-users" target="_blank">http://lists.ros.org/mailman/listinfo/ros-users</a><u></u><u></u></pre>
</blockquote>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif""><u></u> <u></u></span></p>
</div></div></div>
</div>
<br>_______________________________________________<br>
ros-users mailing list<br>
<a href="mailto:ros-users@code.ros.org">ros-users@code.ros.org</a><br>
<a href="http://lists.ros.org/mailman/listinfo/ros-users" target="_blank">http://lists.ros.org/mailman/listinfo/ros-users</a><br>
<br></blockquote></div><br></div><br clear="all"><br>-- <br>Adolfo Rodríguez Tsouroukdissian<br>Senior robotics engineer<br><a href="mailto:adolfo.rodriguez@pal-robotics.com" target="_blank">adolfo.rodriguez@pal-robotics.com</a><br>
<a href="http://www.pal-robotics.com/" target="_blank">http://www.pal-robotics.com</a><br><br>PAL ROBOTICS S.L<br>c/ Pujades 77-79, 4º4ª<br>08005 Barcelona, Spain.<br>Tel. +34.93.414.53.47<br>Fax.+34.93.209.11.09<br>Skype: adolfo.pal-robotics<br>
<a href="http://www.facebook.com/palrobotics1" target="_blank">Facebook</a> - <a href="http://twitter.com/#%21/palrobotics" target="_blank">Twitter</a> - <a href="http://www.youtube.com/user/PALRobotics" target="_blank">PAL Robotics YouTube Channel</a><br>
<br>AVISO DE CONFIDENCIALIDAD: Este mensaje y sus documentos adjuntos, pueden contener información privilegiada y/o confidencial que está dirigida exclusivamente a su destinatario. Si usted recibe este mensaje y no es el destinatario indicado, o el empleado encargado de su entrega a dicha persona, por favor, notifíquelo inmediatamente y remita el mensaje original a la dirección de correo electrónico indicada. Cualquier copia, uso o distribución no autorizados de esta comunicación queda estrictamente prohibida.<span style="border-collapse:collapse;font-size:13px"><span style="border-collapse:collapse;font-size:13px"><span style="font-size:13px"><tt><font face="'times new roman', serif"><font color="#333333"><br>
<br></font></font></tt></span></span></span>CONFIDENTIALITY NOTICE: This e-mail and the accompanying document(s) may contain confidential information which is privileged and intended only for the individual or entity to whom they are addressed. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of this e-mail and/or accompanying document(s) is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender at the above e-mail address.