[ros-users] [Discourse.ros.org] [Quality Assurance] ROS pkg quality metrics: analysing ROS wiki pages

[gavanderhoorn]

This post fits in with the current topic "Making package quality visible in ROS" which is being discussed in the quality working group meetings.

This post is about analysing wiki pages for packages that provide nodes. Other types of packages will most likely need other types of analysis and / or metrics.

---

It's doubtful whether we can actually assess the quality of a wiki page for a ROS package (as that would most likely require an understanding of the text), but looking at some of the tools available, I have the impression that we might actually be able to say at least *something* quantitative about a wiki page.

First: it turns out there are actually (at least) two pages on the wiki that say something about what a good quality wiki page should contain, or at least what a wiki page consistent with other pages should contain:

  - [DocumentationPolicy](http://wiki.ros.org/DocumentationPolicy)
  - [StyleGuide](http://wiki.ros.org/StyleGuide)

Most of this is style related: use consistent naming, try to avoid introducing new styling / markup / layouts, etc. Analysing / checking this might be possible with recent deep learning techniques, but I'll skip that for now. Things that should be (I believe) machine checkable without too much work are:

 1. presence of customary / required sections
 1. ROS API

#### Sections

This will be up for discussion, but a "good enough" wiki page of a package should probably contain at least the following sections (not necessarily in this order):

 1. Package Header (auto-generated)
 1. Overview/Introduction
 1. Table of Contents
 1. Requirements
 1. Installation instructions (could be auto-generated, [ros-infrastructure/roswiki#121](https://github.com/ros-infrastructure/roswiki/issues/121))
 1. Report a Bug (could be auto-generated)
 1. Tutorials (if there are none, make that explicit)
 1. ROS API (see below)

Checking for this is not too difficult: MoinMoin provides access to page contents through the [Page class](https://moinmo.in/MoinAPI/Examples#Page). As MoinMoin pages are plain text files stored on disk, it should also be possible to check them directly (with a script running periodically fi).

#### ROS API

A properly documented package should list the ROS API for all provided nodes. This ROS API consists of topics (published, subscribed), services (provided, called), actions (provided, called), parameters (read, written) and TF frames (required, provided). All of these can be documented using a provided ClearSilver template (`NodeAPI`) and they should be documented per node.

Similar to checking for sections, presence of all of these should be checkable, either via the `Page` class or scanning the page source on disk directly.

Some examples of pages that document the ROS API of their nodes (in no particular order):

 - [move_base](http://wiki.ros.org/move_base?distro=indigo)
 - [single_joint_position_action](http://wiki.ros.org/single_joint_position_action?distro=indigo)
 - [abb_driver](http://wiki.ros.org/abb_driver)
 - [stepback_and_steerturn_recovery](http://wiki.ros.org/stepback_and_steerturn_recovery)
 - [camera_info_manager](http://wiki.ros.org/camera_info_manager?distro=indigo)
 - [polled_camera](http://wiki.ros.org/polled_camera?distro=indigo)
 - [move_slow_and_clear](http://wiki.ros.org/move_slow_and_clear?distro=indigo)


To make sure that the *entirety* of a node's ROS API is documented, it could be extracted using suitable queries against [HAROS v3](https://github.com/git-afsantos/haros) reversed [metamodel](https://github.com/git-afsantos/haros#user-defined-queries) of a package and then checked against the `NodeAPI` template parameters provided on the wiki page.

#### Current status

None of the above is currently checked by the wiki, the buildfarm or any automated system.





---
[Visit Topic](https://discourse.ros.org/t/ros-pkg-quality-metrics-analysing-ros-wiki-pages/4442/1) or reply to this email to respond.