[ros-users] [Discourse.ros.org] [Quality Assurance] What quality metrics do we need to make packages quality visible?

tyagikunal ros.discourse at gmail.com
Mon Mar 5 00:00:30 UTC 2018



[quote="fkromer, post:4, topic:3985"]
Fuzzy testing (in the most basic form) is not really dynamic analysis because the entities are considered as black box (not white Box)
[/quote]
Should Fuzzy testing be moved to a different block containing black-box fuzzing as a sub-component?

Could you please correct me if I'm wrong about black-box fuzzy testing:
* It would be similar to using hypothesis (python package) to create messages and checking output
* It would have templates for different message-types, not for node-specific requirements. Eg: Range input for float message, 3 different ranges for acceleration, etc.

Would node-specific behavior such as accepting messages in strictly increasing time order be included in black-box testing? This obv uses a small but nonetheless inner knowledge of how a node should work.

IMO, white-box fuzzing might be helpful in checking corner cases like a typical test but more on the lines of model-based testing, ie. developed specially for the package or node, specially for non-infrastructure packages. On the other-hand, black-box fuzzers would be generic and their use would involve minimal (one or a few lines only) involvement by the maintainers.

[quote="fkromer, post:4, topic:3985"]
could be way easier than in ROS1
[/quote]
As for development, I think ROS2 with node-lifetimes would make such testing both simple and a necessity. However, a generic fuzzer on the messages, services (and actions) without lifetimes would be similar for ROS1 and ROS2 unless my assumptions are wrong. :confused: 
pyros-dev should make the testing more or less similar, right?





---
[Visit Topic](https://discourse.ros.org/t/what-quality-metrics-do-we-need-to-make-packages-quality-visible/3985/6) or reply to this email to respond.




More information about the ros-users mailing list