RATIONALE: A Security and Safety Testing Ontology for Machine Learning-based systems

Tracking #: 3164-4378

This paper is currently under review
Anne-Laure Wozniak
Raúl Mazo
Sergio Segura

Responsible editor: 
Rafael Goncalves

Submission type: 
Ontology Description
The testing of machine learning (ML) based systems has gained interest in recent years. In particular, the development of critical systems incorporating ML models, such as autonomous vehicles, has raised concerns about their safety and security. Furthermore, given the diversity of models and systems using ML, it is essential to have a good understanding of the testing techniques that can be applied to these models, in order to test them properly and thoroughly. However, while several secondary studies have been published on the subject, there is currently no comparative framework of these testing methods, that would allow practitioners to find the most suitable methods for their needs. In this article, we present RATIONALE: a secuRity and sAfety TestIng ONtoLogy for mAchine LEarning-based systems. An ontology defines the concepts, relationships and individuals that are relevant for modeling a domain. Thus, RATIONALE allows the sharing and reuse of knowledge around the safety and security of machine learning-based systems, in terms of threats, defences, and testing techniques. To make our approach helpful in practice, RATIONALE has been integrated into a web application to make it easily and automatically queryable by target users from a wide range of backgrounds. The completeness and validity of this ontology was assessed against the outcomes of secondary studies on the subject. A usability test was also conducted to assess the usefulness and usability of the web application. Overall, the results support the value of RATIONALE to effectively assists testers on the development and maintenance of ML test methods and tools.
Full PDF Version: 
Under Review