An Ontology-based approach for making Machine Learning systems Accountable

This manuscript was submitted as 'full paper' and should be reviewed along the usual dimensions for research contributions which include (1) originality, (2) significance of the results, and (3) quality of writing.

(1) The idea of using ontology to make machine learning explainable and interoperable is not new. Several studies have been published working on ontology-based ML for textual data or ontology-based deep learning. (Lai, Phung, et al. "Ontology-based Interpretable Machine Learning for Textual Data." 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 2020.; Confalonieri, Roberto, et al. "Trepan Reloaded: A Knowledge-driven Approach to Explaining Artificial Neural Networks." arXiv preprint arXiv:1906.08362 (2019).; Phan, Nhathai, et al. "Ontology-based deep learning for human behavior prediction with explanations in health social networks." Information sciences 384 (2017): 298-313.)

This paper emphasizes on the accountability of ML which seems a novel idea to me. However, the paper discussed the explainability, accountability, and interoperability, but did not address the key difference between these three concepts. It would be good the paper could feature accountability distinguishing from the explainability and interoperability. The most interesting and new part to me is the real-case application of the proposed approach. If the paper could elaborate more on the real-life use case such as what the advantages/limitations are when the ontology approach is applied in practice, is there any new challenges, or changes that need to be made to fit in the real-life application, etc.

Is the code for the method and application public available? I don’t find any link indicating to the source code (except a footnote for a Github issue).

(2)Significance:
The significance of the work is not sufficiently addressed in the article. In section 4, it is not clear to me which part of the proposed approach is original, which components are re-using existing methods/ontologies.
The entire section 3 is describing background knowledge on ontologies which does not contribute to the article a lot. It would be good if section 3 can be shortened or include some original ideas or significance of using ontology-based approaches.
Additionally, the evaluation of the approach is missing in the article. The same for the evaluation of the real-life use case application. It’s challenging to summarize the significance of the work without performance evaluation of the proposed approach.

(3) Quality of writing: The Structure of the paper is clear and easy for readers to follow the logic of the approach (predictive model development-semantic annotation-representation forecast). But I do feel “Discussion” is missing in the article. I expect the paper would also discuss the benefits/limitations of the approach, how it performs compared with other approaches, what has been observed from the real-life use case application, etc.

The readability of the paper needs to be improved by re-phrasing some sentences. In general, many sentences in the paper are too long to keep the key message that the author wants to deliver. Some parts of the paper need to be described in a more clear and understandable way.
The terminologies in the article are confusing sometimes. For example, in the abstract and introduction, the article describes trustworthy AI, XAI, then at the end, it states this paper solves the Machine Learning accountability issue. AI is not the same as ML. In my opinion, ML is a part of AI. The relationship between them should be clear.
In the Introduction and related work section, the article appears some terminologies without explanation. I assume the target readers should be people who have some background knowledge on ML, semantic web, knowledge graph, ontology. However, section 3 gives a very detailed background on ontologies which apparently for the people who are new to the field. It would be great to focus on a specific target group (e.g., either expert group or beginner group).

The authors propose 3 phased framework to support accountability of machine learning models:(1) Dev/deployment of predictive model; (2) Annotation of dev/deployment procedure and the result (3) manage annotation and exploitation by users. The underlying motivation is that The authors argue such ontological approach suitable because it combines features of web compliance, formalisation and automated reasoning capabilities.

Broadly, this seems to be an interesting work that could benefit the community. Albeit, there are considerable concerns to be addressed:

Detailed comments:
(1) Based on the accepted definition of accountability: (a) What are the standards to benchmark a ML algorithm against; (b) How can the standards be enforced in ML to ensure accountability; (c) How can accountability be measured against a standard (e.g the extent of compliance to a standard specification); (d)What happens when there are multiple standards? In my opinion,these questions should also be addressed by an ontology based a based accountability definition.

(2)The authors motivate their work by arguing that "adequate representation of data, processes and workflows involved in AI could make them more accountable". Wrt. the definition of accountability, adequate representation is only sufficient for traceability, and not necessarily compliance to expected standards or norms. Hence, "adequate representation..." may justify systemisation and explicit structure, but does little to motivate the approach as it does not directly map to the need to demonstrate accountability.

(3)Its a bit worrying that all reviewed work is on explainability and nothing on accountability. At least some related work in the area of accountability (albeit not related to semantic web or ML) and pointers to how semantic web may leverage to improve algorithms may suffice. As a parallel example, accountability has become a central principle in demonstrating compliance to data protection regulations.

(4)Good background provided on ontology. But little on why we need to tie ontology to ML to improve accountability - particularly, more on this could be done in section 3.3.

(5)In page 4, where is the rationale/justification that the proposed approach is ML model agnostic. For example, what sort of ML models are more or less amenable to accountability - or are they all equally amenable. Do users understand how such models work? Would such understanding foster accountability (i.e the relation between explainability and accountability being enhanced by an ontology).

(6)Would be great to see more on the utility of the approach. For example, how can the annotation be supported in practice given that it will be manual and intensive.

(7)Following the accepted definition of accountability, how does the 3 phased framework connote accountability, particularly wrt. standards (as defined).

(8)For the exploitation step, in practice it will often involve new results generated for end-users who were not involved with building the ML prediction model. How do you expect to annotate such results at runtime.

(9)Alot of what is proposed during semantic annotation phase seems to focus on traceability and not really accountability wrt. a standard.

(10)Is the 3 step model adjustable to nuances of the nature of ML problem. For example, how it adapts to supervised/unsupervised, regression/classification. Would accountability requirement differ depending on the ML approach. If different, how does Figure 2 account for such differences. Furthermore, is figure 2 complete, consistent with a subset of ML problems or can be generalised- How and why?

(11)The authors mention 122 predictive models were used in the evaluation. What is the breakdown of these models in terms of their type, training, testing and validation approaches. Better insights on methodology will suffice.

The current study makes use of ontologies for ensuring the accountability of Machine Learning systems. They show the feasibility of the proposed approach in a real-world scenario.

*Section (Abstract)*

The abstract of the article is very generic and does not give a precise overview of the proposed approach and the contribution of the paper.

*Section (Introduction)*

The authors say that “Even though the maturity of the Artificial Intelligence (AI) technologies is rather advanced nowadays, according to McKinsey1, its adoption, deployment and application is not as wide as it could be expected”

I would like to have a little bit more concrete view of that with examples. This is not a problem of space to write more details here.

Authors write this:
“According to [8], during the past decade, there has been an increase in AI systems based on black-box models, that is, models that hide their internal logic to the user.”

Maybe the authors could be a bit more precise here. Also, in this study the models used are very basic, how is explainability a problem in the specific case of the author? The authors should convince the reader about the problem first and then state their contribution.

The authors in the introduction say that the proposed methods so far are post-hoc, is it good or not, it is not very much clarified since the proposed approach itself performs some post-hoc explanations (even in that case the explanations are not generated).

*Section (Related Work)*

The related work is not complete w.r.t. the explainability aspect in Knowledge Graphs. For example, cite:
Ilaria Tiddi, Freddy Lécué, Pascal Hitzler: Knowledge Graphs for eXplainable Artificial Intelligence: Foundations, Applications, and Challenges. Studies on the Semantic Web 47, IOS Press 2020
Freddy Lécué: On the role of knowledge graphs in explainable AI. Semantic Web, 2020

*Section 3 (On the Role of Ontologies)*

These are the basic concepts of ontologies, ontology development, and reuse. It is very much unclear to me why these concepts are discussed here in the Journal which is the core of the Semantic Web community. The authors should motivate why it is being discussed here.

*Section 4 (Making Machine Learning Accountable)*

The framework given in Fig. 1 is the general framework. What are the added value and the novelty of this framework and the corresponding explanation of the framework is not very clear to me.

However, the interesting explanation starts in section 4.2.1 where the authors are concretely introducing the existing ontologies for the use-case described later on.

The actual contribution of the authors starts on page 8 section 5.
- First of all the authors should really define concretely what does it mean by accountability in their scenario?
- How do they plan on achieving a trustworthy system?
- What are the contributions of their approach?

My overall impression of the proposed approach is that there is data for a particular scenario. The authors are using KNN for making some predictions (the explanation is mostly from an implementation point of view). The output is then fed to the existing EEPS ontology which is then queryable. I am still wondering how the goal discussed at the beginning of the study is met here. I am not very clear about the contribution of the approach as well as the novelty.