A Knowledge-based Strategy For XAI:The Explanation Graph

Title of the Paper: A Knowledge-based Strategy For XAI: The Explanation Graph

0) Topic:

This paper reports on the use of explanation-graphs for semantic-based explanation generation. It discusses the integration of three knowledge fragments (AI-based system output, private knowledge, public knowledge) to create explanation graphs which themselves can be used as a basis to generate natural language explanations which can be tailored to the audience. The authors use a system to monitor dietary restrictions to explain this proposed hybrid explanation pipeline and evaluate how different explanations can lead to a change in behavior.

1) Originality:

Different searches have shown no indication of plagiarism. Consequently, the paper is original work.

The hybrid-approach described in the paper integrating different knowledge fragments, e.g. public knowledge (i.e. external ontologies) and private resources, to generate explanation graphs which are the starting point for tailored natural language explanations (based on templates), constitutes a (relatively) niche research direction in XAI. In comparison to solely ML driven model-agnostic post-hoc explanations the explanation pipeline described in the paper has the added benefit of integrating human knowledge engineering into ML. This hybrid approach combined with a short empirical analysis on explanation evaluation in section 5 justifies publication.

2) Background and Related Work:

On page 2 legal aspects (the GDPR) are mentioned briefly but are no elaborated on. Legal aspects are, however, precisely the drivers for the need to make so-called "black boxes" transparent, or to make their results re-traceable and thus understandable for human experts. This is especially important in case in “informed consent” in healthcare, which is used as an example on page 5. Therefore, the authors should at least provide the interested reader with references to current work on this topic, e.g. the very recent work by Stoeger et al https://doi.org/10.1145/3458652 and Bibal et al, https://doi.org/10.1007/s10506-020-09270-4.

On page 2 the concept of Causality is mentioned as desideratum of automated decision-making. The authors should also mention the related concept of “Causability” as concept for measuring the quality of explanations by Holzinger et al, https://doi.org/10.1016/j.inffus.2021.01.008

As the authors discuss how explanations can influence the audience if they are “tailored” and how explanations can influence behavior (section 5) the seminal paper by Miller https://doi.org/10.1016/j.artint.2018.07.007 should be provided.

3) Methodology:

The methodology used is introduced in a succinct manner:
Section 2 describes related work in XAI and on knowledge graphs which constitutes the theoretical basis of this paper. It briefly outlines basic XAI terminology and prior research in ontologies. Section 3 and 4 illustrate how knowledge fragments can be used to build an explanation graph and how this explanation graph can be rendered as a natural language explanation. The use of the dietary restriction example is useful to grasp the different theoretical steps. Section 5 focuses on this use case and contains an empirical analysis how different explanations are more or less useful in changing the behavior of the audience. In general, the methodology is sound but the empirical section on explanation evaluation (page 17 seq) and the theory what constitutes a “good” explanation could be more in depth e.g. mentioning prior research in social sciences (see comment on background above). There are also other minor methodological errors: On page 2 and 13 “economic information” is mentioned as “sensitive data” but most privacy laws (e.g. Art 9 para 1 GDPR in Europe) do not regard economic information as “sensitive data”. Reference [46] seems to be identical to [9].

4) Results:

The results are present in sections 6 and 7: The authors illustrate how explanation graphs can be enriched with other private or public knowledge sources, can be rendered into different formats and allow full control over the rendered explanations. They describe the effort in domain and user modeling by domain experts as the main bottleneck of the template-based approach. Section 6 also mentions the main challenges which the authors want to address in the future. The results are presented clearly but the final sections could have been more in-depth as they feel a bit “rushed” and superficial without adding much additional scientific value.
Nonetheless, this reviewer finds the results – the hybrid approach undertaken, the description of the explanation graph and template generation, the brief empirical analysis of different explanations – are important for the international research community.

5) Qualitative Evaluation:

The paper is easy to understand and clearly readably because it does not use unnecessary acronyms or complicated sentence structuring. Regardless, there are still many typos, editing and translation errors which must be remedied (three examples: page 1: “undermines the trustworthy of”, page 8 “cliniciancontaining”; page 13 “the the communication”). Please carefully revise the whole paper take a look at typos and minor language issues, maybe a native English speaker can help.

Generally a pleasant paper and this reviewer recommends acceptance and hopes that the comments given helps the authors to further improve the paper. Good work!

The main novelty of the paper is the idea of explanation graph, which is obtained from several ontologies by matching the results of an intelligent system against concepts in one of the ontologies, and by carrying out alignment processes with other ontologies representing public and private knowledge regarding the area and the user, respectively. The graph itself is useful in order to provide an explanation but, in addition, a natural language explanation is generated from the graph using templates.
The design and integration of the different modules for using explanation graphs in a particular application has to be developed mostly manually, though existing ontologies may be employed. Once built, the explanations are provided automatically.
The proposal has the potential to be a significant advance in providing tools for XAI. The authors illustrate this with a use case and a study of the impact on real users in this setting, with an experimental design involving 120 users divided into Intervention and Control groups, and performing an statistical post-test to support their conclusions.
Overall, I think that the paper deserves to be accepted for publication. My main concern is about quality of English. There are many mistakes in the paper, so the paper requires a careful review before being published. Just as some example, in the abstract: "research is dramatically grown" -> "research has dramatically grown"; "how it can work has a bridge" -> "how it can work as a bridge". In the remaining of the paper, "among the others" -> "among others"; "rules can formalizes" -> "rules can formalize"; "describes a use cases" -> "describes a use case"; "work as bridge" -> "work as a bridge"; "cliniciancontaining"; "udnerlying"; "in the defining" -> "in defining"; "One of the possible methodology" -> "One of the possible methodologies"; "explanation as to be provided" -> "explanation has to be provided"; "the the" (several times); "A A template"; etc.

The paper presents a graph-based representation of AI systems explanation. The goal is to provide a way to represent conveniently the content of this explanation. First, such a graph is extracted from the AI model and then the graph feeds a Natural Language Generation tool, in this case TS4NLE.

The paper is well written in the sense explanations are clear. Unfortunately, the paper has many typos and must be proofread before publication. I will end this review with few typos (but they are too numerous to be listed exhaustively). Some figures may be reduced.

The bibliography at first sight seems complete, even if with a field as XAI it is difficult to pretend to completeness. Nevertheless, ref [1] cannot be cited as an example of XAI: it compares different software for playing games. Even if explainability has been a concern for system experts since the beginning, XAI (and the acronym) are more recent. But reading some of the papers in the bibliography and paper which cites them, I discovered previous work that has the same goal as this paper, in particular the papers of Ismail Baaj and his supervisors:
- Baaj, I. & Poli, J. (2019). Natural Language Generation of Explanations of Fuzzy Inference Decisions. FUZZ-IEEE 2019 International Conference on Fuzzy Systems, June 2019, New Orleans, USA
- Baaj, I., Poli, J. & Ouerdane, W. (2019). Some Insights Towards a Unified Semantic Representation of Explanation for eXplainable Artificial Intelligence (XAI). 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence @ INLG 2019 International Natural Language Generation Conference, October 2019, Tokyo, Japan
- Baaj, I., Poli, J., Ouerdane, W. & Maudet, N. (2021). Representation of Explanations of Possibilistic Inference Decisions @ ECSQARU 2021 European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty, September 2021, Prague, Czechia .

They also argue for a representation of explanations, but they based their work on conceptual graphs and they also as perspective, tell that they will use domain knowledge (that is comparable to two sources of knowledge in this work). They also provide adaptations of CG for possibilistic inference rules. Baaj’s et al work also shows that obtaining a good explanation from a symbolic AI is not as easy as the authors seem to pretend in this article. This diminishes the novelty of this paper.
I also disagree with the authors about the tasks of connectionist approaches (multiclass, multilabel or regression): it is not proper to connectionism and can be applied to symbolic AI.
A representation is defined by syntax and semantics. In this paper, only the syntax (graph) is given, but there is no clue about the semantics.

My concern is also about the explanation produced for black-box models: from the definition of explanation, it is important to give some clues about the mechanism that leads to the decision. An explanation cannot be only a list of inputs or predicates; it is also the combination of them. Moreover, there are different types of explanations (counterfactual, etc.) and the paper does not tell how to generate a corresponding graph. Is it possible to represent disjunctions? Negation?
Finally, the paper gathers good ideas, but lacks of formalism. It does not solve any of the issues that arise from explanation of decisions.

Example of typos:
- Abstract: research is[has] … grown
- P4, l. 14: developes
- P5, l. 46: and integrated
- P8, l.9: to tailoring
- P8, l. 24-26 “she”/”her”
- P8, l.28 cliniciancontaining
- P8, l.40: udnerlying
- …

This paper explains a framework to develop explanation graphs based on user data and ontologies. A use case is presented which deals with cardiovascular diseases and the dietary intake of the users.
The different scenarios to take into account in order to produce explanations (described in page 13) are an interesting aspect that the paper presents. And the 3 kinds of explanations provided (feedback, argument and suggestion) are also of value for the community.

As a full paper, it must be reviewed along the usual dimensions for research contributions which include (1) originality, (2) significance of the results, and (3) quality of writing.

The idea of explanation graphs in XAI is not new. Plus, there is a huge issue regarding originality, since the same system and results have been previously published online and the authors do not explain which is the contribution of their submission with respect to their previous paper:

[53] I. Donadello, M. Dragoni and C. Eccher, Persuasive Explanation of Reasoning Inferences on Dietary Data, in: PROFILES/SEMEX@ISWC, CEUR Workshop Proceedings, Vol. 2465, CEUR-WS.org, 2019, pp. 46–61.

The results of the evaluation shown by the analysis of data shown in Figures 10, 11 and 12 seem to be the exact same results presented in [53] in Figure 4. If this is not the case, the authors must explain explicitly which is the difference between the results presented in [53] and the results presented in this journal paper.
Moreover, the idea about the 3 kinds of explanations provided (feedback, argument and suggestion) is also presented in this paper [53] along with their corresponding templates.
So, I qualify the originality (1) of this paper as very low.

The significance of the results (2) is also low since there are no new results presented with respect to previous publication [53].

And the quality of writing (3) is quite acceptable, but, in my opinion, this paper is too long and it has many typos. The same ideas could have been summarized going straight to the point and to what is new with respect to previous publications of the authors, such as [53].

I cannot assess the data file provided by the authors, since I cannot find any “Long-term stable URL for resources” such as GitHub, Figshare or Zenodo.
There is a website describing the ontology TS4NLE but it belongs to a previous publication of the same authors, exactly reference [52].

Also the authors day that they would like to preserve "privacy and ethical aspects" but the violations that are described in the paper are not dealing with this aspect, they only refer to violating the diet they must follow. For example, in Figure 5, the age and the disease presented by the user (65 & hypertension, respectively) are directly shown, not even encrypted or blurred.

So, for all these reasons, my recommendation is rejecting this paper.

Typos
------
abstract -- it can work has --> as
page 1 -- AI-based decision-making system --> systems
page 2 -- users needs --> users' or user's
page 2 -- rules can formalizes --> formalize
page 2 -- suitable alternative for --> alternatives
page 2 -- a use cases --> case
page 3 -- the contributions limits only to the analysis --> limit
page 4 -- user evaluation The authors of --> missing full stop
page 4 -- SWRL rules inconsistencies --> rule
page 5 -- types of knowledge usable --> usable knowledge
page 8 -- udnerlying --> underlying
page 8 -- clinitiancontaining --> missing blank space
page 9 -- missing parentesis
page 10 -- A described in Section 3 --> As
page 13 -- the the communication --> remove "the"
page 13 -- A more complex scenarios --> In more
page 16 -- A A template example --> remove A