Ontology evolution from RDF streams using possibilistic axiom scoring

This manuscript is a fresh resubmission - not a revision - of a manuscript that I had already reviewed for this same journal. Therefore, I will sometimes comment on the way this manuscript differs from the one (SWJ 3703) I had previously reviewed.

In the context of ontology learning, enrichment, and validation, axiom scoring is the task of evaluating the acceptability of a (candidate) axiom against the known facts. The authors of this manuscript study axiom scoring in a scenario of ontology evolution, whereby new facts (represented by RDF triples) are added to a knowledge base at different times and the underlying ontology needs to be revised based on the newly acquired knowledge.

To this aim, the authors adapt a possibilistic axiom-scoring heuristic proposed in the literature to the case of ontology evolution based on RDF data streams. After defining the problem of axiom testing against a sliding window of a stream of RDF data, focusing on the property axioms of functionality, inverse functionality, transitivity, irreflexiveness, symmetry and asymmetry, they introduce their adaptation of the possibilistic scoring approach and validate it through two experiments.
In the first experiments, they use the CMT ontology to test the extent to which the possibilistic approach is capable of correctly scoring some axioms that are known to hold, using three different sliding window sizes, and compare it to traditional information-retrieval measures like precision and F1-score. The results allow them to conclude that the possibilistic approach is robust and applicable when scoring axioms in streams and with limited data, and more so than a strictly probability-based one.
The second experiment considers an actual scenario of ontology evolution, using the game-related fictional domain of Pokémons, where successive generations (I through IX) provide sets of instances with different properties. To deal with this scenario, they compare three approaches: (i) using the plain possibilistic score, (ii) using same score together with a user-defined threshold for axiom acceptance, and (iii) defining an evolving possibilistic score as a weighted average of the past and present score for each axiom - a sort of moving average. The results suggest that approach (ii) is the most effective of the three to capture with the ontology the changes occurring in the stream of instances.

The idea of applying the possibilistic axiom scoring heuristics to RDF data streams for ontology evolution is novel and the proposed adaptation of the approach is original.
The article is well-written and easy to read. I found a few typos, which are detailed below.
The empirical validation is convincing, although choosing a real-world ontology of practical relevance would have made Experiment II even more compelling; nevertheless, I am inclined to believe that the domain of Pokémon can serve as a simplified model of the phenomena one could observe in real-world scenarios, like the ones of streams of sensor data, which motivates this contribution, as explained in Section 3.1.
Overall, the paper is technically sound, althogh I found issues with some of the definitions and with the terminology. However, these issues are easy to fix and they do not impact the substantial correctness of the proposed approach.

Detailed Comments

To begin with, I am pleased to notice that the remarks I had made on manuscript SWJ 3703 have been taken into account by the authors to prepare this new version.
In particular, I find that the newly written Swction 3.3 answers my remarks, solves my doubts, and is much more intuitive.
While the presentation of the manuscript has improved, some new issues have been introduced.

The first issue has to do with terminology. For some reason, the authors have decided to rename "confirmations" as "examples", probably because the concept is dual to "counterexample"; this has left the term "confirmation" free to be used to denote what formerly was called "selective confirmations". However, this terminological choice is unfortunate and runs into problems. The first one is that the "selective confirmation principle" is known by that name in the literature, including [11]. Now, the authors have changed its name into "selective example principle", which, apart from calling the same principle with another name, is much less semantically motivated: it is not clear how an example can be "selective". Now, the authors have also - inadvertedly, I hope - changed the title of [11] to match their terminology, thus introducing an error in the bibliography. Then, using "confirmation" (a term which is used in the literature) to mean what is called "selective confirmation" in the literature can only create confusion. I think names are important and changing names that have become established in the literature is a decision that should not be taken lightly. My advice is to adhere to the established conventions.

The second issue has to do to Definition 1. In it, the authors define an individual as a set of RDF triples having the same entity as their subject. In the same definition, they introduce the notation FI(i), where i is an individual, to mean the facts about i, defined as all triples with the same subject... Which to me is exactly the same as what the authors call an individual! But if FI(i) = i, why do we need the notation FI(.)?
However, later, in Definitions 4 and 5, the authors talk about a "named individual" i as a subject of properties P(i, x), and they refer to its set of facts as FI(i), which contradicts Definition 1.
So, in the end, I think that after all what the authors call an individual is not a set of RDF triples, but rather an "individual" in the same sense as in description logics, which is in perfect agreement with their usage of a "named individual".

Section 3.4 has been moved where it is now from the Section on experiments. I think this is a good idea, because it introduces one of the contributions made by the authors. However, in so doing, their reference to "this experiment" does not make sense anymore and is out of context. I suggest to replace it with more general terms like "scenario", "application", or the like, and replace the "this" with a qualification of the scenario/application the evolving ARI is designed to address.

Typos and minor issues:

- sliding windows sizes -> sliding window sizes
- through means of -> by means of
- The alignment of some formulas should be fixed
- ... is to not include ... -> ... is not to include ...
- decision reached \ using / ARI

Assessment of the data file

A link is provided by the authors to the GitHub repository of the TICO_Lite tool.
Upon inspection, this repository appears to be well organized, but a README file is missing. I warmly recommend that the authors add one to the repository.
The provided resources appear to be complete for replication of experiments.
All the data artifacts used in the article are there and they appear to be complete.

Thank you for the opportunity to read and review the paper on "Ontology evolution from RDF streams using probabilistic axiom scoring."

The authors present their work on deriving axioms from data streams, in the context of evolving ontologies.

The paper addresses an interesting and relevant aspect to the semantic web community. The focus in this work is on "recognizing the property axioms that should be added with" the detected new properties. A methodology was presented, and experiments with their results are reflected in the paper.

The paper can benefit from improvements that I present below.

First, the paper lacks a research question that should be clearly mentioned early in the paper. This helps guide the research method and experiment design/evaluations.

Related to this point, the research motivation should be better articulated. The research gap is not clear. Mentioning that TICO (Time Constrained instance-guided Ontology Evolution), which is one of the authors' previous work, doesn't handle axioms detection is not enough. The research problem should be better introduced to clarify the importance of axiom detection in the context of ontology evolution.

Another potential way that could improve the readability of the paper is to introduce an example, scenario, or use case that clarifies the potential impact and applications of this research. Is the focus, for example, on comparing two versions of the same ontology? Or are we talking about a stream of RDF triples. What is particular about RDF streams? And how manipulating axioms will be beneficial to evolve an ontology? Is it to avoid inconsistencies, support reasoning, etc. Having such an example early in the paper helps the reader understand more the research scope and motivation.

It was hard to contextualize the research in the field of ontology evolution. I would recommend expanding the related work on ontology evolution to clearly identify the gap and focus the research accordingly.

In the background section, multiple claims are mentioned with missing references. For example, on page 2, the related works on linguistic vs machine learning-based approaches on ontology evolution are missing. These are just examples, and a thorough check across all sections would be needed to add related references.

At the level of research methodology, the TICO that this research relies on is blackboxed. I suggest giving more details about TICO to better clarify the process followed in this paper. What is the role of TICO? Does it output a "named property" that is part of an RDF triple resulting from a previous ontology evolution activity? And then the axiom scoring process that this paper is handling is triggered? I suggest having a high-level framework of the methodology to help improve legibility and how the proposed different components interact.

With respect to the experiments, I suggest clarifying the purpose of the experiments with a clear connection to the research question(s).

It seems that one of the major results of Experiment 1 is to identify the appropriate sliding window size? In other words, the proposed method is highly dependent on the sliding window size, which was determined empirically through Experiment 1. It is not clear why only 10, 50, and 100 sliding window sizes were selected? Isn't it better to run the experiment while incrementally increased the window size between 1 and 100? Then pick the best?

Another question about the setup of Experiment 1 is that it's applied to a static ontology (i.e., non-evolving) with a fixed set of triples. I may have missed it, but it wasn't clear to me how the RDF streams were constructed or relevant in this case?

In Section 4.3, one of the first insights is that "we consider that 50 is a good middle ground to allow for sufficient proper classification of individuals." This is confusing as the experiment is not about "classifying individuals". Isn't it about having an existing individual already classified? And your experiment is checking the detection of axiom property? Having a better aligned research question, evaluation objectives, and related insights are definitely needed.

In Table 7, isn't the hasAuthor property a functional property?
With respect to experiment 2, the choice of use case is clearer, where you have evolving data across different generations of Pokémon. However, the way it is presented focuses extensively on the data changes and their impact. It reads more like a report that fits better as an appendix. I recommend moving some of this content to an appendix, this will leave room to focus on the evaluation results and further discussion on how the work drives the research in this field forward.

Instead of having 2 sections each describing a separate evaluation, I suggest having a section about the evaluation/experiment, and a section about the results and related discussion.
The conclusion would benefit from reflecting on future research areas.

Additional comments:
- Page 5: "Contrary to Tettamanzi’s approach" --> Add reference
- Page 8: "it is not possible to have all the data that is needed for the remaining individuals to count as confirmations." --> it is not possible to have all the data needed...
- Page 10: "provides its own confusion matrix" --> provides its own confusion matrix
- Page 22: "weak forms of AIR" --> weak forms of ARI?

I hope you will find my comments useful for improving the paper. Good luck!

The paper is a resubmission of the previous work titled "Applying possibilistic axiom scoring to instance-guided ontology evolution in RDF streams". I do appreciate certain changes to the paper, including the reductions on OWL/DL for the sake of overview figures and the like. The authors still analyze a technique evaluating the nature of properties, here called axioms, like functionality, symmetry, or irreflexivity. Compared to the first paper, notions like strong/weak (counter-)examples are better explained and assumptions try to explain their existence for certain properties.

While Sect. 3 now starts with an informal description of the "problem" (by the way, what is the actual problem?), the wall of "definitions" remains afterwards. I do appreciate leaving out the unnecessary ones from the previous version of the paper, but still think that the style in which definitions are stated is inappropriate. For instance, definition 1 may stop after the first sentence. The rest of the paragraph has a descriptive, or notational, character rather than a definitorial. What is left open by the definition is the subject itself. Is the same as the individual $i$ or is there something I'm missing here. Definitions 2 to 4 are no real definitions. They introduce symbols or describe how a system operates. Is $cal(S)$ the stream or the elements over which the stream is formed? Is $cal(S)$ an actual entity that produces the stream elements (i.e., the individuals)? The symbolic nature of the sliding window (i.e., $cal(S)cal(w)$) does not make sense. As it is described, it is a data structure (i.e., a queue). As such, it could be formalized like this and therefore get its proper operators (like enqueue or dequeue), but it also seems to be a minor irrelevant detail that could also be just stated without any symbols in use.

Overall, I still find the contribution (the adaptation of ARI accounting for the streaming scenario) a little weak. The reason is that weighting previous knowledge against the new knowledge seems not to be unexpected, but if others would like to accept the paper for the journal as it is, I will not object. I do like the scenarios, although the evolutionary character of the Pokemon example is somewhat questionable, but still seems to fit its purpose. For a potential final submission, please proof-read your paper carefully. Some comments are listed below.

*Minor Comments:*
- there is inconsistent usage of the interval notation $[x,y]$ (sometimes written as $[x-y]$ or $[x;y]$), but it also has different meanings; in some places it is really the closed interval between $x$ and $y$, but sometimes it also abbreviates the set ${ x, y }$
- in the abstract, "[...] has been applied to complete and large datasets [...]" is somewhat misleading, at least up to my reading: I thought it has been applied to complete the datasets, but this does not make any sense in the context of the rest of the sentence
- Def. 2: "unbound" -> unbounded
- Def. 4: "constrains" -> constraints
- I still find the words "transitiveness", "reflexiveness", and the like weird and would expect them to be replaced by transitivity, reflexivity, etc.