Review Comment:
The paper studies the scalability limits of OWL 2 RL reasoners on mobile platforms.
Roughly, the claimed contribution consists of the following parts:
(1) identification of several subsets and extensions of OWL 2 RL ontology language,
(2) implementation of a framework that allows to run reasoners on mobile platforms and measure their performance,
(3) benchmarking two reasoners on the subsets using the framework.
Presentation.
The writing style of the paper, especially of its first, formal part, is quite ambiguous. It operates various notions, such as ‘rule’, ‘ontology’, ‘clause’, ‘assertion, ’data pattern’, but never gives their definitions. So, it is quite difficult to understand the precise meaning of what is written, and sometimes
it seems just wrong. In fact, the only definition in the paper, Definition 1, is broken because it is recursive: IR is defined as a union of \alpha and \beta, while \alpha and \beta are defined in terms of IR. Another example is Codes 8 and 10: they use undefined notation (C_1, R_t, etc.), and it is not very clear what do they mean. Code 2 and Code 4 are also strange: Code 2 does not have any variables, that is, it is a fact (or, ‘axiom’ if I understand authors’ terminology correctly), not a rule that is applicable to each annotation property, while the rule in Code 4 has variables in the head, but nothing in the premise, so it generates infinite number of triples (or, a triple for each pair (?lt, ?dt) in the active domain, depending on the semantics). The second half of the paper, corresponding to parts (2) and (3) above, is written in a more clear way, but it is not very easy to understand as well because of the problems with the first part.
Contribution.
As far as I can see, the contribution of the paper is rather limited. In particular, part (1), described above, is quite simple, and not very well motivated. The declared aim of the search of the subsets (and extensions) is to minimise the number of rules and optimise the performance of reasoning (in fact, I do not see any reason why these two should be related). However, the minimisation is just claimed, but never shown, that is, it is not clear whether any of the resulting subsets are indeed minimal in any sense. Moreover, some of the subsets are not even equivalent to the original, so I do not see the point of comparison of the performance for non-equivalent sets of rules. At least, the word ‘optimisation’ is hardly applicable here. Another question is the presence of ‘domain-based’ subsets: in essence, some rules are eliminated because they are not applicable on the particular dataset, and then the reasoning performance (in terms of time) of the original set and the subset are compared; but the elimination step, which is normally a data-dependent part of reasoning, is not counted (and even done on a server), so the claim that the performance on the subset is better than on the whole set is just vacuous. Part (2), that is, implementation of the framework, probably took most of the time in this piece of research; however, it is just infrastructural, and can hardly be considered as contribution per se. Finally, part (3), could be moderately interesting, but suffers hardly from the problems of part (1) (and does not contain any surprising results).
Summary.
The presentational problems of the paper require quite essential rewriting of the first part of the paper, but probably fixable with a reasonable amount of effort. However, the main problem of the paper is a luck of good motivation, technical difficulty (apart of an engineering effort in part (2)), and reasonable explanation why the presented results are interesting and important. Without these, I cannot recommend acceptance.
|