Situation-Driven Multi-Purpose Adaptation in the Web of Things

This paper has some originality, but it is not clearly documented by the authors. There are many sections of the paper which are not clear, or where the contribution of the authors is not specified. Evaluation is questionable, I do not clearly understand its purpose.

More specifically:

How do you exactly define WoT and why your framework is WoT-enabled? Please clarify
Is it the fact that you model features as web resources? And if yes, which architectural elements of WoT do you use exactly?

Which is your contribution in relation to related work? I do not clearly see it.
Is it the adaptation rules based on contextual information by means of semantic web technologies? Is this something new? I am not convinced.

Since you selected an agricultural-related scenario, you should have mentioned this pioneering work in the field, which is also very relevant to your work (I am surprised you did not mention it):
Andreas Kamilaris, Feng Gao, Francesc X. Prenafeta-Boldú and Muhammad Intizar Ali. Agri-IoT: A Semantic Framework for Internet of Things-enabled Smart Farming Applications. In Proc. of the IEEE World Forum on Internet of Things (WF-IoT), Reston, VA, USA, December 2016.

You should also mention the CityPulse project and the relevant work made there. Since you mention SPITFIRE, then you should mention CityPulse too.
Puiu, Dan, Payam Barnaghi, Ralf Tönjes, Daniel Kümper, Muhammad Intizar Ali, Alessandra Mileo, Josiane Xavier Parreira et al. "Citypulse: Large scale data analytics framework for smart cities." IEEE Access 4 (2016): 1086-1108.

Why do we need rules for adaptability? Please motivate better.

How cross-domain adaptation takes place? Your scenario is about one scenario only, right?

How are the rules (defined by domain experts) are automatically translated to SPARQL SELECT queries? And if not automatic, then why not?
Furthermore (in this direction), how does the Situation Generation take place? If it is just translation of high-level rules to RDF triples, where is your innovation?

“to determine each possible contextual situation, for each purpose” - please explain

Which are the criteria for the possibility scores?

Evaluation picks a very specific scenario. It is very difficult to test correctness and to generalize.

The paper aims to tackle the research question of multi-purpose contextual adaptation in WoT applications.
Authors claim their contributions as follow:

1. A theoretical adaptation framework to support domain-independent multi-purpose adaptation
2. A generic model for/set of meta-rules to generate adaptation rules that will in turn allow to infer ranked adaptation possibilities
3. A comprehensive solution for both designers and experts to efficiently process context models and adaptation rules using an incremental reasoning engine

I think the paper is heading towards an interesting research direction which is motivated by emerging use cases and scenarios. However, the paper's
content and contribution are not sufficient to be published as a full paper in this special issue given that the publishing cycle
of a special issue has a time constraint for the follow-up revisions. Please find my detailed comments in following.

My first concern is how much the work presented in the paper is new when comparing with many other papers of the authors such
[5],[35],[36], [37]. Reading through the paper, there is a lot technical content that are referred to these papers, this makes quite
difficult to justify whether the contributions are contained in this paper. Moreover, it seems authors assume that the readers are familiar with ASAWoO platform
and the project, the concepts like avatar, meta rules, and avatar architecture are not the well-know concepts, at the least to the reviewer.
In overall, the contributions of the paper are not substantial and not well presented.

The second concern is the clarity of the technical details. Authors tend to use various "jargons" without proper explanations and definitions.
There are quite many technical, notation, presentation and formation issues across most of the sections as listed in below.

In Definition 1 for Contextual dimension whereby the contextual dimension is defined via contextual instances and observations. However, contextual instances and
observations are not formally defined. The descriptive definition such as "A contextual instance is a high-level piece of contextual information" with other examples, {Hot, Warm, Cold}
does not provide formal meaning for them. Also, d={i_d}, notation i_d has not been introduced so far, then, " d\in D where DI a the set of available..." ,
if I understand correctly d is a set, then d \in D, that means D is a set of set?, it's a bit confusing to me, the example with Temperature, {Hot, Warm, Cold}
are not properly explained this definition.

In Definition 2 authors define adaption purpose as a set of contextual instance, ap={i_ap}, and then ap\in AP, however, then authors specify AP as a set
with a list of items {Imp,Comp,Exp, Prtc,CdL} which are very odd to me. Again, there is no formal definition them ( {Imp,Comp,Exp, Prtc,CdL} ). From
their descriptive definitions, it's not clear to me whether they are functions or data elements or rules?

The Definition 3 continues the confusion with the definition of "context model as a two-dimensional set of contextual instances...", I think a detailed example
showing how such definitions are represented as real data elements must be provided. The descriptive definitions provided are not proper way to
present what so called "theoretical adaptation framework".

In Definition 4, authors try to ground/linked the above 3 definitions to RDF Triples and SPARQL queries, and transformation rules, however,
the definition of contextual situation does not show a formal relationships with the semantics, notations of RDF and SPARQL. I think all examples
in RDF and SPARQL syntaxes show authors' struggle in mapping formal definitions in Definition 5, 6, 7 and 8 to data and processing logics
and algorithms. This leads to my question why authors did not use notations, definitions and semantics of RDF, SPARQL and OWL for
such formal definitions instead.

The authors state that they "tackle the research question of multi-purpose contextual adaptation in WoT applications", however, the research
question is not clearly defined the paper, and, to the best of my knowledge, it is not a well-known one in the literature. The section 1 on
introduction, the authors just give some leads to the research question by shortly presenting some details of their project. Then, the related work section, authors
spend a lot of space to present various things but the content does not give a clearer picture on "what is the problem?", "why it is a problem?".
Note that authors claim that one of the contributions is a " theoretical framework".

For the statement: "Existing adaptation solutions are either tightly coupled with their application domains (as they rely on domain-specific context models)
or offered as standalone software components that hardly fit in Web-based and semantic architectures.",
is there any strong reference or evidence for this argument, and also it's not clear to me what is "semantic architectures"?

For this statement, "..to comply with Web standards (e.g. resource-oriented architectures, semantic Web, Web of things)...", I don't think "resource-oriented architectures,
semantic Web, Web of things" are Web standards.

In section 4.1.2, authors throw in two jargons " meta-model" and "identical reasoning mechanisms" which I have no clue what they mean.

Authors mention, "reasoning techniques", "incremental reasoning" in here and there but the technical details are not clearly presented in the paper.
For instance, in second paragraph of section 4.2.2, authors mention " At runtime, contextual instances are inserted in and deleted from the semantic repository,
which is equipped with both a SPARQL endpoint and an OWL2 RL incremental reasoner", it's unclear for me why OWL2 RL incremental reasoner is needed,
and why OWL2 RL not OWL 2 QL? In section 7, authors have an explanation in the footnote and just a quick indication processing times, but the details are not substantial.

For section 5, I found that the explanations on how the rule management process work by simply giving examples under SPARQL queries are not
sufficient in terms of technical depth, for example, are they general enough? are they semantically correct?

For section 6 on evaluation, I have a lot of doubts on the evaluation methodology and the outcome interpretation. First, authors claim the correctness
by walking through the examples, and providing example results, then claiming the correctness of the outcomes, I think they are very unconvincing. Besides, I wonder why authors
do not include the evaluations runtime performance, optimisation of adaption of rule set but move them to discussion section 7 instead.

The paper describes the ASAWoO framework that relies on Semantic Web standards to make adaptation decisions for various concerns, using contextual information. It proposes a context management component that evaluates rules to cope with contextual information.

The claimed contribution of the paper are a theoretical adaptation framework to support domain-independent multi-purpose adaptation, a generic model that allows the definition of "meta-rules" to generate adaptation rules that will in turn allow to infer ranked adaptation possibilities and a comprehensive solution for both designers and experts to efficiently process context models and adaptation rules using an incremental reasoning engine.

This paper has some originality and follows an interesting research direction with emerging use cases and scenarios that are clearly aligned with the requirements of the Call of the Special Issue. However, there are many sections of the paper that are unclear, not well motivated and there are several issues in regards to the claimed contributions. One interesting aspect of these problems can already be observed in the paper title and abstract which claims to provide a "multi-purpose [contextual] adaptation". But nowhere in the paper it is actually explained what is meant with multi-purpose adaption? Is the changing context meant to be the purpose?

Overall, there are many such issues in the paper and in the following I try to give more detailed comments on the major issues:

Detailed Comments:

Lack of rigour in definitions: The authors first define contextual dimensions, but fail to define contextual instances and the observations contained within. It also seems as if there is a finite set of contextual instances in a given context and then they choose a limited set of those for the contextual dimension, but this is unclear too? Also, is there any requirement on these instances to be different or mutually exclusive, etc. and are they considered to be always value based (It is stated later in the paper that this is the case) and from the proposed context model with the following dimensions: Wind, Dryness, Temperature, Battery, Memory, CPU and Resolution, it appears as if they are. If they are, though, do the values have to be evenly spaced, i.e. what is the relation between warm, hot and cold, numerically. Intuitively, warm is probably closer numerically to Hot than to Cold, but is that defined in a given use case. What about location?

The list of items {Imp, Comp, Exp, Prtcl, CdL} in Definition 2 is somehow arbitrary, or at least not formally defined. From their descriptive definitions, it is unclear whether they are contextual instances or functions. Also, how do some of those relate to IOPE's in the semantic Web Service domain?
Definition 4 lacks the formal mapping of contextual situation to RDF semantics,. i.e. how triples are generated as in Table 1.

Self-contained: The paper is not particularly self-contained. In several instances the reader is referred to other papers of the authors, e.g. [36] and [37] where ready-to-query adaptation possibilities for multiple adaptation purposes from both static information and raw sensor data using semantic reasoning are proposed. This is functionality that is eluded to in this paper, but not explained. At least a high-level overview needs to be presented in here. Also, the paper at hand needs to clearly define what are the contributions beyond what has been published before.

Reasoning: There are several claims in the paper that reasoning is performed at several stages of the framework. For example, that there are "avatar internal semantic reasoner to be transformed into contextual instances using transformation rules." or in section 4.2.2, the authors mention "At runtime, contextual instances are inserted in and deleted from the semantic repository, which is equipped with both a SPARQL endpoint and an OWL2 RL incremental reasoner." Both reasoning tasks are unclear and not described in the paper. Also, why is an OWL2 RL incremental reasoner needed? In the discussion, the authors claim that the expressivity levels of their solution are in AL+.

Evaluation: The evaluation is the biggest shortcoming of the paper. The evaluation does not prove the correctness of the examples, rather it is a walkthrough of the examples and what values are computed, but it is not compared to a benchmark (gold standard), nor formally proven to be correct. The efficiency evaluation, i.e. the performance of the system seems to evaluate the adaption process, but it is unclear where? In the avatars? Overall? How many runs were performed? There are only two numbers mentioned, 650ms and 30ms respectively, while the latter makes no sense, given that it apparently also includes the decision step which the users where given 100ms? Was that deducted from the 30ms? And why 100ms? How is that a common threshold for user attention as claimed? How can any user react in 0.1 seconds? Maybe seconds are meant here? In any case, this claims need to referenced. Overall, the performance evaluation needs to be split up into the different parts of the computation, run several times and then results presented. Together, the current evaluation of the effectiveness and efficiency of the evaluation are very unconvincing.

Usability: The authors claim that designers and experts alike can define the rules according to the proposed "meta-model", which in essence is a subset of SPARQL. It is even claimed later in the paper that the contextual instances (that are used for the rules) can be inferred at design time from raw sensor data using rule-based semantic reasoning. This claim is unfounded. Later in the paper, the authors also claim that the method to semi-automatically generate rules at application design time is easier than writing the rules in the first place. This claim has to be supported by some usability study, as it is unclear if the proposed meta-model "rules" are easier than writing if/then else rules in the first place. Also, why not using SPARQL templates, i.e. SHACL constraints where there is already some tool support.

Overall, the paper describes an interesting approach to deal with sensor data and perform adaptions based on rules that are supposed to be fairly easy to design by naive users. However, there are many shortcomings with the paper as stated above, while it is unclear what is the delta in this paper, compared to previous publications.