Review Comment:
== Problem ==
The paper investigates the problem of trusts in the context IoT devices. As IoT devices are increasing and becoming more and more pervasive, making aware people about which data is collected, by whom and for what is used is getting crucial. The problem is relevant and, as shown in the rest of the paper, Semantic Web technologies can play a key role to cope with it.
== Solution ==
To tackle this problem, authors present a platform, Trusted Tiny Things, to manage the trustability of IoT devices. The idea is to annotate IoT devices with a description of their features, their owner, etc., and reason over them to let the user know what they do, and let the user decide if to interact with it.
The decision of what information to collect about the devices and how to use it have been elicited through a survey involving 77 participants. An ontology, T^3, has been built on top of PROV-O. The reasoning is managed through SPIN.
The paper does not provide any link to the platform, and in general, it is not clear if it is made available. This should be the core contribution of the paper, but it has already been presented in a previous publication of the same authors: http://ceur-ws.org/Vol-1280/paper9.pdf. The content is almost the same - the two sections are switched, but the text does not add any information and does not describe any new feature. I think that it would be possible to add more details, for example explaining the technical decisions (e.g., why SPIN to manage the reasoning? why TDB?).
== Evaluation ==
This section is not in the previous workshop paper. Two experiments have been conducted.
In the first, a group of users have been asked to use the platform in the context of public transportation. Authors were interested in understanding if users were wanted to have more information about the IoT devices they interact with, to control the data generated and to get feedback about the interface.
In the second, time performance is evaluated.
Authors use mainly percentages to analyse the results (in particular in the second experiment). However, percentages offer a partial view of the results. The lack of absolute values limits the understand of the obtained results.
Moreover, Experiment 2 is not reproducible, since the experimental settings are not described, and the data and software are not available.
To summarise, I believe that at the current status, this paper is not suitable for publication. The evaluation section, as is, is not enough to let the paper be published as a new contribution. Sections 4 and 5 should be extended to increase the amount of contribution, and Section 6 needs to be largely revised. It would be interesting to perform some experiments to study the scalability of the platform. Finally, the related work section should be improved, since at the moment the number of references is very limited (11 references).
|