Creating Occupant-Centered Digital Twins Using the Occupant Feedback Ontology Implemented in a Smartwatch App

This paper addresses an interesting topic, integrating and exploiting building occupants' feedback leveraging a domain ontology. Authors propose a novel Occupant Feedback Ontology to provide a semantic description of occupants' feedback provided via a smartwatch application (Mintal).
The paper is well written (there are a few typos and the structure of the Sections should be revised, see e.g. a subsection "1.1 End user functionality" in Sect. 6).

The State-of-the-art reported in Section 3 is exhaustive. However, it should be completed by a paragraph describing how the proposed solution differs from (or is similar to) the works mentioned. This should help in explicitly stating the novelty of OFO and Mintal. Abbreviations should be explicitly glossed when they first appear (LEED, BREEAM, etc.) to provide readers with the appropriate context.
This paper lacks of a "Limitations" discussion, which could be included in Section 8.

There are a few key questions that need to be addressed in this work:

a) Is OFO developed following a particular Ontology Engineering Methodology? It seems to be a "classic" waterfall approach. Did it involve collaboration in the conceptualization phases and/or did it rely on domain experts? How can OFO be a "shared conceptualization" if it is not developed in collaboration with domain experts? The CQs 6 & 7 seem to be more "general questions" rather than questions used to inform your conceptualization: the CQs should be used to help you structure the domain but in Sect. 7.8 it is clear that part of their answer point outside the proposed ontology.

b) The opening of Section 5.1 presents some statements that need to be better addressed: what do the Authors mean by "taxonomical patterns are incomplete by definition"? Isn't the Occupant Property Taxonomy a taxonomy per se (is it incomplete, too)? It is not clear to what extent QUDT is reused and how the reuse helps OFO keep less "complex"; Authors should delve into details, explaining which aspects QUDT was reused to cover for.

c) Again on reuse, the Building Performance Ontology (BOP) was mapped with OFO. However, there exist already several works on this topic (some examples: 10.1016/j.autcon.2015.05.002; 10.1080/19401493.2016.1243730; 10.3390/electronics8050485). Similarly, the SOSA and SSN ontologies could have been reused to describe the observations made by devices into the environments. Many concepts and relationships modeled in the couple OFO+BOP were represented in some other works and models. The reasons why these existing ontologies were not considered for reuse should be motivated.

d) Wrt the concepts modeled in OFO, some of them may be mapped to existing and well-known ontologies. However, no mention of mapping OFO+BOP is done in the manuscript. This would increase the reusability of the proposed model(s).

e) Mintal gives the possibility to provide feedback related to indoor temperature, air quality, acoustic comfort, and visual comfort. These indoor comfort metrics are usually strictly regulated by countries: does the proposed ontology keep this fact into consideration?

f) It is not clear how the health-related data (those available such as heart rate and body presence, as well as those not yet available - O2 saturation, stress level [not easy to retrieve], sleep quality) can be used in this model to adjust comfort. Why are these metrics necessary and how do they contribute to the feedback? Are they going to be semantically modeled, and if yes, can the Authors detail any idea about it?

The authors present a work on collecting occupant feedback in buildings. The paper is written in good English. The problem addressed is that closed- and open-loop control of building services are based on sensor measurements, however, using this approach it is impossible to satisfy in all cases the individual comfort preferences of an occupant. The problem is well known and described in the literature by various authors. Traditionally occupants would complain their dissatisfaction to a facility manager, which would manually change setpoints in the automation system to (hopefully) improve the comfort level of users.

Hence, solutions as proposed in the paper are highly relevant that:

- Automatically and continuously collect occupant feedback in a scalable and convenient way and;
- Link the collected feedback to additional and/or existing building information.

The ultimate goal is that occupant feedback is integrated in a way, such that setpoints in the automation system automatically adapt to the users’ needs as he/she reports complaints.

In the abstract and introduction, the authors state the following contributions:

1. A scalable method to acquire continuous occupant feedback and directly integrate with other building information;
2. A formal model, the Occupant Feedback Ontology (OFO) to capture occupant feedback and directly integrate feedback with building information;
3. A smartwatch app, Mintal, which implements the ontological model and allows users to state their indoor environmental quality feedback;
4. The creation of occupant-centered digital twins that support decision making.

The approach presented using a smartwatch app to collect occupant feedback is a viable solution. The solution architecture (Fig. 2) presented in their use case allows for the scalable acquisition of occupant feedback. The aspect of “integrate with building information” is treated too vague throughout the paper and needs further clarification and thorough explanation:

- Which concepts and relationships are utilised to link to building information? How is actually building information obtained from source formats? The description in the use case indicates that for the studied building it was implemented manually. If the presented approach is supposed to be scalable, an automated solution for this task needs to be found.

- The ontology presented in chapter 5 and depicted in Fig. 3 shows a concept ofo:Location. In the query presented in Listing 12 “bot:hasElement” is utilised for linking to a bot:Element. Is this mismatch intended?

The formal model, the Occupant Feedback Ontology (OFO) is presented in chapter 5. The presented explanations and visualisation allow to understand the modelling choices. The authors define seven competency questions to evaluate their modelling and later present queries in SPARQL implementing these questions. The web-based documentation is accessible and is helpful to retrieve the ontology as well as find further explanations.

The ontology is well designed to fit the specific purpose of the use case. Unclear, is why in many cases known modelling patterns have been used but, new concepts have been defined. Well-known ontology engineering methods stipulate the reuse of existing ontologies (1), however, no explicit links could be found for concepts in OFO. Some concepts could for instance be mapped to:

- ofo:Person -> foaf:Person?
- ofo:Location -> bot:Space? Cf. (4)
- ofo:FeatureOfInterest -> sosa:FeatureOfInterest
- ofo:Datapoint -> brick:Point cf. (2)
- ofo:hasPropertyState -> opm:hasPropertyState cf. (3)
- ofo:Wearable -> s4wear:Wearable (https://saref.etsi.org/saref4wear/Wearable)

Moreover, the following questions remain open:

5. CQ3 -> “Location or object”: The created relationships in the ontology suggest to only referring to ofo:Location. How to relate to an object? Cf. also the quest in Listing 12 which relates to a bot:Element?

The app called Mintal presented in section 6 allows occupants to return feedback. The app immediately issues the generated data as RDF triples. Similar apps have been implemented and described by other authors cf. (5)(9). To assess the value of this contribution the authors should review existing solutions in more detail according to some criteria and clearly justify, what distinguished Mintal from existing apps.

Lastly, the notion of a “occupant-centered digital twin” is mentioned in the title and abstract, however it remains unclear what is meant by the authors. If the core focus of this paper relies in these topics, the authors should revise the manuscript accordingly.

In addition, in a couple of statements in the paper the authors claim that decision making is enhanced. However, it remains unclear what decision making is meant, and at which point in the building lifecycle. The authors should revise the manuscript to clarify on this.

Given the above evaluation I suggest performing a major revision of the paper. I would propose to revise the structure of the paper as follows:

- Introduction
- Related Work -> revise the content in Introduction and chapter 2 and 3 into one structured related work section. The outcome of this section should clearly state, why the development of OFO and Mintal is necessary
- OFO
- Mintal
- Use case including experimental setup
o Place here the content of chapter 4 and 7. In particular, section 7.8 use case seems to be misplaced at its current position
- Discussion
- Conclusion

A revision of this work should moreover focus on the actual content and clarifying on the novelty aspects of the presented contributions:

- What made the development of OFO necessary? Why not use existing ontologies presented in the past (BOT, OPM, HEX, SOSA, s4wear, …)
- What distinguishes the presented app and methodology from existing approaches, in particular to the contributions by:
o (6) (9) (10) (7)
- How does the work relate to the overall concept of a digital twin in the built environment?
For the readers of this journal, it would be important to know in more detail, what is actually the benefit of semantic web technologies in the presented use case? The authors state that “Semantic web technologies were applied to solve data interoperability issues”. It would be great to learn in more detail, which interoperability issues have been observed and how semantic web technologies helped to overcome these.

I am looking forward receiving your answer and the revised manuscript.

Best,

Georg

Some notes with minor comments I´d like to share.

Chapter 3 Occupant Feedback

1. While the authors present a comprehensive overview of a full body of literature a final conclusion and assessment is missing. What is the gap in existing ontologies that they are filling?

Conclusion

The conclusion narrative should be revised. It reads more like an introduction
2. “We found two challenges in the literature.” -> Please provide citations
3. “The literature mentions various issue” -> it would be helpful if you point to cited works here or to the related work section. It remains unclear to the reader, what is “the literature”

Please check throughout the whole paper that abbreviations are introduced once and used afterwards consistently.
o “First, we presented the Occupant Feedback Ontology. OFO “ -> E.g. “Occupant Feedback Ontology (OFO). OFO …”
o „The EMA method“
o „BOP ontology“

References:

Below references I have been referring to in this review:

(1) Pinto, H. S., & Martins, J. P. (2004). Ontologies: How can they be built? Knowledge and information systems, 6(4), 441-464.
(2) Balaji, B., Bhattacharya, A., Fierro, G., Gao, J., Gluck, J., Hong, D., ... & Whitehouse, K. (2016, November). Brick: Towards a unified metadata schema for buildings. In Proceedings of the 3rd ACM International Conference on Systems for Energy-Efficient Built Environments (pp. 41-50).
(3) Holten Rasmussen, M., Lefrançois, M., Bonduel, M., Anker Hviid, C., & Karlshøj, J. (2018). OPM: An ontology for describing properties that evolve over time. In CEUR Workshop Proceedings (Vol. 2159, pp. 24-33). CEUR Workshop Proceedings.
(4) Rasmussen, M. H., Lefrançois, M., Schneider, G. F., & Pauwels, P. (2021). BOT: the building topology ontology of the W3C linked building data group. Semantic Web, 12(1), 143-161.
(5) Abdallah, M.; Clevenger, C.; Vu, T.; Nguyen, A. Sensing Occupant Comfort Using Wearable Technologies. In Proceedings of the 2016 Construction Research Congress, San Juan, Puerto Rico, 31 May–2 June 2016; pp. 940–950.
(6) Barrios, L.; Kleiminger, W. The Comfstat—Automatically sensing thermal comfort for smart thermostats. In Proceedings of the 2017 IEEE International Conference on Pervasive Computing and Communications (PerCom), Kona, HI, USA, 13–17 March 2017, pp. 257–266.
(7) Ramsauer, D., Dorfmann, M., Tellioğlu, H., & Kastner, W. (2022). Human Perception and Building Automation Systems. Energies, 15(5), 1745.
(8) Qiu, H., Schneider, G. F., Kauppinen, T., Rudolph, S., & Steiger, S. (2018). Reasoning on Human Experiences of Indoor Environments using SemanticWeb Technologies. In ISARC. Proceedings of the International Symposium on Automation and Robotics in Construction (Vol. 35, pp. 1-8). IAARC Publications.
(9) P. Jayathissa, M. Quintana, T. Sood, N. Nazarian, and C. Miller, Is your clock-face cozie? A smartwatch methodology for the in-situ collection of occupant comfort data, in: Journal of Physics: Conference Series, 2019. doi:10.1088/1742-6596/1343/1/012145.
(10) F.M. Gray, H. Dibowski, J. Gall, and S. Braun, Occupant Feedback and Context Awareness: On the Application of Building Information Modeling and Semantic Technologies for Improved Complaint Management in Commercial Buildings, in: IEEE International Conference on Emerging Technologies and Factory Automation, ETFA, 2020. doi:10.1109/ETFA46521.2020.9212164.

This paper describes a very good piece of research on monitoring occupant feedback with a smartwatch application *Mintal*, using an ontology *Ontology Occupant Feedback (OFO)* to represent the knowledge about the building, occupants, and their feedback.

* The approach is original and well positioned wrt the state of the art, and extend recent W3C working and community groups specifications.
* The main contributions of the article: the OFO ontology, the Mintal application, the system architecture for continuous occupant feedback monitoring, are significant and FAIR. Most resources are available online on github, and the ontologies are documented and published online following the best practices, with a permanent w3id identifier.
* The whole article is very well written and structured, and easy to follow. The figures are mostly well designed and self-explanatory, and their sources are available in the article repository on github, or on the ontology documentation page.

However there is still some room for improvement as listed below, and I therefore recommend this paper for minor revision.

What may be lacking from this paper is a stronger evaluation with respect to other approaches from the state of the art:
- how better does the system performs wrt some selected approaches listed in Section 2. Is this the first system that is openly available and can be reproduced ?
- what representational gap does the ontology cover wrt ontologies listed in Section 3.

The introduction positions the research well, and lists the main contributions of the paper.

Section 2 provides a state of the art on occupant feedback systems, and justifies the adoption of a micro ecological momentary assessment method in this paper, with wearables such as smart watches. Section 3 focuses on how Semantic Web Technologies were used to integrate occupants and their feedback in other building information.
* As a conclusion of this state of the art, a coverage comparison of these ontologies would be useful to justify why OFO needs to be developed.
* The recent SAREF4WEAR ontology https://saref.etsi.org/saref4wear may be of interest to this state of the art.

The OFO ontology is described canonically in Section 5, with seven competency questions that are later used to evaluate the ontology in Section 7.
* It is not clear which of these questions wouldn't be answered by the ontologies from the state of the art.
* OFO is used jointly with BOT, PROV, QUDT, and OBO, but it's clear that OFO is aligned to the W3C&OGC SOSA/SSN. An explicit alignment would be useful.
* There are too many links that overlap on Figure 3, making it hard to follow them without background knowledge on the SSN naming patterns.

As a co-developer of SOSA/SSN and BOT, on which this work is grounded, I do approve the design choices of the author for OBO, OFO, and the separate Occupant Property Taxonomy that can accept future contributions.
* The property chain axioms are relevant, however apart from introducing these axioms the authors do not demonstrate how the reasoning actually works. These axioms could for example be leveraged in querying a sparql endpoint with an entailment regime. For example the query for CQ7 could potentially be simpler with less UNIONs ?

The Mintal application is described in Section 6, and satisfies six requirements. This section is well written and the resources are available online for reproducing the experiments. I see one minor aspect that could be improved:
* It is not clear which of these requirements are not covered by the state of the art apps and systems described in Section 2.

The Section 7 demonstrate the use of the OFO ontology and its implementation in the Mintal application, by showing actual answers to the different competency questions listed in Section 5. Subsection 7.8 is very useful and demonstrate the approach in a small use case with continuous feedback monitoring in a small flat.

The discussion and conclusion are interesting and to the point.

Other minor comments:

- Reference 11 refers to the *old* SSN. Other references should be used for the new SOSA/SSN ontology
- Many acronyms are used before they are defined. PMV/PPD, LEED, BREEAM, POE, EMA, ...