Towards Ontology-based Expert System Development and Evaluation for Rice Disease Identification and Control Recommendation

The paper discusses an ontology-based expert system developed to support technical and non-technical users for rice diseases and pests identification.

Authors’ contributions are threefolds. First, they identified and evaluated existing domain ontologies to finally propose revised consensus ones. Second, they developed an expert system called RiceMan that combines semantic Web technologies, ontologies and mobile applications. Third, both ontologies and RiceMan were evaluated to ensure its usefulness and usability in various aspects.

It is not clear how this current work differs from the other works already published. https://doi.org/10.1007/s00354-019-00072-0,
10.4018/JITR.2017100103
10.1145/3291280.3291786
Thus my advice to authors is to clarify this point in the manuscript.
--
(1) originality
The paper shows some originality both in the approach and the practicability in a real use-case
(2) significance of the results
The paper shows good results either for the application than for the evaluation by users
(3) quality of writing
good quality
--

Section 2 - Background.
This section describes existing ontologies, online farming knowledge system and expert system for plant diagnosis. Furthermore, authors evaluated the relevance of selected ontologies for their purpose and according to 7 criteria.

Major concern:
Some important domain ontologies are missing such as the Plant Trait Ontology and the Phenotype And Trait Ontology. Why did the authors not evaluate them ?

Minor concern:
There are some missing URL links to project pages.

Section 3 - Ontology Development for RiceMan
This section is an important part of the contribution. First, authors define some scientific questions to better evaluate the scope of the ontology usage. Then, they explain the revised design of both RiceDO and TreatO candidate ontologies.

Major concern:
Authors did not discuss terms reuse from other ontologies and how potential mappings between ontologies could be managed.

Section 4 - RiceMan Application Development
This section is an important part of the contribution. First, authors define some main functionalities and actors of the application. Then, they show the middleware layer design that combines the query engine and the reasonner. Finally, they show several user’s interface that allow them to query the system.

Section 5 - RiceMan: Ontology and System Evaluation
This section discusses the results of several evaluations done for the ontology development and the RiceMan system. For sure, it is an important part of the result as it analyzes the strength and weakness of the system and how authors plan to improve it. The evaluation of the ontologies shows very high scores showing that they are suitable to answer research questions. The evaluation of the RiceMan application shows an upper average score showing there is space for improvements. Authors identify some language barrier to use the application. They plan to develop a Thai language specific interface for the next release.

Section 6 - Conclusion
Authors plan to extend the ontologies along with agronomists in order to cover a wider knowledge.

References

Minor concern:
Ref 7 have a more recent and paper published in a journal
Ref 8 : please provide a web URL for the working group
Ref 11 please provide a web URL
Ref 12 & 13 please provide a web URL
Ref 15 & 22 please provide a web URL

In this article, the authors describe their work in improving two ontologies they already developed: the Rice Disease Ontology and the Treatment Ontology, for describing diseases in rice plantations and their treatments, so that these could be effectively adopted in an application, RiceMan, for discovering diseases and recommending on how to treat them. Both the ontologies and the platform have been evaluated, mainly by means of competency questions and user feedback reported on a System Usability Scale.
The article is an easy and understandable read, yet the English wording is not adequate for a journal publication. I started reporting some typos and sentences I would fix, but I noticed soon that, while the quality of the writing does not affect much the understandability, the article is overly affected by many typos and incorrect expressions. While this is not affecting my opinion on the possibility to publish it, I consider it a necessary step, if the article is accepted for publication, to revise it or to have it revised by a proficient English writer.

The ontology URI does not resolve on the Web. The authors provide the link to the project in GitHub; nonetheless, if this is a mature work that has to be presented on a journal, the related ontologies should be published according to SW/LOD best practices.

Concerning the ontologies, I am a bit skeptical about the excessive modeling of everything as classes. E.g, from fig. 1, I see that not just diseases, but even colors are classes. It is not clear to me how these instances are mapped. It is difficult for me even to judge certain choices that might seem not obvious e.g why is the same instance of spotOnLeaf having as color an individual that is both Brown and BrownishYellow? I might understand that presence of two colors being mapped on the same spotOnLeaf instance, but I don’t see why it is a same color individual that has to belong to two different classes. I try to guess what the objective is: take a newly created individual (e.g., representing an observed, unknown disease, and the effects of it on a plant) that manifests some characteristics, classify it by means of reasoning, and get the disease as a resulting computed class. However, the same could be possible by getting the classification through the reasoner and, in turn, the instances (i.e. the diseases, or treatments, depending on what is searched) of the retrieved classes as candidates for identifying the observed disease. Maintaining the dataset would also be easier, as many new facts could be represented through new data in the ABox instead of new axioms in the TBox. I am indeed surprised that nobody objected this (or possibly they are lost in the small percentages that gave a negative response on the CQs in table 2?). This raises another question: what kind of participants performed that part of the evaluation? For instance, if they are only domain experts (at least DEs seem to be required for replying to the first CGs about appropriateness, but the article only provides the composition of the pool for the part of the evaluation discussed in 5.3.1: DEs only, even though for that part this is acceptable, and 5.3.2), do they have the proper knowledge for evaluating the modeling or they just intuitively checked the constraints wrt the knowledge they have of the domain?
While I am not strongly objecting a-priori and definitively the choices being made, these should be better clarified along the paper (and yet from the point where they are presented) instead of merely prosing what is written in Manchester syntax (which is obvious for most readers proficient in the field) as done in section 3.2.1

Concerning the evaluation, I find a little vague the description of the setting and, while I recognize the effort put in providing different angles on the evaluation, it does not provide real numbers for grasping the value added by the automatic support of the tool. The CQs are clear, but the dimension of the problem not that much. The authors mention 18 kind of diseases; now, wrt the claim in section 4.1: “Without Intelligent tools, they [the agronomists] need to memorize all complex relationships in order to support farmers.”, 18 diseases do not seem to be a big issue for somebody who has studied and is continuously updated for that specific job. While a relatively small dimension doesn’t diminish the value and importance of automatic means for disease recognition and for suggesting remedies (e.g. the authors mention that farmers still prefer to rely on agronomists rather than applications, but this could change in the future if the applications provide sufficient reliability and ease of use), the evaluation should at least better outline any complexities, if present. e.g. besides the number of diseases, what is the size of the search space? Rough estimation of all the parameters involved in the classification problem, of the range of the properties representing these parameters, could provide this background.

Overall, I see an interesting effort and an honest evaluation that, despite some obscure points, has been conducted on several aspects. The part on ontology modeling should be better justified. Possibly, all the “historical” part on the improvements performed on the first versions of the ontologies might be compacted a little (especially considering that we are not talking about ontologies with a large user base in the past, so it is of little interest learning about all the improvements over past errors), making room for more insights on the ontology themselves, as they are now. It is not necessary to describe each detail (they are not small) but I would suggest addressing my expressed concerns by motivating more and more clearly the choice that have been done. The evaluation could also provide more information on the pool of participants for what concerns the first part and on the different dimensions involved in the classification, so to make it clear what the value added is in adopting such expert system even for a prepared agronomist.

I cannot recommend the paper for acceptance as-is and, honestly, I can’t tell if a major revision would be positive: it is indeed necessary to let the reviewers better understand their work and if it is worthy of being published at this stage of maturity.

A FEW TECHNICAL REMARKS

I see that, other than ontologies, thesauri have been considered as well (e.g. AGROVOC). Have the authors looked into NALT (the NAL thesaurus) ? It’s the thesaurus of the National Agricultural Library of the U.S. Department of Agriculture (USDA). As far as I recall, it’s less “horizontal” than Agrovoc, definitely not multilingual (a point in which Agrovoc excels) but it should have a very deep coverage of many aspects related to agriculture, possibly those on plant and pesticides.

A note about Agrovoc: more than axioms (its vocabulary includes the properties hasPest/pestOf, which would be surely interesting for the topic of the presented article), as mentioned in the paper, the vocabulary is missing factual data (the properties are populated with 4 plant/pest pairs). After all, it is a thesaurus, and despite the sporadic leaps at managing more structured information, the core objectives remain hierarchy and multilingualism.

I suggest including the following reference, as it is a rice crop planning system and has thus a certain overlap with the presented article:
MEDES '15: Proceedings of the 7th International Conference on Management of computational and collective intElligence in Digital EcoSystems October 2015 Pages 250–257 https://doi.org/10.1145/2857218.2857272

What is the meaning of “feasible” in section 3.1 ? I guess the use of the word is wrong. Does it mean “recognizable” there?

TYPOs and MINOR REMARKS

abstract it's - - > their
Numerous existing knowledge --> knowledge is uncountable and can’t be regarded as “numerous”
S2. Why “despite” ? the two things: ontologies and literature are not in contradiction
S2. Why the references on the rice ontology include Agroportal? If that was meant as a reference because it is hosted there (no better reference?) then it should be more precise. Even the other reference is too vague: rather than citing the WG that created it, a link to the specifications should be provided. Same for reference 12 to PPO
Note on table 1: not coverage --> “not covered” or “no coverage”
S2.1-r43: “as instances of the ontology” , better to say “as individuals in the ontology”.
P4r11: “sustainable farming”
P5r9: “the focused was to define” --> “the focus was on defining”
Tables 2,3 are not consistent in the header. The header is always “Average (%)” but the values are: only average in the single records in table 2 and then average and percentage in the concluding record, and only percentage in table 3

Overall Comments and Summary
This paper details the modifications made to the existing ontologies RiceDO and TreatO, and RiceMan, a semantic-based framework, which were all three introduced in a 2019 paper by the same authors (Jearanaiwongkul et al., 2019).

In the current manuscript, the authors outline the steps they undertook to revise the RiceDO based on several existing resources: Rice Knowledge Bank Thai version (Thai-RKB), Rice Knowledge Bank (RKB) by IRRI, and the Rice Ontology (RO) (aka Crop Ontology Rice Trait Ontology). They describe revisions to the TreatO ontology, based on the information from IRRI’s RKB’s fact sheets and other plant disease management resources. These changes came about after they had consulted two (unnamed) ontology engineers and a number of issues had been identified.

They also detail the changes made to the RiceMan application, the system architecture and user interface, and how the Observation Data are constructed and composed from multiple observations. The remainder of the paper details an evaluation they undertook of the ontologies and the RiceMan system.

While the manuscript has a number of issues, it does represent a valuable and significant contribution to the field. The approach of integrating the two ontologies for diagnosis and treatment recommendations in a smart phone application is unique, to my knowledge. Also, the added functionality of extrapolating over time and geographical locations will be useful for their users.

Overall, the manuscript is excessively long. The Background section could be condensed and made more concise, and sections 2.2 and 2.3 could be combined. I also suggest eliminating the introductory paragraphs in each section for example:
(P2, L5): “We take a look into the literature of modeled knowledge
base in agriculture area.”
(P5, L20): “This section designs and develops ontologies for rice disease identification and control recommendation to be used by RiceMan system.”
(P13, L32): “We discuss the design and execution of our evaluation in this section.”
There are a large number of grammatical issues and problems with the wording of the sentences throughout the manuscript. There are numerous issues with mixing up past and present tenses, and missing or misused articles. I suggest the authors should enlist the help of an English-speaking editor to address these issues.

In the Background section, the authors review a number of existing ontologies and semantic web technologies. In Section 2.1, the authors present a number of ontologies and vocabularies and there are a number of inaccuracies. AGROVOC (not ‘Agrovoc’) should not be referred to as an ontology- it is a controlled vocabulary. The Crop Ontology Rice Trait Ontology should not be referred to as the “RO” as that acronym refers to the Relation Ontology (http://www.obofoundry.org/ontology/ro.html). The correct prefix for the Crop Ontology Rice Trait Ontology is CO_320 and it is inaccurate to say that it is modeled in OWL, as it is not. Some of the ontologies are not properly referenced, such as the Plant Ontology (Cooper et al., 2013; Cooper et al., 2018; Walls et al., 2019) and the Crop Ontology Rice Trait Ontology (Arnaud et al., 2013; Cooper et al., 2018), as it is incorrect to cite Agroportal as the reference for the CO Rice Ontology. In addition, the authors should also include the Rice Diagnostic tool ‘Rice Doctor”, developed by IRRI (http://www.knowledgebank.irri.org/decision-tools/rice-doctor).

It's confusing that some of the resources are discussed in the past tense, such as the Plant Ontology, PlantVillage and Plantwise Knowledge Bank, while others are discussed in the present tense, such as AGROVOC. Some of the sentences have a mixture of present and past tenses.
Table 1 contains inaccurate information about the domains of the PO and the CO Rice Trait Ontology and would be more useful if it was a comparison of the other existing applications and web technologies detailed in Section 2.2 and 2.3.

In Section 3, the authors detail the design and revisions to the RiceDO and TreatO ontologies, based on the Competency Questions and the evaluation of the ontologies by two unnamed ‘ontology engineers’. Later, in Section 5.2, the revised ontologies were evaluated by “five evaluators who were ontology engineers and ontology experts” Since there is no information as to who did the evaluations and what their credentials were, it is difficult to put much stock into these evaluations. One of the most important principles of ontology design is to reuse relevant parts of existing ontologies. In the RiceDO, the rice diseases are imported from the PDO, but the ‘PlantPart’ and ‘GrowthStageGroup’ branch of the ontology does not import the relevant classes from the Plant Ontology. None of the classes in the RiceDO have textual definitions or unique identifiers (besides the classes from PDO). I suggest the authors should consult the list of principles of good ontology design (http://www.obofoundry.org/principles/fp-000-summary.html) by the OBO Foundry (Smith et al., 2007).

In Section 4 System Requirements, and Figure 3 Use Case Diagram shows the primary users as being farmers, agronomists and scholars. The inclusion of knowledge engineers as “users” of the system does not make sense and could be removed. (P9, L34)

Figure 6 and its description seems out of place. I suggest moving it up to Section 4, where the RiceMan Application development is discussed.

Figure 8 needs to be cleaned up to improve readability and labeled with numbers similar to Figure 6.

Section 5, the details of the evaluation of the RiceMan and the ontologies is much too long and detailed. This could be summarized in a few paragraphs and the extensive details and tables could be presented in a supplementary file.

Finally, the opening statement of the Conclusion is inaccurate, as the RiceMan application was introduced in the 2019 paper by the same authors. Also, the authors suggest that “Though RiceDO and TreatO are built for RiceMan, they can be extensively applied and reused for other development of expert
systems for agriculture domain.” This is not true as these ontologies are presented, but may be possible with some revisions to bring them into compliance with accepted standards.

Literature cited

Cooper L, Meier A, Laporte M-A, Elser JL, Mungall C, Sinn BT, Cavaliere D, Carbon S, Dunn NA, Smith B, et al (2018) The Planteome database: an integrated resource for reference ontologies, plant genomics and phenomics. Nucleic Acids Research 46: D1168–D1180

Cooper L, Walls RL, Elser J, Gandolfo MA, Stevenson DW, Smith B, Preece J, Athreya B, Mungall CJ, Rensing S, et al (2013) The Plant Ontology as a tool for comparative plant anatomy and genomic analyses. Plant and Cell Physiology 54: e1–e1

Jearanaiwongkul W, Anutariya C, Andres F (2019) A Semantic-Based Framework for Rice Plant Disease Management. New Gener Comput 37: 499–523

Smith B, Ashburner M, Rosse C, Bard J, Bug W, Ceusters W, Goldberg LJ, Eilbeck K, Ireland A, Mungall CJ, et al (2007) The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotech 25: 1251–1255

Walls RL, Cooper L, Elser J, Gandolfo MA, Mungall CJ, Smith B, Stevenson DW, Jaiswal P (2019) The Plant Ontology Facilitates Comparisons of Plant Development Stages Across Species. Front Plant Sci. doi: 10.3389/fpls.2019.00631