Review Comment:
Summary: the paper describes a new ontology for Quranic Recitation (Tajweed). There are many knowledge sources derived from Islamic heritage such as primary sources of the Quran and the Hadith, but there are insufficient semantic models that describe them for purposes of semantic annotation, and subsequent information retrieval and knowledge-based reasoning. To validate the ontology, they use it to annotate the Quran, and validate the model using a criteria-based approach in combination with expert validation and data-driven validation.
Although I am not an expert on Quranic recitation, the design choices seem well motivated, the work seems to be comprehensive and a relevant extension to existing work. The work has been carefully evaluated in a hybrid approach, both qualitatively with domain experts as well as quantitatively, which I appreciate. The only small concerns relating to the evaluation I have are that (i) the user scenarios and competency questions seem to have been developed by the authors themselves (at least, it was not mentioned how they came to be) whereas it would have been interesting when these would have come from the domain, and (ii) due to my lack of domain knowledge, the data-driven evaluation was hard to follow.
My major concern with the paper is that (i) the motivation and domain of Quranic recitation, as well as the system architecture and annotation process, are not described clearly, (ii) the unclear writing style and paper structure make the paper a tough read, and (iii) there are many spelling and grammatical errors. Additionally, the paper could be written more concisely. The SWJ journal mentions descriptions of ontologies should be short papers, whereas this paper consists of 22 pages. Detailed arguments could potentially be submitted as supplementary files.
More specifically:
- convincing evidence of the relevance and quality of the ontology must be given, and although the authors have (IMHO) succeeded in the latter, the former should be addressed. Some use cases are briefly enumerated later, but the introduction would benefit from a clear description of the field (on rules and the recitation process) and a motivation for how the ontology can be used in practice.
- the structure of the text is not always logical and consistent. E.g., the section 'system architecture' for instance contains descriptions related to the annotation process, data cleaning, and evaluation.
- consistency and correctness of terminology, spelling and use of capital letters is lacking (e.g., the tajweed ontology, Tajweed ontology, the Tajweed Ontology, the Tajweed ontology model, the ontology-based quranic tajweed knowledge model, the Tajweed ontology knowledge model).
- Often, the determinant of a sentence is missing (e.g., page 12 line 30 'Rule engine --> 'The rule engine')
- there are many mistakes in use of time and plural/singular (Table 5, SWRL iqlab rule: 'This rule implies [...] and have' --> 'This rule implies [...] and has')
I have annotated some spelling and grammatical mistakes only for the abstract and first section, to show examples.
ABSTRACT
Specific comments:
Semantic Web Rule Language --> The Semantic Web Rule Language
Expert driven --> Expert-driven
criteria based --> criteria-based
structure of ontology --> structure of the ontology
the ontology model --> the ontology (for consistency)
results from the experts --> results from the expert-driven evaluation
Unclear and grammatically incorrect sentence: "Results from the experts (?) were incrementally improved before evaluating it (?) with the next expert which (?) results in 100% accuracy."
Unclear and grammatically incorrect sentence: "Tajweed rules were evaluated ... Holy Quran."
data driven --> data-driven
INTRODUCTION
General comments:
* I suggest the authors add one or more references related to keyword search through Quranic heritage, and Tajweed in specific. In general it would be interesting to have a short description on the relevance of semantic search over keyword search in Quranic texts.
* Many readers might have no background and no knowledge on the content/ structure and appearance of these texts, nor of "recitation" in general -> what is Tanween or Un vowel noon, what do articulation points, characteristics, letter occurrences and rules mean in this context? Part of this could be further explained in the background, but the relevance of a semantic model for the Tajweed should be made clear early on in the introduction. Some of this information is actually described in the github link containing the Tajweed ontology.
* I suggest not to use letters as well as numbers in the same enumeration.
Specific comments:
line 48 col 1: will make easier --> will make it easier, the intelligent systems --> intelligent systems
Line 38 col 2: literal meaning --> The literal meaning
Line 41-42 col 2: ungrammatical/unclear sentence: from which it originates (?) and have some characteristics (?)
Line 47 col 2: matching keywords approach --> could benefit from a reference
OWL -> would be better to write it out when mentioned for the first time and link to the OWL specifications.
Page 2 line 2-3 col 1: ontology based --> ontology-based
Page 2 line 9 col 1: a complete annotated Quranic Tajweed dataset has been constructed of high accuracy --> unclear. What was annotated and what does it mean to have a high accuracy dataset.
Bhybrid approach --> a hybrid approach
over-view of the literature review work --> overview of the literature
Background and related work
* The related work would benefit from a section that describes the body of literature related to for instance Linked Data and semantic annotation of cultural heritage. I would personally place section 2.1 on ontology engineering methodologies in the methodology section, as it relates more to the methodology you have chosen than the scope of your research.
* I suggest to start with a section on the background of Quranic recitation. For a layman it is difficult to figure out what articulation points, characteristics, letter occurrences and rules are in the domain of Quranic Tajweed, and how they are useful.
* I suggest to revise section 2.2, since the title does not seem to reflect the content, and is therefore difficult to understand.
Section 3:
* I suggest the authors split this section into fewer sections, for instance a development process section and a model section which would describe modeling decisions. These could then be further subdivided using your subsections.
* I suggest the authors rewrite the the rules in Table 5, as they are long and difficult to understand by a layman.
Section 4:
This section describes the system architecture and the annotation process using the ontology
* Some terms are mentioned but not explained (Tajweed Factory, Automated Ontology Population)
* The section is difficult to follow. I suggest the author organise it according to Figure 9., as at the moment, section 4 as well as Figure 9 are difficult to understand. What makes section 4 unclear is that there is no distinction between the architecture of the system (which components does it contain), the data processing step, and the actual annotation process (how is that system used for annotation, by whom, was there agreement on annotation between annotators (IAA), what type of data does it ingest in what form, etc). Figure 9 does not seem to be supported by the text.
A separate description of the dataset that you annotate (examples of the Quranic Tajweed, I see a short description occurs later in section 5.3), the system, the process, and examples of the resulting output dataset, would greatly help.
Additionally, some parts of this Architecture section already contain sentences about validation ("When validating the results on the Quranic text, the rule of Noon-Sakinah and MeemSakinah was not predicted") which is discussed only later in section 5. These parts are difficult to interpret, as we do not know yet what this validation looks like and how it is performed.
Section 5:
Completeness: [...] by using SPARQL queries -> what was the outcome? How did you assess completeness this way?
* Data-driven evaluation: are you comparing the rules in your populated ontology with those in the data source for which the github link is provided? I suggest the authors provide a clearer description of the evaluation task, and a more consistent naming for their ontology as well as populated ontology, so that it is clear which resources are compared.
Data file assessment:
1. https://github.com/ramshaamin/ArabicLettersOntology
The folder contains an OWL ontology accompanied by a README.md containing a clear description of the background and available resources. The conceptual image however is unreadable due to the use of a black background, and the README could benefit from a per-class or predicate description with for instance their namespaces, related properties/classes and domains and ranges.
2. https://github.com/ramshaamin/TajweedThesisV5/
The folder lacks a README.md, making it difficult to figure out what is in the data folder, and how to run it. Java code which integrates OWL API libraries in an eclipse project appear to be there as well, but these are only briefly mentioned in the paper without explanation on how to set the environment up (which software is used, what are the prerequisites). Some folders appear to refer to a Tajweed Factory, which is also briefly mentioned by name in the paper, but is not further explained.
From looking through the files, it appears that most of the files mentioned in the paper appear to be in the folder: the Tajweed ontology, the rules and the Tajweed factory. Although the paper mentions the annotated Quran dataset (the populated Tajweed ontology) should be in either one of these folders, but seems to be missing. Preferably provide it as a SPARQL endpoint. A nice addition would be some example queries to run on top of the populated ontology.
|