Review Comment:
Second Review for Teaching Knowledge Graph for Knowledge Graphs Education
Review #1
- Reviewer Concern: It aims to centralize and structure educational materials related to KGs but it lacks sufficient evaluation, real-world validation, and a clear comparison with prior educational KGs
- Comments on revision: Partially addressed. Revision added a SUS study.
- Reviewer Concern: Additionally, unclear scalability strategy, and difficulty in reproducibility, missing discussions on licensing, privacy, and governance reduces the paper’s impact.
- Comments on revision: Partially addressed. Added more details on licensing, privacy, and reproducibility, but the maintenance and scaling strategy remains unclear.
- Reviewer Concern: While the literature review is thorough, it does not extensively compare its ontology to existing educational ontologies (e.g., EduKG, KnowEdu). A comparative analysis would strengthen the argument and position it better.
- Comments on revision: Addressed sufficiently with explicit gaps called out.
- Reviewer Concern: There is no discussion on how this ontology improves upon existing educational schemas. A direct comparison with other KG-based educational models would strengthen the argument.
- Comments on revision: Addressed sufficiently
- Reviewer Concern: The proposed approach reads incremental rather than as a fundamentally new paradigm.
- Comments on revision: Partially addressed with the addition of implementation details in methodology.
- Reviewer Concern: Does not clearly differentiate how this is better from prior works such as mEducator and EOKG.
- Comments on revision: Partially addressed (mEducator differentiation still missing and would benefit from additional detail).
- Reviewer Concern: Lacks exploration or insights into external integration with existing educational platforms (e.g., Coursera, edX, Moodle, or university repositories).
- Comments on revision: Addresses with the new Ethics and legal concerns section.
- Reviewer Concern: There is no demonstration of how this system improves learning outcomes, which weakens its practical relevance.
- Comments on revision: Partially addressed. Goal’s definition made explicit as to capture the learning outcomes rather than enforce a specific instructional learning outcome
- Reviewer Concern: Quite a few minor typos impacting readability.
- Comments on revision: Partially addressed. Minor typos persist (e.g. adresses). Many broken references in the paper for Listings, Figures, etc.
- Reviewer Concern: How exactly the system doesn't fail when simple SPARQL queries fail is not mentioned (similarity check etc)
- Comments on revision: Not addressed in the revision.
- Reviewer Concern: The semantic similarity approach for course matching is briefly mentioned but not tested for accuracy or effectiveness, how effective is the SBERT model mentioned for similarity check between course title and user search; there is no quantitative evaluation of how well it matches resources.
- Comments on revision: Only partially addressed in the revision, though the section stating this has been restructured instead.
- Reviewer Concern: The interface usability testing is not detailed. A formal usability evaluation (e.g., System Usability Scale) would strengthen the argument for adoption.
- Comments on revision: Addressed sufficiently, but has a missing link to a figure.
- Reviewer Concern: Lacks proper documentation (no readmes or empty readme files), making it difficult for a novice to reproduce or track the steps for reproducing
- Comments on revision: Addressed with Git repo readme
- Reviewer Concern: No benchmark datasets are provided in the repo to validate the ontology and query performance.
- Comments on revision: Not addressed in the revision.
- Reviewer Concern: How scalable is the ontology? What happens when hundreds of new courses/knowledge are added that can't conform to the same required field structure?
- Comments on revision: Not addressed in the revision. Scalability and maintenance plans are not explicit.
- Reviewer Concern: What are the accuracy and recall rates of the semantic similarity search?
- Comments on revision: Only partially addressed in the revision. Only SUS and qualitative feedback were provided.
- Reviewer Concern: Is there a reason why there is no discussion on integrating this KG leveraging existing educational platforms?
- Comments on revision: Addresses with the new Ethics and legal concerns section.
- Reviewer Concern: How is personal instructor (metadata for course) data stored, and what are the privacy and licensing policies?
- Comments on revision: Addressed. Minimal public instructor data is stored, and everything is linked to a public reference for privacy and license management.
- Reviewer Concern: Key Revisions Needed:
- Provide more details on evaluation (e.g., user studies, ontology validation, benchmarking).
- Provide more details on comparison with existing educational KGs.
- Provide more details on scalability and governance concerns.
- Provide more details on data privacy and licensing issues.
- Comments on revision: Partially addressed. Benchmarking and scaling plans are still missing. It would benefit from additional detail.
Review #2
- Reviewer Concern: One key area for improvement in the introduction is the early definition of key educational terms.
- Comments on revision: Addressed sufficiently.
- Reviewer Concern: Another point that needs more balance is the discussion of intended users. Lines 24-33 focus heavily on how teachers will use the KG, but there’s little mention of students or learners.
- Comments on revision: Partially addressed. Additional detail on the usefulness with links back to the original motivation usecases would benefit the paper.
- Reviewer Concern: On the technical and methodological side, the introduction could benefit from more details about data collection and the KG construction pipeline.
- Comments on revision: Addressed sufficiently with abstract mentions and contributions, which are later explained.
- Reviewer Concern: Finally, the list of contributions could use some expansion. The paper presents a lot of technical and methodological details, but the introduction doesn’t fully capture their scope.
- Comments on revision: Sufficiently addresses the concerns enough for the introduction.
- Reviewer Concern: The Related Work section has areas where it could be clearer and more structured.
- Comments on revision: Addressed sufficiently.
- Reviewer Concern: Following this classification, the discussion appears to focus primarily on literature related to assisted learning; the section tends to list prior work without critically engaging with it or clearly positioning it within the context of the current research.
- Comments on revision: Addressed sufficiently. Related work section extended to increase breadth.
- Reviewer Concern: Section 3.1. Second paragraph: Given that a semi-structured approach to construct the KG was used, the phrase "generating an educational teaching KG" might not be the most precise wording.
- Comments on revision: Addressed by rephrasing accordingly.
- Reviewer Concern: Section 3.1. Second paragraph: “Although there are some recent courses ….by the semantic web community”. Is there any way to support this claim?
- Comments on revision: Addressed and rephrased to give examples and more details.
- Reviewer Concern: Section 3.1. Second paragraph: “Based on input we received…” this lacks specificity regarding the nature of the input received.
- Comments on revision: Addressed by rephrasing accordingly.
- Reviewer Concern: Section 3.1. Second paragraph: Please include links for both the Alan Turing Institute and the COST Action on Distributed Knowledge Graphs (DKG).
- Comments on revision: Addressed.
- Reviewer Concern: Section 3.2. "capturing intricate relationships" and "enabling semantic interoperability", this could be more precisely defined.
- Comments on revision: Addressed. Expanded with example based explanation.
- Reviewer Concern: The research questions would benefit from greater precision and clearer definitions of key terms. In RQ1, for example, the phrase "minimum pedagogical requirements" is quite broad and could mean different things.
- Comments on revision: Partially addressed. Additional details on the definitions of the terms used in RQs specific to the current paper context would benefit readability.
- Reviewer Concern: Section 3.3. The phrase "supplementary resources were obtained from authors’ private lists with resources" requires elaboration. Lines 25–27: It is essential to be absolutely clear about the sources of the collected data. The Alan Turing Institute, COST Action DKG, and publicly available courses are mentioned, but further clarification is needed.
- Comments on revision: Addressed sufficiently.
- Reviewer Concern: Regarding the data collection form: • Was this distributed online or in a paper format? • How were Semantic Web experts identified and contacted for participation? • The mention of "requesting personal information" needs clarification. What specific types of personal data were collected, and for what purpose? • The reference to URLs is vague, does this mean URLs of course webpages, specific course materials, lecture recordings, or other resources? This should be made explicit. • The "optional template" needs clearer explanation.
- Comments on revision: Partially addressed.
- Reviewer Concern: The phrase "by a group of three participants" should specify the nature of their collaboration. How was this collaboration organised?; The sentence "The collection of educational materials began with voluntary contributions from the course instructors" should appear earlier in the section as it logically introduces the data collection process.
- Comments on revision: Partially addressed.
- Reviewer Concern: Regarding Table 1, the explanation lacks detail on how this information was presented to the experts and how it guided them in the data collection process. More context would help readers understand its role in guiding expert contributions.
How were the data collected from the Alan Turing Institute group?
- Comments on revision: Not addressed in the revision.
- Reviewer Concern: Section 3.4. What “diverse formats” were collected? Please give examples. A short overview of the referenced paper [1] here would be helpful, along with a clear signpost to Table 5 for better readability. Line 11: What type of raw data is being referred to? Please specify.
The sentence "We implemented a data pipeline to enable..." would benefit from a signpost directing readers to Section 5 for further details.
- Comments on revision: Not addressed in the revision.
- Reviewer Concern: Section 3.5."As Semantic Web experts are typically not trained..." Aren’t they the instructors mentioned in the previous section? If so, wouldn’t they already be familiar with educational parameters? Or are you referring to researchers and practitioners in general? If it’s the latter, consider using a different term to avoid confusion with the previously mentioned participants. Line 24: What specific educational parameters are being referred to? Please clarify.
- Comments on revision: Rephrased to adequately explain. Addressed
- Reviewer Concern: Section 4.1. Line 2 – Please add a signpost to Table 5.
- Comments on revision: Unclear
- Reviewer Concern: Section 4.2. The definitions of skill and knowledge should be introduced earlier. Lines 31–32: The last module should be written as a new paragraph for consistency. Lines 32–33: Who carried out the annotation? Please specify. Section 4.2: Please provide concrete examples for each module by referencing specific ontological components that were defined. Line 42: "The classification was developed through an iterative process, leveraging both pedagogical expertise and feedback from project stakeholders." Could you elaborate on how this iterative process was conducted and what specific input was incorporated?
Page 11 Line 4: What constitutes the bare minimum? Please elaborate.
- Comments on revision: Partially addressed. Defines knowledge topics and skills directly in the problem framing of Section 1. Additional details added
- Reviewer Concern: Section 4.3.
Lines 14–15: Explain the reasoning behind the shift from slides to theoretical educational material. Is this change influenced by the structure of the vocabulary used, please explain.
SPARQL queries 4.3.1(d) and 4.3.2(e) are essentially addressing the same query but for different entities (topic vs. course). Please ensure consistency in the terminology used to refer to "laboratory".
The section states, "We group the SPARQL queries thematically as presented in Table 5" which lists four groups: sub(topic), course, dataset, and material. However, the section only presents SPARQL queries for sub(topic), course, and material. Where are the SPARQL queries for the dataset?
- Comments on revision: Partially addressed. Consistency is still not maintained across
- Reviewer Concern: Section 5.1.
It would be helpful to include a brief overview of the LOT methodology, along with its four key activities, to give readers a clearer understanding of its workflow and relevance to the study.
- Comments on revision: partially addressed. Included an abstract conceptual flow.
- Reviewer Concern: Section 5.1.1.
Line 2: please specify the format/ formats of the semi-structured input.
This section is quite brief, especially considering that it’s the only place where the technical details of the process are discussed. While it outlines the general methodology and tools used, it lacks the depth needed to make the approach fully clear. Expanding on each step with more detailed explanations and concrete examples would not only improve clarity but also enhance the reproducibility of the framework, which is one of the key contributions of the paper. If the goal is to provide a practical guide for constructing educational knowledge graphs, then adding more details would make the framework more actionable for others looking to implement it. A major limitation of this section is that while it lists tools like OWL2YARRRML, Yatter, and Morph-KGC, it doesn’t illustrate how they work in practice. For instance, when discussing the transformation of semi-structured data into RDF, an actual example dataset could be introduced, perhaps a simple table of educational concepts or learning resources. The authors could then walk through how this dataset is mapped using YARRRML templates, translated into RML, and eventually converted into RDF triples. Including a small snippet of a YARRRML mapping template and its corresponding RML translation would be particularly helpful to give readers a practical sense of how the transformation process is structured. Additionally, this section would greatly benefit from a figure that visually represents the transformation pipeline. Given that multiple tools are involved, a diagram showing how data flows from its semi-structured form through the mapping and transformation process into RDF would help clarify the steps.
- Comments on revision: Not addressed in the revision.
- Reviewer Concern: The statement "we also ensure high performance and scalability during the transformation process thanks to Morph-KGC" is lacks empirical support. Does Morph-KGC achieve better scalability compared to other RML engines? If there are benchmarks or performance comparisons available, including those results would strengthen the claim and provide readers with concrete evidence of its advantages.
- Comments on revision: Removed references to Morph-KGC. Partially addressed.
- Reviewer Concern: The proposed use of LOT4KG and OCP2KG for schema-level changes is well-founded, but the specific mechanism by which these tools "automatically propagate ontology changes over RML mappings" should be briefly elaborated. Do they merely update mappings, or do they also validate consistency with existing RDF data?
- Comments on revision: The revision resolves this issue by removing the section and restructuring the surrounding content. Partially addressed.
- Reviewer Concern: The explanation of life cycle changes is vague, particularly when discussing how modifications at the data level will be handled. The phrase "there is still missing a novel solution" is ambiguous, does this mean that no existing methods are sufficient, or that no framework currently exists to address this issue? The mention of IncRML for incremental updates is promising but lacks details. How does IncRML determine which triples need to be regenerated? Does it work based on a versioning mechanism, a change-tracking log, or another strategy?
- Comments on revision: The revision resolves this issue by removing the section and restructuring the surrounding content.
- Reviewer Concern: Section 5.2.1.
The of the interface design process is quite broad and lacks the depth needed to fully explain how the interdisciplinary collaboration informed the final design. It highlights the involvement of experts in pedagogy, the Semantic Web, teaching, and software development, however, it does not specify how their expertise influenced decision-making. What specific contributions did each domain bring to the design process? Another key aspect missing from this description is how the experts were recruited and selected.
- Comments on revision: The revision resolves this issue by removing the section and restructuring the surrounding content.
- Reviewer Concern: Section 5.2.2.
This section lacks some details. Was a pre-trained SBERT model used or has it been fine-tuned on a domain-specific dataset. Also, given that the system is expected to scale, mentioning the storage mechanism would be useful, so, how is the latent space structured? Is it stored as a simple lookup table, a graph-based structure …?
Regarding the fact that the similarity computation is extended to sub-topics and educational resources, are all these entities embedded separately, or is there a shared representation across courses, topics and materials? If similarity is computed at multiple levels, how is the final ranking determined?
- Comments on revision: The revision resolves this issue by removing the section and restructuring the surrounding content.
- Reviewer Concern: Section 6.1.
The passage claims that enabling an "ontology-driven data creation pipeline will minimise errors in the data." However, it does not define which errors are being minimised and how OWL's reasoning ensures correctness.
- Comments on revision: Addressed by examples of error type definition.
- Reviewer Concern: Section 6.2.
The section does not specify how the two stakeholders were identified and recruited.
The section states that only two stakeholders were interviewed. This is a very small sample for deriving meaningful conclusions. Please explain why two stakeholders were deemed sufficient, and If constraints such as time and availability limited the sample size, acknowledge this as a limitation.
The section did not the method of analysis (thematic analysis? coding scheme?).
The section does not clearly distinguish whether the stakeholders are evaluating the actual KG or providing conceptual input on what an ideal Teaching KG should include. Please make this explicit.
The phrase "to inform the design and implementation" is vague. Does this mean the KG is already in development, or is stakeholder input being gathered before any KG is built?
Before introducing the questions, it would be better to describe the key themes that were targeted when designing the interviews (similar to what we see in Table 10).
- Comments on revision: Partially addressed. Still missing detailed information on recruitment and stakeholder information, and the overall process. It would benefit from additional detail.
- Reviewer Concern: Section 6.2.1.
This section requires significant restructuring for better clarity and readability. In addition to the typos and grammatical errors that need to be corrected, the overall flow is difficult to follow. The current format makes the responses feel like a rigid Q&A list rather than a natural discussion of stakeholder insights. One major issue is inconsistency in attribution. The section first states that findings come from "the interview 1", but later refers to "participants suggested including labs...", it is unclear whether responses came from a single interviewee or both.
- Comments on revision: Addressed by removal and restructure.
- Reviewer Concern: Furthermore, the use of numbers in parentheses (e.g Q2, Q3, …) to refer to questions is confusing and disrupts readability. If these numbers indicate participant responses to specific questions, this should be made explicit. However, I strongly recommend removing this notation and instead integrating brief descriptions of the questions into the discussion for a more natural flow.
- Comments on revision: Addressed. Restructured into a table.
- Reviewer Concern: Additionally, not all answers to the 13 questions were included and there is no explanation for why certain responses were omitted. This omission raises concerns about selective reporting and should be clarified.
- Comments on revision: Partially addressed. Explains the number of questions but not if any responses are omitted.
- Reviewer Concern: To improve structure and readability, I suggest organising the findings into thematic categories rather than presenting them as a loosely connected list. This would provide a clearer narrative structure and make it easier to follow key insights. Also, stakeholder insights could be reframed as a set of requirements, which could then be mapped to the KG in Section 6.2.2 for better alignment between feedback and implementation.
- Comments on revision: Addressed with themes made explicit.
- Reviewer Concern: Finally, were there any overlapping opinions between the two interviewees? If so, this should be highlighted to emphasise common themes rather than treating responses as isolated inputs.
- Comments on revision: Partially addressed with 4 common themes.
- Reviewer Concern: Section 6.2.2.
These thematic categories should be explicitly linked to the interview questions and used to guide the structure of the previous section. Currently, the text does not specify how these themes were derived, making it unclear whether they emerged organically from the data or were predefined.
- Comments on revision: Theme derivation, tables 10 & 11 address this.
- Reviewer Concern: Section 7
The section begins with the statement "…we describe four use cases" yet only three use cases are actually presented.
This section requires significant restructuring as it does not fit well within the overall flow of the paper. The primary purpose of this section is to demonstrate the functionalities of the KG interface, yet these functionalities were already described in Section 5.2. Presenting them again at this stage disrupts the logical sequence of the paper, especially since the evaluation and results have already been discussed. This section feels a bit out of place. Moving it earlier might help the flow.
- Comments on revision: Addressed by restructuring and removing the section
- Reviewer Concern: Additionally, many of the functionalities described here have already been introduced earlier, such as presenting search results as tables or graphs (which has been described in use case 1 and 3!). Repeating this information does not add value and creates unnecessary redundancy. The screenshots of the interface are already informative and sufficient in illustrating the system’s capabilities, making this section largely unnecessary. Instead of repeating these details, I suggest incorporating a more detailed depiction of the interface into Section 5.2 for a more cohesive presentation.
- Comments on revision: Addressed by restructuring and removing the section
- Reviewer Concern: Furthermore, rather than demonstrating advanced capabilities of the Teaching KG, the use cases primarily describe basic UI interactions. These descriptions do not effectively highlight the system’s unique strengths or advanced functionalities. If use cases are to be included, they should focus on more complex, value-adding scenarios, such as advanced querying or integration with external resources. Otherwise, they should be omitted entirely in favor of a stronger system description in Section 5.2.
- Comments on revision: Addressed by restructuring and removing the section
- Reviewer Concern: In general, the placement of this section after the evaluation and results feels out of place. Readers would typically expect a discussion or conclusion at this stage, rather than the introduction of new interface details. If this section is necessary, it should appear earlier in the paper, within the system description in section 5.2. Otherwise, I strongly recommend removing it and integrating relevant content into the earlier sections where it logically fits.
- Comments on revision: Addressed by restructuring
- Reviewer Concern: Section 8
given the nature of the evaluation, certain claims should be more cautiously framed to reflect the preliminary nature of the findings. Currently, the evaluation is based on a very limited number of participants who did not actually use the KG in practice. As a result, the insights gathered should be considered exploratory rather than confirmatory. The authors must indicate that these results do not fully validate the effectiveness of the KG in real-world educational settings, but rather offer preliminary feedback that informs future improvements.
- Comments on revision: Partially addressed. Phrased as an alpha prototype from preliminary feedback and motivation.
- Reviewer Concern: GitHub repo: Please expand the README file to provide comprehensive information about the project, including setup instructions and descriptions of each directory and file.
- Comments on revision: Addressed sufficiently.
Additional review comments:
We kindly suggest that the authors carefully proofread the manuscript to ensure that all references to figures, tables, and sections are accurate and that no broken or missing links remain. Additionally, please verify that the restructuring of the previous Section 7 has not resulted in the omission of important details and that the novelty and usefulness of the work are clearly preserved.
| 
