Open Science Data: Impending changes to the review process at the Semantic Web journal

From its beginning, the Semantic Web journal has taken a lead in bringing more transparency to academic publishing, due to its open and transparent review process, and due to its transition to a gold standard open access journal. Truly Open Science, however, involves much more, and we are now turning our attention to another core aspect of Open Science: the stable provision of data and software relevant for a published paper.

Beginning May 1st (the date may still change slightly), the Semantic Web journal will therefore impose a requirement on authors to provide data and software relevant to a paper submission to the maximum extent to which this is feasible. Furthermore, data and software are required to be hosted at a long-term stable URL, without any subsequent changes allowed to be made. The adequacy of the provided data and software for assessment of the paper submission, as well as for the replication of any experiments reported on, will be assessed during peer review. While these requirements will be put in place May 1st, there will be a transition period for paper resubmissions, until August 31st, i.e. revised versions submitted until August 31st can, but do not have to, meet the new requirements.

A more detailed description of the new process and requirements, as they will initially be rolled out, is given below - and they will also be reflected on the For Authors and For Reviewers pages. We welcome any feedback and comments (contact the editors-in-chief), and we expect that our process and guidelines will be refined as we obtain more experience with it.

Changes pertaining to manuscript submission

Authors provide

  • URL for accompanying data and software (to one archive, e.g. zip, file).
  • Checkbox agreement that the provided URL will be long-term stable and that no changes to the data at that URL will be made after submitting the camera-ready manuscript after acceptance.
  • Checkbox agreement that these files will be backed up by SWJ upon acceptance of the manuscript, and will be provided by SWJ to requestors in case the author’s link goes dead. Author confirms that they have the legal rights to provide the data for this purpose.

Author instructions

  • Upon submission of a manuscript to the Semantic Web journal, authors are asked to provide access to all resources that are critical for the evaluation and reproduction of the presented work. The data and software you provide, together with detailed instructions, must be sufficient to replicate your experiments (if this is applicable), and/or to inspect any data artifacts (such as knowledge graphs or ontologies) you may have created and are reporting.
  • Authors are expected to provide these as a single file (e.g., a ZIP archive) via a publicly accessible URL. The resources must be well-organized within the file, and a README file should be provided with detailed guidance to ease usage. The URL should be long-term stable, such as through GitHub, Figshare, or Zenodo. Alternative services are acceptable but must be justified and be in some sort of similar repository format. The appropriateness of an alternative service will be subject to peer review.
  • Please note that the files should not be changed during review and after the final camera-ready version has been provided. The files can be reuploaded after each review phase to incorporate feedback from the reviewers. If provision of some resources is not possible, this should be addressed explicitly in the submitted manuscript and cover letter.
  • The Semantic Web journal will make a long-term back-up copy of this file.
  • Reviewers will include in their assessments the quality and organization of the resources provided by the authors, and indicate whether the provided materials are sufficient for replication of experiments.

Required data file

  • Authors are expected to provide these as a single file (e.g., a ZIP archive) via a publicly accessible URL. The URL should be long-term stable through GitHub, Figshare, or Zenodo. The resources must be well-organized within the file, and a README file should be provided with detailed guidance to ease usage.

Changes pertaining to the reviewing

Reviewer instructions

  • Authors are requested to provide a URL to a file (e.g., a ZIP archive) with resources. There are two reasons for this, namely
    • (1) in order to have available all resources required to replicate experiments, and
    • (2) in case the paper produces data artifacts (such as knowledge graphs or ontologies) so that there is a stable version available for future reference.
  • Please include in your review an assessment of the data file along the following dimensions:
    • (1) Organization of the file, i.e., it is the authors’ responsibility to organize the data file such that your assessment is not too complicated. There should be a README file.
    • (2) If applicable, whether the provided resources appear to be complete for replication of experiments and if not, why. We understand that this is difficult to assess with absolute certainty. Please make a reasonable assessment (e.g., based mainly on the README file), and indicate in the review to what depth you have done the assessment.
    • (3) Assess the appropriateness of the chosen repository if it is not GitHub, Figshare, or Zenodo. Does it follow repository-discoverability? Does it, or will it, have a DOI?
    • (4) If applicable, whether created data artifacts appear to be fully included in the file - if not, then the authors should give convincing reasons for this in the submitted paper. We understand that this is difficult to assess with absolute certainty. Please make a reasonable assessment (e.g., based mainly on the README file), and indicate in the review to what depth you have done the assessment.

Changes pertaining to the camera-ready version process

Authors provide

  • URL for accompanying resources (to one archive, e.g. ZIP archive).
  • Checkbox agreement that the provided URL will be long-term stable and that no changes to the data at that URL will be made after submitting the camera-ready manuscript after acceptance.
  • Checkbox agreement that these files will be backed up by SWJ upon acceptance of the manuscript, and will be provided by SWJ to requestors in case the author’s link goes dead. Author confirms that they have the legal rights to provide the data for this purpose.

New FAQ entries:

Q: I cannot share my data file through GitHub, Figshare, or Zenodo. What are acceptable alternatives?

A: If you cannot use GitHub, Figshare, or Zenodo, then in your cover letter or in the manuscript, justify the use of your chosen repository. In general, acceptable alternatives are established, domain repositories, e.g. OBO Foundry. Such repositories are expected to be long-term stable and transparently serve the hosted data. Adequacy of the repository will be assessed by the reviewers.

Q: I cannot share my data at all. What are the next steps?

A: We require that the authors share all relevant data. If this is not possible, please send an email to the Editors-in-Chief at contact @ semantic-web-journal.net with your reasons for not being able to do so, and we can discuss alternative options.

Acknowledgement: Realizing the described Open Science Data changes to the Semantic Web journal academic publishing process is supported by the National Science Foundation under Grant No. 2032628. Any opinions, findings, and conclusions or recommendations expressed herein are those of the authors and do not necessarily reflect the views of the National Science Foundation.

References

Pascal Hitzler, Krzysztof Janowicz, Cogan Shimizu, Lu Zhou, Andrew Eells, Open Science Data and the Semantic Web Journal. Editorial. Semantic Web 12 (3), 2021: 401-402.