Call for Artifacts

Research reproducibility is essential for scientific integrity and further scientific discovery. At ICSOC, we promote rigorous evaluation and sharing of data, code, and methodologies to enhance the credibility and impact of scientific contributions. To encourage reproducibility, ICSOC incorporates formal verification into its review process and introduces the respective badges.

Authors of accepted main track papers are invited to submit a companion artifact description paper to the special Artifact Evaluation track. These artifacts are independently reviewed and validate the submitted research artifacts. Accepted Artifact Evaluation paper will get either Open Artifacts or Verified Artifacts badge.

Submitted reproducible artifacts must link to all relevant research materials for the accepted main-track papers, such as datasets, source code, and supplementary materials, and must be shared via reputable, persistent public repositories. We recommend Zenodo for this purpose. Such public repositories offer stable, long-term access to research outputs, providing a reliable source for references and citations. To support open science, it is strongly advised that these artifacts be released under open data licenses, allowing for broad use and redistribution. The artifacts are supposed to enable the validation and reproduction of the main paper.

Hence, the Artifact Evaluation paper should include clear guidelines for using these linked materials. For executable artifacts (e.g., code), instructions on how to run them and how to interpret the results should be provided. The use of executable containers or VM images is encouraged. For data artifacts, a clear description (metadata) and information on the origins and provenance of the data must be given. For other types of artifacts (e.g., model specifications, standards), the authors should exercise their judgment in providing the information necessary to assess the validity of the respective artifacts.

In case a reproduction is only possible using specialized hardware or access to special resources, authors are encouraged to provide and explain the reproduction on a reasonable test environment (including the limitations of the environment for the results) and provide additional data artifacts to allow reviewers to validate the obtained results and the reproducibility of the artifact separately.

The artifact submissions will be evaluated based on the clarity of the instructions and the ability of the Artifact Evaluation Committee to verify the results reported in the original ICSOC paper. During the review phase, the evaluation committee may reach out to the authors if there are issues with the reproductions.

Important dates

All deadlines are in Anywhere on Earth time (AOE = GMT-12). Check the time in the AOE Zone here: https://time.is/Anywhere_on_Earth

Paper instructions

• Papers should be formatted according to Springer's LNCS Formatting Guidelines.
• Submissions must be in English and must not exceed 6 pages.
• The title of the paper MUST be "Artifact for '{Original title of the main ICSOC paper}'"
• In the paper, the authors should provide the following information:
  • A link to a public repository (i.e., Zenodo) containing the artifact(s). A DOI or permanent link is strongly encouraged.
  • Clear documentation and a step-by-step guide on how to reproduce the results reported in the paper.
  • (When applicable) A description of the experimental setup, including hardware and software dependencies. Instructions to run the code. Code containers are encouraged.
  • (When applicable) A description of the data origins and methods of data collection.
  • Licensing information to clarify usage rights.
  • A clear statement of any additional requirements (e.g., cloud service accounts, specific hardware) that may prevent reproducibility. When the work relies on cloud services, authors should provide instructions on how reviewers can replicate the environment at minimal cost (e.g., locally or free-tier services). If replicating the full original environment is practically infeasible, authors are encouraged to provide a simplified setup that allows to verify the basic executability and integrity of the code and provide instructions in provided data artifacts that allow the review and verification of the obtained results.

Submission instructions

All artifact papers must be submitted electronically to the Conference Submission System. The "Artifact Evaluation" track should be selected during submission.

Submission link: https://easychair.org/conferences/?conf=icsoc2026

The evaluation will be single-blind, and the Artifact Evaluation Committee will have access to the camera-ready version of the accepted main paper. This means there is no need to anonymize the artifact!

Contact information

For any questions regarding the Artifact Evaluation track, please contact the Reproducibility Chair, Sebastian Werner <werner@tu-berlin.de>.