ISWC 2022 Call For Challenges

A great way to advance the state of the art in a given domain is to create competition. We invite you to propose an ISWC 2022 Challenge, in which you define an open competition on a problem of your choice within the Semantic Web domain.

Important Dates – All deadlines are 23:59 AoE (anywhere on earth)

Submission Deadline for Challenge Proposals March 24, 2022
Notification of Challenge Acceptance April 5, 2022

Topics

For ISWC 2022, Challenge proposals are invited for any challenge involving Semantic Web tasks, including but not limited to

  • Ontology and knowledge graph alignment
  • Ontology and knowledge graph quality assurance (QA)
  • Knowledge graph construction and refinement
  • Query and reasoning scalability
  • Open information extraction
  • Semantic Web and data mining
  • Semantic Web, machine learning and neuro-symbolic AI
  • Link prediction
  • Question answering
  • Stream processing and reasoning
  • Semantic table understanding

ISWC 2022 will be a hybrid conference that will take place on 23-27 October online and in Huangzhou. Recent Challenge examples can be found at:

https://iswc2021.semanticweb.org/challenges, 

https://iswc2020.semanticweb.org/program/semantic-web-challenges/

Proposal Stage

ISWC 2022 will run multiple challenges with the aim to evaluate and compare software solutions for the Semantic Web and Knowledge Graphs in a systematic way. In order to propose a challenge, we ask you to:

  • specify a task to be addressed
  • provide an evaluation dataset
  • define evaluation measures to compare the performance of participating systems​

Challenge Stage

After acceptance of the challenge to ISWC 2022, challenge organizers are responsible for:

  • providing a website
  • publishing challenge tasks, datasets, and measures
  • selecting participants who can present their solutions at the ISWC 2022

Showcase Stage

At ISWC 2022, each challenge will receive one slot during the main conference, during which contestants present their accepted solutions. Participants will also have the option to present their systems during the poster and demo session.

Proposal Submission Guidelines

The contents of a proposal should include the following (in 2–4 pages)

  • names and affiliations of the organizers
  • description of the challenge, including its relevance to the Semantic Web community
  • process for selecting, acquiring, and preparing the data for training and evaluation, including conditions of availability. It is strongly recommended, but not required, to describe the data in relation to the FAIR principles.
  • procedure for evaluating the performance of participating systems, including the availability of evaluation software
  • a timeline based on the template for all Challenges provided below
  • number of expected participants with supporting evidence (expression of interests, discussions in forum/mailing lists, attendance in previous editions, etc.)
  • plan for dissemination of the challenge (targeted mailing lists, social media, etc.)
  • any other relevant information, including previous evaluations for the proposed challenge

Selection criteria

  • Relevance to the Semantic Web community
  • Potential number of interested participants
  • Rigor and transparency of task description and evaluation procedure

Please submit through EasyChair and select the track “Challenge-Proposal”.

https://easychair.org/conferences/?conf=iswc2022 

Tentative timeline template for Challenge Organizers

The tentative timeline template for organizing a Challenge is as follows: All deadlines are 23:59 AoE (anywhere on earth).

Activity Due date
Challenge Website ready 28 April 2022
1st call for challenge participants 28 April 2022
Release of the dataset 13 May 2022
Submission of systems 14 July 2022
Notification of system results 15 August 2022
Participation invitations to present at ISWC 2022 15 August 2022
Challenge presentations 23-27 October 2022

Note the deadlines and mostly the dataset release deadline and system submission deadline can be flexibly adjusted according to the challenge, especially if the challenge includes several evaluation rounds (e.g., http://www.cs.ox.ac.uk/isg/challenges/sem-tab/2020/index.html)

Semantic Web Challenge Chairs

Catia Pesquita
LASIGE, Faculdade de Ciências, University of Lisbon
clpesquita@fc.ul.pt

Daniele Dell’Aglio
Aalborg University, Denmark
dade@cs.aau.dk