CHiME-9 Call for task proposals

CHiME is a challenge series whose purpose is to produce scientific advances in the field of distant microphone speech processing. Since CHiME-7, each edition of the CHiME Challenge features independent tasks organised by teams who work within a schedule defined by the CHiME Steering Group. Teams are responsible for managing the tasks, i.e., providing data and instructions, scoring submissions and results, etc. Once the proposal is conditionally accepted, the CHiME Steering Group works to support teams in the development of their task by helping with the design, schedule, webpage/platform and organise a CHiME workshop where the tasks will be presented via regular consultations.

The CHiME Steering group is now inviting task proposals for the CHiME-9 Challenge. Proposed tasks will be evaluated according to the following criteria:

  • relevance to the CHiME goals
  • methodology - i.e., likelihood to lead to robust and novel scientific findings
  • accessibility - i.e., how easy is it for teams to engage with
  • feasibility - likelihood to complete on schedule, risk management
  • reproducibility - e.g., openness of data, baseline
  • diversity - amongst organisers and expected participants, e.g. academia vs industry, geography, gender etc.

We wish to foster a collaborative proposal process that minimises overlap between proposals. To this end, if you are considering proposing a task, please sign up to the CHiME mailing list and feel free to discuss your plans. Or if you wish information to be shared more privately, i.e., amongst other bidders only, then you may share details or initial plans with the CHIME Steering group who will then share with others whose plans may overlap.

Proposal

Task bidders should submit a task proposal for review by the Steering Committee. The proposal should be a maximum of two pages (pdf format preferred) including the following information:

  • Task name
  • Coordinators and their roles
  • Keywords (e.g. enhancement, diarization, speaker recognition, etc)
  • Definition (one sentence)
  • Short description (incl. research question the task is tackling)
  • Dataset description: development, evaluation (short description, how much data is already available and prepared, how long would it take to prepare the rest, mention if you allow external data/transfer learning or not)
  • Evaluation method / metric
  • Baseline system (brief description, planned method if you do not have one from a previous challenge, current readiness including an expected readiness date)
  • Schedule for the challenge preparation including the expected timeline for the data collection and annotation of the training, development and test sets, baseline preparation and testing, etc.
  • List of invited participants who are likely to participate
  • Contact person (for main communication, website)

Pitching Session

We will be running an optional “proposal pitching” session at the CHiME 2024 workshop in Kos, Greece on 6th September. This will be an opportunity for teams who are developing a submission to get some feedback from members of the CHiME community. If you are interested in presenting in this session please contact us to discuss by 26th August.

Important Dates

For the CHiME-9 schedule, see the CHiME important dates page.

Submission

The proposal should be sent to the CHiME Steering group (chimechallenge@gmail.com) by 25th October. The Steering Group will review the proposals and provide feedback by 8th November. In case task amendments are requested, the task acceptance decision will be finalized by 4th December.

If you want any advice on preparing your proposal, please do not hesitate to contact us.