(N.B the PASCAL challenge is now officially closed and the results are available here. Instructions and data are being left on-line so that the benefit of groups wishing to compare their algoriths with those that have been submitted.)
Challenge Guidelines
In order to reach a broad audience we have tried to avoid setting rules that might artificially disadvantage one research community over another. However, to keep the task as close to an application scenario as possible, and to allow systems to be broadly comparable, there are some guidelines that we would like participants to follow.
The systems should not exploit the SNR labels in the test data. Systems should not exploit the fact that the same utterances are used at each SNR. As already stated on the instructions page, the speaker identity is available from the utterance file name and you can assume this to be known. Noise background. The temporal location of the utterance has been given in order that the utterance can be located in the continuous CHiME audio. It is acceptable to exploit knowledge of the utterance's acoustic context (i.e. the audio occurring before and perhaps even after the utterance). There will probably be a lot of variation in the extent to which system's use this knowledge, so please be clear about the extent to which the context is used. Parameter tuning. We have provided the labels for the final test set so that participants can evaluate their own systems. However, please do not abuse this by tuning free parameters on the final test data. Tune free parameters on the development set and only run the final test set when you are satisfied with your system's tuning. We will expect to see results for both the development and test sets in the final papers.
Preparing your submission
Results can be submitted to the CHiME workshop in the form of either an extended abstracts for poster presentation (maximum 2 pages) or a full papers (maximum 6 pages). Manuscripts should be prepared using the same format as for Interspeech 2011 submissions. Please use the author kit from the Interspeech site, which includes templates for LaTeX and Microsoft Word.
Submit your paper
Please use the online submission system to upload your paper. The deadline for submissions has been extended until 21st April. Submission is now closed.
Submit your results
Recognition results: if you are producing recognition results directly then you should include them in your abstract or paper. To avoid any possible confusion, please also send a separate email to chime@dcs.shef.ac.uk reporting the keyword recognition accuracies for the test set at each of the six SNRs. Signals: If you are generating separated signals then please first try and use the evaluation tool provided in order to generate a recognition result. If you are unable to get the tool to produce a satisfactory result then please contact the organisers for advice and technical support. As a last resort we can arrange to collect your signals for remote evaluation.