Submission

What do I need to prepare for evaluation?

The evaluation involves two types of wearable devices:

  • Aria smart glasses
  • Hearing Aid (HA) devices

Each device will be evaluated separately, but we encourage participants to prepare signals for both devices.

Required Output

For each device and each session, you must submit three audio signals corresponding to the people the device-wearer is conversing with.

You can identify the relevant speakers using the metadata file:
📄 metadata/sessions.dev.csv

This CSV file provides:

  • The session ID
  • The positions of the Aria and HA device wearers
  • The participant IDs (Pxxx) in each position

Example sessions.dev.csv 

session,aria_pos,ha_pos,pos1,pos2,pos3,pos14
dev_02,3,4,P094,P093,P091,P092
dev_03,2,1,P097,P096,P095,P098
dev_04,3,1,P118,P115,P116,P117
dev_05,1,2,P124,P123,P125,P126
dev_06,4,3,P134,P131,P132,P133
dev_07,1,3,P149,P147,P148,P150
dev_09,2,4,P160,P161,P159,P162
dev_10,3,4,P179,P181,P182,P180
dev_11,2,1,P186,P184,P183,P185
dev_12,1,2,P187,P188,P189,P190

For example, in session dev_02:

  • The Aria wearer is in position 3 (P091) and is speaking with P092, P093, and P094
  • The HA wearer is in position 4 (P092) and is speaking with P091, P093, and P094

File Naming Format

Use the following format for each audio signal:

<session_ID>.<device>.<participant_ID>.wav

For example, for session dev_02, you need to prepare the following signals.

Aria signals:

  • dev_02.aria.P092.wav
  • dev_02.aria.P093.wav
  • dev_02.aria.P094.wav

HA signals:

  • dev_02.ha.P091.wav
  • dev_02.ha.P093.wav
  • dev_02.ha.P094.wav

Additional Requirements

  • 🎧 Sampling Rate: 16 kHz
  • ⏱️ Duration: Full session length (36.5 minutes per session)
  • 📂 File Organization: All signals should be placed in the same directory
  • 📊 Total Files:
    • Dev Set: 12 sessions → 36 signals per device
    • Eval Set: 9 sessions → 27 signals per device

How do I evaluate my signals?

  • Submission instructions for formal evaluation of the evaluation set will be provided closer to the deadline.

  • To evaluate the development set, place all the enhanced signals into the same directory and then use the evaluation code that has been provided along with the baseline system code (see Baseline System.