Results

Results are presented in two sets of tables: Unconstrained LM - where teams are allowed to modify the baseline LM; Baseline LM - where the use of the baseline LM is enforced. In the unconstrained case not all systems modified the baseline LM. Teams that did so are marked with an asterisk.

1-Channel, Unconstrained LM

Systems that are not using the baseline RNLMM are marked with an asterisk.

Rank	Team	Paper	Dev Sim	Dev Real	Eval Sim	Eval Real
	* Du et al.	[paper]	6.61	4.55	11.81	9.15
	* Menne et al.	[paper]	7.40	5.14	12.36	9.29
	* Heymann et al.	[paper]	6.69	5.20	11.14	9.34
	* Tachioka et al.	[paper]	7.15	5.15	12.15	11.13
	* Tran et al.	[paper]	7.33	6.08	10.88	11.19
	Fujita et al.	[paper]	7.36	5.89	9.23	11.42
	Alam et al.	[paper]	9.27	6.79	13.72	12.70
	Qian and Tan	[paper]	7.93	6.29	12.95	13.91
	del-Agua et al.	[paper]	11.13	9.95	15.72	16.11
	* Tanaka et al.	[paper]	10.15	8.06	15.67	16.58
	Matassoni et al.	[paper]	9.51	9.02	16.09	16.87
	Bayestehtashk and Shafran	[paper]	12.05	9.83	19.15	18.63
	Xiao et al.	[paper]	14.25	11.43	21.37	20.94
	Baseline	-	12.98	11.57	20.84	23.70

[TOP]

2-Channel, Unconstrained LM

Rank	Team	Paper	Dev Sim	Dev Real	Eval Sim	Eval Real
	* Du et al.	[paper]	3.46	2.33	5.74	3.91
	* Menne et al.	[paper]	4.04	3.02	5.27	5.32
	* Heymann et al.	[paper]	4.05	3.54	5.16	5.96
	* Tachioka et al.	[paper]	4.63	3.50	8.03	7.28
	* Tran et al.	[paper]	5.10	4.32	7.57	8.33
	Fujita et al.	[paper]	5.88	4.22	7.32	8.61
	Qian and Tan	[paper]	5.73	4.79	8.74	9.14
	Wang et al.	[paper]	7.18	5.58	8.77	9.64
	Alam et al.	[paper]	6.69	5.13	10.34	9.97
	Xiao et al.	[paper]	7.06	5.86	10.72	10.46
	Zhang et al.	[paper]	6.26	5.48	7.80	11.02
	del-Agua et al.	[paper]	8.93	7.96	12.06	12.82
	* Schrank et al.	[paper]	8.29	6.46	14.34	13.71
	Bayestehtashk and Shafran	[paper]	8.78	7.33	13.91	13.81
	Baseline	-	9.50	8.23	15.33	16.58

[TOP]

6-Channel, Unconstrained LM

Rank	Team	Paper	Dev Sim	Dev Real	Eval Sim	Eval Real
	* Du et al.	[paper]	1.78	1.69	2.12	2.24
	* Menne et al.	[paper]	2.34	2.19	2.68	2.91
	* Erdogan et al.	[paper]	1.95	2.11	1.97	2.98
	* Heymann et al.	[paper]	2.34	2.73	2.76	3.48
	* Tachioka et al.	[paper]	2.32	2.30	4.18	4.31
	Fujita et al.	[paper]	3.33	2.68	4.15	4.68
	* Tran et al.	[paper]	3.39	3.18	4.97	5.00
	Wang et al.	[paper]	3.71	3.65	4.38	5.69
	Zhang et al.	[paper]	3.66	3.62	4.83	6.00
	Qian and Tan	[paper]	4.37	3.73	6.29	6.41
	Xiao et al.	[paper]	4.12	4.28	4.99	6.43
	* Xiang et al.	[paper]	3.78	4.58	4.95	6.55
	Alam et al.	[paper]	5.07	4.00	7.53	7.23
	* Schrank et al.	[paper]	4.73	4.22	7.02	8.99
	Bayestehtashk and Shafran	[paper]	6.34	5.38	10.10	10.09
	Baseline	-	6.77	5.76	10.90	11.51

[TOP]

Results obtained when using the baseline RNNLM

Teams were asked to report one set of results using the baseline RNNLM.

1-Channel, Baseline LM

Rank	Team	Paper	Dev Sim	Dev Real	Eval Sim	Eval Real
	Du et al.	[paper]	8.24	6.10	13.62	11.15
	Heymann et al.	[paper]	7.18	5.50	11.68	9.88
	Tran et al.	[paper]	-	-	-	12.92
	Fujita et al.	[paper]	7.36	5.89	9.23	11.42
	Alam et al.	[paper]	9.27	6.79	13.72	12.70
	Qian and Tan	[paper]	7.93	6.29	12.95	13.91
	del-Agua et al.	[paper]	11.13	9.95	15.72	16.11
	Tanaka et al.	[paper]	10.86	9.09	16.49	17.40
	Matassoni et al.	[paper]	9.51	9.02	16.09	16.87
	Bayestehtashk and Shafran	[paper]	12.05	9.83	19.15	18.63
	Xiao et al.	[paper]	14.25	11.43	21.37	20.94
	Baseline	-	12.98	11.57	20.84	23.70

[TOP]

2-Channel, Baseline LM

Rank	Team	Paper	Dev Sim	Dev Real	Eval Sim	Eval Real
	Du et al.	[paper]	4.89	3.56	7.30	5.41
	Heymann et al.	[paper]	4.45	3.80	5.38	6.44
	Tran et al.	[paper]	-	-	-	9.77
	Fujita et al.	[paper]	5.88	4.22	7.32	8.61
	Qian and Tan	[paper]	5.73	4.79	8.74	9.14
	Wang et al.	[paper]	7.18	5.58	8.77	9.64
	Alam et al.	[paper]	6.69	5.13	10.34	9.97
	Xiao et al.	[paper]	7.06	5.86	10.72	10.46
	Zhang et al.	[paper]	6.26	5.48	7.80	11.02
	del-Agua et al.	[paper]	8.93	7.96	12.06	12.82
	Schrank et al.	[paper]	8.54	6.66	14.46	14.02
	Bayestehtashk and Shafran	[paper]	8.78	7.33	13.91	13.81
	Baseline	-	9.50	8.23	15.33	16.58

[TOP]

6-Channel, Baseline LM

Rank	Team	Paper	Dev Sim	Dev Real	Eval Sim	Eval Real
	Du et al.	[paper]	2.61	2.55	3.06	3.24
	Menne et al.	[paper]	3.79	4.16	4.04	5.17
	Erdogan et al.	[paper]	3.28	3.34	3.35	4.44
	Heymann et al.	[paper]	2.75	2.84	3.11	3.85
	Fujita et al.	[paper]	3.33	2.68	4.15	4.68
	Tran et al.	[paper]	-	-	-	6.18
	Wang et al.	[paper]	3.71	3.65	4.38	5.69
	Zhang et al.	[paper]	3.66	3.62	4.83	6.00
	Qian and Tan	[paper]	4.37	3.73	6.29	6.41
	Xiao et al.	[paper]	4.12	4.28	4.99	6.43
	Xiang et al.	[paper]	4.22	4.64	5.51	7.15
	Alam et al.	[paper]	5.07	4.00	7.53	7.23
	Schrank et al.	[paper]	4.14	4.83	6.16	9.17
	Bayestehtashk and Shafran	[paper]	6.34	5.38	10.10	10.09
	Baseline	-	6.77	5.76	10.90	11.51

[TOP]