In Task 1, the models are expected to extract the monophonic voice signal from the 3D mixture that contains various background noises. The evaluation T1 metric for this task is a combination of the short-time objective intelligibility (STOI), which estimates the intelligibility of the output speech signal, and word error rate (WER), computed to assess the effects of the enhancement for speech recognition purposes. The T1 metric lies in the range [0,1], where the higher the value, the better. Further information can be found in the official documentation picture_as_pdf.
L3DAS22 Challenge results for Task 1 are depicted in the following interactive chart.
The table below shows the L3DAS22 Challenge rank for Task 1, including all the scores for the sake of comparison.
| Rank | Team Name | WER | STOI | T1 Metric |
|---|---|---|---|---|
| 1 | ESP-SE | 0.019 | 0.987 | 0.984 |
| 2 | BaiduSpeech | 0.025 | 0.975 | 0.975 |
| 3 | PCG-AIID | 0.032 | 0.972 | 0.970 |
| 4 | NPU | 0.047 | 0.955 | 0.954 |
| 5 | BBD | 0.049 | 0.945 | 0.948 |
| 6 | KLSACU | 0.053 | 0.943 | 0.945 |
| 7 | CMU Robust Speech | 0.066 | 0.934 | 0.934 |
| 8 | SEU Speech | 0.064 | 0.917 | 0.926 |
| 9 | SCUTer | 0.085 | 0.920 | 0.918 |
| 10 | HLT@NanKai | 0.091 | 0.923 | 0.916 |
| 11 | Team Spatial | 0.101 | 0.888 | 0.894 |
| 12 | Wang Wang Team | 0.123 | 0.910 | 0.894 |
| 13 | CCA-SE | 0.138 | 0.897 | 0.879 |
| 14 | GREAT Sound Lab | 0.092 | 0.848 | 0.878 |
| 15 | EPUSPL | 0.176 | 0.861 | 0.843 |
| - | Baseline | 0.212 | 0.878 | 0.833 |
| 16 | Soha Nossier | 0.420 | 0.581 | 0.580 |
| 17 | NIIT | 0.997 | 0.389 | 0.196 |
In task 2, the models are expected to predict a list of the active sound events and their respective location at regular intervals of 100 milliseconds. The evaluation T2 metric is a location-sensitive detection error computed on each time frame. It consists of measuring the Cartesian distance between the predicted and true events, and then computing the F score. The T2 metric lies in the range [0,1], where the higher the value, the better. Further information can be found in the official documentation picture_as_pdf.
L3DAS22 Challenge results for Task 2 are depicted in the following interactive chart.
The table below shows the L3DAS22 Challenge rank for Task 2, including all the scores, for the sake of comparison.
| Rank | Team Name | Precision | Recall | T2 Metric |
|---|---|---|---|---|
| 1 | Lab9 DSP411 | 0.706 | 0.691 | 0.699 |
| 2 | CQUPT AudioLab | 0.600 | 0.584 | 0.592 |
| 3 | FMSG | 0.577 | 0.574 | 0.575 |
| 4 | Team JLESS | 0.587 | 0.561 | 0.574 |
| 5 | SHRC of PKU | 0.485 | 0.488 | 0.487 |
| - | Baseline | 0.423 | 0.289 | 0.343 |
| 6 | DeepHearing | 0.334 | 0.285 | 0.307 |
| 7 | Anomaly | 0.056 | 0.054 | 0.055 |
Based on the challenge results the following awards and benefits are assigned.
The first 2 ranked teams for each task are awarded thanks to the support of Kuaishou Technology.
| Ranking Position | Task | Team | Prize |
|---|---|---|---|
| 1st | 1: 3D SE | ESP-SE | 600$ |
| 1st | 2: 3D SELD | Lab9 DSP411 | 600$ |
| 2nd | 1: 3D SE | BaiduSpeech | 250$ |
| 2nd | 2: 3D SELD | CQUPT AudioLab | 250$ |
The first 5 ranked teams are allowed to submit a paper to ICASSP 2022. Given the number of submissions to the two tasks, we accept papers from the first 3 ranked teams from Task 1 and from the first 2 ranked teams from Task 2, as listed in the following table. Papers will undergo a regular peer-review process. The format should be consistent with ICASSP regular paper. The deadline for the paper submission is January 31 (strict deadline).
| Ranking Position | Task | Team |
|---|---|---|
| 1st | 1: 3D SE | ESP-SE |
| 1st | 2: 3D SELD | Lab9 DSP411 |
| 2nd | 1: 3D SE | BaiduSpeech |
| 2nd | 2: 3D SELD | CQUPT AudioLab |
| 3rd | 1: 3D SE | PCG-AIID |
The L3DAS22 Challenges has been endorsed by the International Speech Communication Association (ISCA), which has offered to accept papers under the ISCA Archive and provide a DOI for each paper. Each Team is allowed to submit a paper. Papers will undergo a regular peer-review process. The deadline for the ISCA paper submission is February 16.
Each Team is allowed to upload an interactive demo of the proposed model on Replicate. Please ask us for a private link that allows you to create a new Replicate account and upload your model freely.
If you are unable to view the interactive charts due to your geographical location, click here to replace them with images.