Upload README.md
Browse files
README.md
CHANGED
|
@@ -146,7 +146,6 @@ All training parameters are listed in table below.
|
|
| 146 |
The main research question was: "How does adding additional, related languages impact the quality of the model?" - we explored it in the Slavic language family.
|
| 147 |
___BiDi___ models are our baseline before expanding the data-regime by using higher-level multilinguality.
|
| 148 |
|
| 149 |
-
|
| 150 |
Datasets were downloaded via [MT-Data](https://pypi.org/project/mtdata/0.2.10/) library.
|
| 151 |
The number of total examples post filtering and deduplication varies, depending on languages supported, see the table below.
|
| 152 |
|
|
@@ -228,7 +227,7 @@ Metric used: Unbabel/wmt22-comet-da.
|
|
| 228 |
| **M2M-100** | 87.0 | 89.0 | 92.1 | 89.7 | 88.6 | 86.4 | 88.4 | 87.3 | 89.6 | 84.6 | 89.4 | 88.4 | 92.7 | 86.8 | 89.1 | 89.6 | 90.3 | 86.4 | 88.7 | 90.1 |
|
| 229 |
| **NLLB-200** | 88.1 | 88.9 | 91.2 | 88.6 | 90.4 | __88.5__ | 90.1 | 88.8 | 89.4 | __85.8__ | 88.9 | 87.7 | 91.8 | 88.2 | 88.9 | 88.8 | 90.0 | __87.5__ | 88.6 | 89.4 |
|
| 230 |
| **Seamless-M4T** | 87.5 | 80.9 | 90.8 | 82.0 | __90.7__ | __88.5__ | __90.6__ | __89.6__ | 79.6 | 85.4 | 80.0 | 76.4 | 91.5 | 87.2 | 81.2 | 82.9 | 80.9 | 87.3 | 76.7 | 81.0 |
|
| 231 |
-
| **OPUS-MT Sla-Sla** | __88.2__ | 82.8 | - | 83.4 | 89.1 | 85.6 |
|
| 232 |
| **OPUS-MT SK-EN** | - | - | - | - | - | - | 89.5 | - | - | - | - | - | - | __88.4__ | - | - | - | - | - | - |
|
| 233 |
| _Our contributions:_ | | | | | | | | | | | | | | | | | | | | |
|
| 234 |
| **BiDi Models**<span style="color:green;">*</span> | 87.5 | 89.4 | 92.4 | 89.8 | 87.8 | 86.2 | 87.2 | 86.6 | 90.0 | 85.0 | 89.1 | 88.4 | 92.9 | 87.3 | 88.8 | 89.4 | 90.0 | 86.9 | 88.1 | 89.1 |
|
|
@@ -238,7 +237,7 @@ Metric used: Unbabel/wmt22-comet-da.
|
|
| 238 |
| **MultiSlav-4slav** | - | 89.7 | __92.5__ | 90.0 | - | - | - | - | 90.2 | - | 89.6 | 88.7 | 92.9 | - | 89.4 | 90.1 | __90.6__ | - | 88.9 | __90.2__ |
|
| 239 |
| **MultiSlav-5lang** | 87.8 | __89.8__ | __92.5__ | __90.1__ | 88.9 | 86.9 | 88.0 | 87.3 | __90.4__ | 85.4 | 89.8 | __88.9__ | 92.9 | 87.8 | __89.6__ | __90.2__ | __90.6__ | 87.0 | __89.2__ | __90.2__ |
|
| 240 |
|
| 241 |
-
<span style="color:red;">◊</span> system of 2 models *Many2XXX* and *XXX2Many*
|
| 242 |
|
| 243 |
<span style="color:green;">*</span> results combined for all bi-directional models; each values for applicable model
|
| 244 |
|
|
|
|
| 146 |
The main research question was: "How does adding additional, related languages impact the quality of the model?" - we explored it in the Slavic language family.
|
| 147 |
___BiDi___ models are our baseline before expanding the data-regime by using higher-level multilinguality.
|
| 148 |
|
|
|
|
| 149 |
Datasets were downloaded via [MT-Data](https://pypi.org/project/mtdata/0.2.10/) library.
|
| 150 |
The number of total examples post filtering and deduplication varies, depending on languages supported, see the table below.
|
| 151 |
|
|
|
|
| 227 |
| **M2M-100** | 87.0 | 89.0 | 92.1 | 89.7 | 88.6 | 86.4 | 88.4 | 87.3 | 89.6 | 84.6 | 89.4 | 88.4 | 92.7 | 86.8 | 89.1 | 89.6 | 90.3 | 86.4 | 88.7 | 90.1 |
|
| 228 |
| **NLLB-200** | 88.1 | 88.9 | 91.2 | 88.6 | 90.4 | __88.5__ | 90.1 | 88.8 | 89.4 | __85.8__ | 88.9 | 87.7 | 91.8 | 88.2 | 88.9 | 88.8 | 90.0 | __87.5__ | 88.6 | 89.4 |
|
| 229 |
| **Seamless-M4T** | 87.5 | 80.9 | 90.8 | 82.0 | __90.7__ | __88.5__ | __90.6__ | __89.6__ | 79.6 | 85.4 | 80.0 | 76.4 | 91.5 | 87.2 | 81.2 | 82.9 | 80.9 | 87.3 | 76.7 | 81.0 |
|
| 230 |
+
| **OPUS-MT Sla-Sla** | __88.2__ | 82.8 | - | 83.4 | 89.1 | 85.6 | - | 84.5 | 82.9 | 82.2 | - | 81.2 | - | - | - | - | 83.5 | 84.1 | 80.8 | - |
|
| 231 |
| **OPUS-MT SK-EN** | - | - | - | - | - | - | 89.5 | - | - | - | - | - | - | __88.4__ | - | - | - | - | - | - |
|
| 232 |
| _Our contributions:_ | | | | | | | | | | | | | | | | | | | | |
|
| 233 |
| **BiDi Models**<span style="color:green;">*</span> | 87.5 | 89.4 | 92.4 | 89.8 | 87.8 | 86.2 | 87.2 | 86.6 | 90.0 | 85.0 | 89.1 | 88.4 | 92.9 | 87.3 | 88.8 | 89.4 | 90.0 | 86.9 | 88.1 | 89.1 |
|
|
|
|
| 237 |
| **MultiSlav-4slav** | - | 89.7 | __92.5__ | 90.0 | - | - | - | - | 90.2 | - | 89.6 | 88.7 | 92.9 | - | 89.4 | 90.1 | __90.6__ | - | 88.9 | __90.2__ |
|
| 238 |
| **MultiSlav-5lang** | 87.8 | __89.8__ | __92.5__ | __90.1__ | 88.9 | 86.9 | 88.0 | 87.3 | __90.4__ | 85.4 | 89.8 | __88.9__ | 92.9 | 87.8 | __89.6__ | __90.2__ | __90.6__ | 87.0 | __89.2__ | __90.2__ |
|
| 239 |
|
| 240 |
+
<span style="color:red;">◊</span> system of 2 models *Many2XXX* and *XXX2Many*, see [P5-ces2many](https://huggingface.co/allegro/p5-ces2many)
|
| 241 |
|
| 242 |
<span style="color:green;">*</span> results combined for all bi-directional models; each values for applicable model
|
| 243 |
|