NbAiLab
/

nb-sbert-v2-base

@@ -8,6 +8,10 @@ tags:
 - dataset_size:527098
 - loss:MultipleNegativesRankingLoss
 base_model: NbAiLab/nb-bert-base
 widget:
 - source_sentence: The man talked to a girl over the internet camera.
   sentences:
@@ -64,7 +68,7 @@ model-index:
 # SentenceTransformer based on NbAiLab/nb-bert-base
-This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [NbAiLab/nb-bert-base](https://huggingface.co/NbAiLab/nb-bert-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
 ## Model Details
@@ -74,9 +78,12 @@ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [N
 - **Maximum Sequence Length:** 512 tokens
 - **Output Dimensionality:** 768 dimensions
 - **Similarity Function:** Cosine Similarity
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
 ### Model Sources
@@ -108,7 +115,7 @@ Then you can load this model and run inference.
 from sentence_transformers import SentenceTransformer
 # Download from the 🤗 Hub
-model = SentenceTransformer("sentence_transformers_model_id")
 # Run inference
 sentences = [
     'While Queen may refer to both Queen regent (sovereign) or Queen consort, the King has always been the sovereign.',
@@ -157,7 +164,7 @@ You can finetune this model on your own dataset.
 #### Semantic Similarity
-* Dataset: `sts-dev`
 * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
 | Metric              | Value      |
@@ -165,6 +172,17 @@ You can finetune this model on your own dataset.
 | pearson_cosine      | 0.8478     |
 | **spearman_cosine** | **0.8495** |
 <!--
 ## Bias, Risks and Limitations
@@ -181,7 +199,7 @@ You can finetune this model on your own dataset.
 ### Training Dataset
-#### Unnamed Dataset
 * Size: 527,098 training samples
 * Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
@@ -204,11 +222,12 @@ You can finetune this model on your own dataset.
       "gather_across_devices": false
   }
   ```
-### Evaluation Dataset
-#### Unnamed Dataset
 * Size: 1,500 evaluation samples
 * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
 * Approximate statistics based on the first 1000 samples:
@@ -230,7 +249,7 @@ You can finetune this model on your own dataset.
       "gather_across_devices": false
   }
   ```
 ### Training Hyperparameters
 #### Non-Default Hyperparameters
@@ -347,58 +366,58 @@ You can finetune this model on your own dataset.
 </details>
 ### Training Logs
-| Epoch  | Step | Training Loss | Validation Loss | sts-dev_spearman_cosine |
-|:------:|:----:|:-------------:|:---------------:|:-----------------------:|
-| 0.0243 | 100  | 1.8923        | -               | -                       |
-| 0.0486 | 200  | 0.8525        | -               | -                       |
-| 0.0729 | 300  | 0.6760        | -               | -                       |
-| 0.0971 | 400  | 0.5891        | -               | -                       |
-| 0.1000 | 412  | -             | 1.5082          | 0.8397                  |
-| 0.1214 | 500  | 0.5567        | -               | -                       |
-| 0.1457 | 600  | 0.5245        | -               | -                       |
-| 0.1700 | 700  | 0.5024        | -               | -                       |
-| 0.1943 | 800  | 0.4654        | -               | -                       |
-| 0.2001 | 824  | -             | 1.5451          | 0.8390                  |
-| 0.2186 | 900  | 0.4796        | -               | -                       |
-| 0.2428 | 1000 | 0.4528        | -               | -                       |
-| 0.2671 | 1100 | 0.4577        | -               | -                       |
-| 0.2914 | 1200 | 0.4443        | -               | -                       |
-| 0.3001 | 1236 | -             | 1.5638          | 0.8455                  |
-| 0.3157 | 1300 | 0.4201        | -               | -                       |
-| 0.3400 | 1400 | 0.4010        | -               | -                       |
-| 0.3643 | 1500 | 0.4063        | -               | -                       |
-| 0.3885 | 1600 | 0.3955        | -               | -                       |
-| 0.4002 | 1648 | -             | 1.5716          | 0.8446                  |
-| 0.4128 | 1700 | 0.3798        | -               | -                       |
-| 0.4371 | 1800 | 0.3772        | -               | -                       |
-| 0.4614 | 1900 | 0.3933        | -               | -                       |
-| 0.4857 | 2000 | 0.3793        | -               | -                       |
-| 0.5002 | 2060 | -             | 1.5650          | 0.8499                  |
-| 0.5100 | 2100 | 0.3862        | -               | -                       |
-| 0.5342 | 2200 | 0.3730        | -               | -                       |
-| 0.5585 | 2300 | 0.3463        | -               | -                       |
-| 0.5828 | 2400 | 0.3556        | -               | -                       |
-| 0.6003 | 2472 | -             | 1.5952          | 0.8503                  |
-| 0.6071 | 2500 | 0.3614        | -               | -                       |
-| 0.6314 | 2600 | 0.3479        | -               | -                       |
-| 0.6557 | 2700 | 0.3508        | -               | -                       |
-| 0.6799 | 2800 | 0.3463        | -               | -                       |
-| 0.7003 | 2884 | -             | 1.5957          | 0.8471                  |
-| 0.7042 | 2900 | 0.3453        | -               | -                       |
-| 0.7285 | 3000 | 0.3327        | -               | -                       |
-| 0.7528 | 3100 | 0.3269        | -               | -                       |
-| 0.7771 | 3200 | 0.3333        | -               | -                       |
-| 0.8004 | 3296 | -             | 1.5891          | 0.8493                  |
-| 0.8014 | 3300 | 0.3370        | -               | -                       |
-| 0.8256 | 3400 | 0.3254        | -               | -                       |
-| 0.8499 | 3500 | 0.3348        | -               | -                       |
-| 0.8742 | 3600 | 0.3213        | -               | -                       |
-| 0.8985 | 3700 | 0.3376        | -               | -                       |
-| 0.9004 | 3708 | -             | 1.5881          | 0.8495                  |
-| 0.9228 | 3800 | 0.3362        | -               | -                       |
-| 0.9471 | 3900 | 0.3246        | -               | -                       |
-| 0.9713 | 4000 | 0.3215        | -               | -                       |
-| 0.9956 | 4100 | 0.3143        | -               | -                       |
 ### Framework Versions

 - dataset_size:527098
 - loss:MultipleNegativesRankingLoss
 base_model: NbAiLab/nb-bert-base
+datasets: NbAiLab/mnli-norwegian
+license: apache-2.0
+language:
+- 'no'
 widget:
 - source_sentence: The man talked to a girl over the internet camera.
   sentences:
 # SentenceTransformer based on NbAiLab/nb-bert-base
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [NbAiLab/nb-bert-base](https://huggingface.co/NbAiLab/nb-bert-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. The easiest way is to simply measure the cosine distance between two sentences. Sentences that are close to each other in meaning, will have a small cosine distance and a similarity close to 1. The model is trained in such a way that similar sentences in different languages should also be close to each other. Ideally, an English-Norwegian sentence pair should have high similarity.
 ## Model Details
 - **Maximum Sequence Length:** 512 tokens
 - **Output Dimensionality:** 768 dimensions
 - **Similarity Function:** Cosine Similarity
+- **Training Dataset:** Subset of [NbAiLab/mnli-norwegian](https://huggingface.co/datasets/NbAiLab/mnli-norwegian)
+- **Language:** Norwegian and English
+- **License:** Apache 2.0
+### EU AI Act
+This release is a non-generative encoder model whose outputs are vectors/scores rather than language or media. Its intended functionality is limited to representation, retrieval, ranking, or classification support. On that basis, the release is preliminarily assessed as not falling within the provider obligations for GPAI models under the EU AI Act definitions, subject to legal confirmation if capability scope or marketed generality changes.
 ### Model Sources
 from sentence_transformers import SentenceTransformer
 # Download from the 🤗 Hub
+model = SentenceTransformer("NbAiLab/nb-sbert-v2-base")
 # Run inference
 sentences = [
     'While Queen may refer to both Queen regent (sovereign) or Queen consort, the King has always been the sovereign.',
 #### Semantic Similarity
+* Dataset: [STS Benchmark](https://public.ukp.informatik.tu-darmstadt.de/reimers/sentence-transformers/datasets/stsbenchmark.tsv.gz)
 * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
 | Metric              | Value      |
 | pearson_cosine      | 0.8478     |
 | **spearman_cosine** | **0.8495** |
+#### [MTEB (Scandinavian)](https://embeddings-benchmark.github.io/mteb/)
+| Metric              | Value      |
+|:--------------------|:-----------|
+| Mean (Task)         |            |
+| Mean (TaskType)     |            |
+| Bitext Mining       |            |
+| Classification      |            |
+| Clustering          |            |
+| Retrieval           |            |
 <!--
 ## Bias, Risks and Limitations
 ### Training Dataset
+#### Subset of [NbAiLab/mnli-norwegian](https://huggingface.co/datasets/NbAiLab/mnli-norwegian)
 * Size: 527,098 training samples
 * Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
       "gather_across_devices": false
   }
   ```
+<!-- Removing Evaluation results because they don't actually measure much-->
+<!-- ### Evaluation Dataset
+#### [STS Benchmark](https://public.ukp.informatik.tu-darmstadt.de/reimers/sentence-transformers/datasets/stsbenchmark.tsv.gz)
+* Partition: dev
+* Language: English
 * Size: 1,500 evaluation samples
 * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
 * Approximate statistics based on the first 1000 samples:
       "gather_across_devices": false
   }
   ```
+-->
 ### Training Hyperparameters
 #### Non-Default Hyperparameters
 </details>
 ### Training Logs
+| Epoch  | Step | Training Loss | sts-dev_spearman_cosine |
+|:------:|:----:|:-------------:|:-----------------------:|
+| 0.0243 | 100  | 1.8923        | -                       |
+| 0.0486 | 200  | 0.8525        | -                       |
+| 0.0729 | 300  | 0.6760        | -                       |
+| 0.0971 | 400  | 0.5891        | -                       |
+| 0.1000 | 412  | -             | 0.8397                  |
+| 0.1214 | 500  | 0.5567        | -                       |
+| 0.1457 | 600  | 0.5245        | -                       |
+| 0.1700 | 700  | 0.5024        | -                       |
+| 0.1943 | 800  | 0.4654        | -                       |
+| 0.2001 | 824  | -             | 0.8390                  |
+| 0.2186 | 900  | 0.4796        | -                       |
+| 0.2428 | 1000 | 0.4528        | -                       |
+| 0.2671 | 1100 | 0.4577        | -                       |
+| 0.2914 | 1200 | 0.4443        | -                       |
+| 0.3001 | 1236 | -             | 0.8455                  |
+| 0.3157 | 1300 | 0.4201        | -                       |
+| 0.3400 | 1400 | 0.4010        | -                       |
+| 0.3643 | 1500 | 0.4063        | -                       |
+| 0.3885 | 1600 | 0.3955        | -                       |
+| 0.4002 | 1648 | -             | 0.8446                  |
+| 0.4128 | 1700 | 0.3798        | -                       |
+| 0.4371 | 1800 | 0.3772        | -                       |
+| 0.4614 | 1900 | 0.3933        | -                       |
+| 0.4857 | 2000 | 0.3793        | -                       |
+| 0.5002 | 2060 | -             | 0.8499                  |
+| 0.5100 | 2100 | 0.3862        | -                       |
+| 0.5342 | 2200 | 0.3730        | -                       |
+| 0.5585 | 2300 | 0.3463        | -                       |
+| 0.5828 | 2400 | 0.3556        | -                       |
+| 0.6003 | 2472 | -             | 0.8503                  |
+| 0.6071 | 2500 | 0.3614        | -                       |
+| 0.6314 | 2600 | 0.3479        | -                       |
+| 0.6557 | 2700 | 0.3508        | -                       |
+| 0.6799 | 2800 | 0.3463        | -                       |
+| 0.7003 | 2884 | -             | 0.8471                  |
+| 0.7042 | 2900 | 0.3453        | -                       |
+| 0.7285 | 3000 | 0.3327        | -                       |
+| 0.7528 | 3100 | 0.3269        | -                       |
+| 0.7771 | 3200 | 0.3333        | -                       |
+| 0.8004 | 3296 | -             | 0.8493                  |
+| 0.8014 | 3300 | 0.3370        | -                       |
+| 0.8256 | 3400 | 0.3254        | -                       |
+| 0.8499 | 3500 | 0.3348        | -                       |
+| 0.8742 | 3600 | 0.3213        | -                       |
+| 0.8985 | 3700 | 0.3376        | -                       |
+| 0.9004 | 3708 | -             | 0.8495                  |
+| 0.9228 | 3800 | 0.3362        | -                       |
+| 0.9471 | 3900 | 0.3246        | -                       |
+| 0.9713 | 4000 | 0.3215        | -                       |
+| 0.9956 | 4100 | 0.3143        | -                       |
 ### Framework Versions