README.md

metadata

tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
widget:
  - text: >-
      Everyone could play a part, he said, adding that if industries and private
      individuals could reduce gas consumption by 10%, it would help avoid an
      emergency.
  - text: Can we really believe that the nuclear weapon will never be used again?
  - text: >-
      In Washington, President George Bush said that Mr Gorbachev's decision
      showed statesmanship and strengthened efforts to build 'enduring
      relationships based on co-operation.'
  - text: >-
      A similar paralysis of adequate American response would follow a Soviet
      threat to Europe.
  - text: You know, I’ve always found about the British, they don’t need much help.
metrics:
  - f1_macro
  - f1_binary
pipeline_tag: text-classification
library_name: setfit
inference: true
base_model: codefuse-ai/F2LLM-v2-80M
model-index:
  - name: SetFit with codefuse-ai/F2LLM-v2-80M
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: f1_macro
            value: 0.7436245252306022
            name: F1_Macro
          - type: f1_binary
            value: 0.631578947368421
            name: F1_Binary

SetFit with codefuse-ai/F2LLM-v2-80M

This is a SetFit model that can be used for Text Classification. This SetFit model uses codefuse-ai/F2LLM-v2-80M as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: codefuse-ai/F2LLM-v2-80M
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 40960 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
positive	'He and other thinkers anguishing over the inadequacy of global mechanisms advocate a new set of principles which would allow for intervention in a nation in crisis.' 'Plus, there are escalating concerns over cyberthreats, territorial disputes in the South China Sea and the spread of Ebola virus.' 'Now, says Urban, Russia, China and India have such strong conventional forces, and America has cut its forces so much, that in the event of a conflict "the US would be left with the choice of nuclear escalation or backing down".'
negative	'Positioning nuclear missiles in the exclave would give the Kremlin the ability to strike most central European capitals.' 'Discard the Trident missile programme.' "We're really going to choose between what I consider to be an old way of governing ourselves, of high levels of spending, high taxes, an ever more intrusive bureaucracy, or a new course, a new era if you will, and Governor Bush and I want to offer that new course of action."

Evaluation

Metrics

Label	F1_Macro	F1_Binary
all	0.7436	0.6316

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("fefofico/crisis_trained_allnat")
# Run inference
preds = model("Can we really believe that the nuclear weapon will never be used again?")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	24.5399	67

Label	Training Sample Count
negative	361
positive	178

Training Hyperparameters

• batch_size: (256, 256)
• num_epochs: (5, 5)
• max_steps: -1
• sampling_strategy: oversampling
• num_iterations: 20
• body_learning_rate: (1e-06, 1e-06)
• head_learning_rate: 0.003
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0233	1	0.4268	-
0.4651	20	0.4109	-
0.0118	1	0.3813	-
0.2353	20	0.3829	-
0.4706	40	0.3552	-
0.7059	60	0.3192	-
0.9412	80	0.2901	-
1.0	85	-	0.2554
1.1765	100	0.2728	-
1.4118	120	0.2668	-
1.6471	140	0.2596	-
1.8824	160	0.2556	-
2.0	170	-	0.2423
2.1176	180	0.2489	-
2.3529	200	0.2442	-
2.5882	220	0.239	-
2.8235	240	0.2354	-
3.0	255	-	0.2356
3.0588	260	0.2281	-
3.2941	280	0.2244	-
3.5294	300	0.2227	-
3.7647	320	0.2175	-
4.0	340	0.2114	0.2313
4.2353	360	0.2103	-
4.4706	380	0.2055	-
4.7059	400	0.2048	-
4.9412	420	0.2029	-
5.0	425	-	0.2297

Framework Versions

• Python: 3.12.13
• SetFit: 1.1.3
• Sentence Transformers: 3.4.1
• Transformers: 4.57.6
• PyTorch: 2.11.0+cu128
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}