Whitnet/boc-distilbert-financial-sentiment

BOC DistilBERT Sentiment

A DistilBERT model fine-tuned for three-class financial sentiment classification (negative / neutral / positive). It was built as part of the HKUST ISOM5240 group project "AI-Powered Financial Information Monitoring for Bank of China", where it serves as one of two models used to screen Bank of China disclosures and related financial news.

This is a research prototype. It is a screening aid for human analysts — not investment advice, and not evidence of any regulatory violation or realised financial event.

---

Model details

Field	Value
Base model	distilbert-base-uncased
Architecture	DistilBertForSequenceClassification
Parameters	~66M (6 layers, 768 hidden dim, 12 heads)
Max sequence length	512 (tokenizer); 256 used at inference in the app
Number of labels	3
Vocabulary size	30,522
Fine-tuned weight dtype	float32
Model size on disk	~257 MB (model.safetensors ≈ 256 MB)
Tokenizer	BertTokenizer (uncased)
Fine-tuning framework	Hugging Face Transformers 5.x, PyTorch 2.x (CPU)

---

Label mapping

The canonical label order is:

0 = negative     1 = neutral     2 = positive

This is reflected in config.json as:

"id2label": {"0": "negative", "1": "neutral", "2": "positive"},
"label2id": {"negative": 0, "neutral": 1, "positive": 2}

⚠️ Important for interoperability. ProsusAI/finbert, used as the comparison model in this project, orders its native outputs as {0: positive, 1: negative, 2: neutral}. The two models must be normalised by label name, not by raw index. Mapping by index silently swaps positive and negative predictions.

---

Training data

Item	Value
Source	Financial PhraseBank v1.0
Subset	Sentences_AllAgree.txt (100 % annotator agreement)
Original statements	2,264
After duplicate removal	2,259
Split	Stratified 70 / 15 / 15, random_state=42
Training samples	1,581
Validation samples	339
Test samples	339

Duplicate sentences were removed before splitting to prevent leakage.

---

Training procedure

Hyperparameter	Value
Learning rate	2e-5
Train batch size	16
Eval batch size	16
Epochs	3
Weight decay	0.01
Eval / save strategy	Per epoch
Best-model selection	accuracy (higher is better)
Logging	Every 20 steps
Seed	42
Device	CPU
Mixed precision	Not used

Validation history

Epoch	Eval loss	Eval accuracy	Eval macro F1	Eval weighted F1
1	0.3251	0.8348	0.6683	0.8127
2	0.1255	0.9617	0.9537	0.9621
3	0.0991	0.9705	0.9642	0.9706

---

Evaluation results

Held-out test set (Financial PhraseBank, n=339)

Metric	Value
Accuracy	0.9587
Macro F1	0.9458
Weighted F1	0.9590
Negative precision	0.9545
Negative recall	0.9130
Negative F1	0.9333
Misclassified	14 / 339

Confusion matrix (test set)

	Pred Negative	Pred Neutral	Pred Positive
Actual Negative	42	0	4
Actual Neutral	1	202	5
Actual Positive	1	3	81

Comparison with ProsusAI/finbert (same test set)

Model	Accuracy	Macro F1	Weighted F1	Misclassified
This model (fine-tuned DistilBERT)	0.9587	0.9458	0.9590	14
ProsusAI/finbert (pretrained)	0.9823	0.9775	0.9824	6

FinBERT scores higher on this benchmark. However, on the small Bank of China in-domain set (20 human-labelled records, Step 36) this model performed better overall (accuracy 0.70 vs 0.65; macro F1 0.7009 vs 0.6444). With only 20 records, per-class figures there are indicative only.

---

License and usage restrictions

This model is released under CC BY-NC-SA 3.0 — Attribution-NonCommercial-ShareAlike 3.0 Unported.

The license is inherited from the training data, not chosen by the authors:

Component	License
Base model distilbert-base-uncased	Apache-2.0
Training data: Financial PhraseBank v1.0	CC BY-NC-SA 3.0 (NonCommercial)
This fine-tuned model	CC BY-NC-SA 3.0 (inherits the NonCommercial and ShareAlike terms)

Because the training corpus is licensed NonCommercial, this model inherits that restriction. The base model's permissive Apache-2.0 license does not override it — the derived weights were produced from the NonCommercial data.

What this means

Permitted

• Academic research and teaching
• Personal, non-commercial experimentation
• Benchmarking and reproducibility studies

Not permitted without a separate license

• Commercial use of any kind
• Use in a commercial product or paid service
• Redistribution under a more permissive or proprietary license

Obligations

• Attribution — credit Financial PhraseBank (Malo et al., 2014) and distilbert-base-uncased.
• ShareAlike — derivatives must carry the same license.

Commercial licensing

Financial PhraseBank is distributed by its original authors. For commercial use of the data — and therefore of this model — contact the dataset authors: Pekka Malo and Ankur Sinha (Aalto University).

This is a good-faith summary of the upstream terms, not legal advice. Consult the original licenses before any commercial use.

---

Intended use

Suitable for:

• Exploratory sentiment screening of English financial news and bank disclosures.
• A research baseline for financial sentiment classification.
• One input into a human analyst's review workflow.

Not suitable for:

• Investment, trading, or credit decisions.
• Regulatory, compliance, or legal determinations.
• Any automated decision affecting a person or institution.
• Non-English text.
• Treating a confidence score as a calibrated probability of correctness.

---

Limitations

1. Small in-domain evaluation set. The Bank of China evaluation uses only 20 human-labelled records; per-class metrics are indicative only.
2. Possible event overlap in the BOC evaluation set — some records describe the same news event on the same date.
3. Possible pretraining overlap. Financial PhraseBank sentences may resemble text seen during distilbert-base-uncased pretraining.
4. Direction-blindness. The model can follow sentiment vocabulary rather than metric direction. For example, "the non-performing loan ratio decreased to 1.25 %" is favourable but has been predicted negative. The application displays a caveat for such phrasing; it does not override the model.
5. English only.
6. Confidence is not calibrated. It is the softmax score, not P(correct).
7. Not a risk judgement. A negative prediction is a sentiment signal, not a finding of wrongdoing or financial distress.

---

How to use

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "YOUR_USERNAME/boc-distilbert-sentiment"   # replace after publishing

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

text = "Bank of China reported a 5% increase in net profit."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)

with torch.no_grad():
    logits = model(**inputs).logits
probs = torch.softmax(logits, dim=-1)[0]

label = model.config.id2label[int(probs.argmax())]
print(f"{label}  (confidence {float(probs.max()):.4f})")

---

Citation

@misc{boc-distilbert-sentiment-2026,
  title  = {BOC DistilBERT Sentiment: Financial Sentiment Classification},
  author = {HKUST ISOM5240 Group Project},
  year   = {2026},
  note   = {Fine-tuned from distilbert-base-uncased on Financial PhraseBank v1.0}
}

Acknowledgements

• Base model: distilbert-base-uncased
• Dataset: Financial PhraseBank (Malo et al., 2014)
• Comparison model: ProsusAI/finbert