CleverBoi
Collection
CleverBoi is a curated collection of data that emphasizes logic, inference, science, code, math and empathy, and its fine tuned language models.
•
16 items
•
Updated
•
2
CleverBoi
The CleverBoi series is based on models that have been fine tuned on a collection of data sets that emphasize logic, inference, math and coding, also known as the CleverBoi data set.
This model has been fine tuned for 1 epoch on the CleverBoi-Data-20k data set.
Uploaded model
- Developed by: theprint
- License: apache-2.0
- Finetuned from model : unsloth/Mistral-Nemo-Instruct-2407-bnb-4bit
This mistral model was trained 2x faster with Unsloth and Huggingface's TRL library.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 17.57 |
| IFEval (0-Shot) | 20.46 |
| BBH (3-Shot) | 31.65 |
| MATH Lvl 5 (4-Shot) | 8.61 |
| GPQA (0-shot) | 8.50 |
| MuSR (0-shot) | 11.43 |
| MMLU-PRO (5-shot) | 24.76 |
- Downloads last month
- 341
Hardware compatibility
Log In
to view the estimation
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Model tree for theprint/CleverBoi-Nemo-12B-v2
Base model
unsloth/Mistral-Nemo-Instruct-2407-bnb-4bitDataset used to train theprint/CleverBoi-Nemo-12B-v2
Collection including theprint/CleverBoi-Nemo-12B-v2
Evaluation results
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard20.460
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard31.650
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard8.610
- acc_norm on GPQA (0-shot)Open LLM Leaderboard8.500
- acc_norm on MuSR (0-shot)Open LLM Leaderboard11.430
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard24.760