train_codealpacapy_789_1767650456

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the codealpacapy dataset. It achieves the following results on the evaluation set:

• Loss: 0.9208
• Num Input Tokens Seen: 24964664

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5727	1.0	1908	0.5040	1246616
0.4818	2.0	3816	0.4911	2496088
0.4558	3.0	5724	0.4849	3746728
0.587	4.0	7632	0.4801	4999712
0.4584	5.0	9540	0.4774	6245072
0.5131	6.0	11448	0.4759	7491776
0.4679	7.0	13356	0.4737	8735728
0.4536	8.0	15264	0.4751	9981168
0.5711	9.0	17172	0.4765	11227560
0.4537	10.0	19080	0.4763	12474576
0.3392	11.0	20988	0.4757	13719896
0.5766	12.0	22896	0.4785	14970024
0.3798	13.0	24804	0.4776	16222408
0.5339	14.0	26712	0.4810	17474664
0.4884	15.0	28620	0.4835	18722440
0.3629	16.0	30528	0.4844	19970248
0.4764	17.0	32436	0.4862	21217120
0.3909	18.0	34344	0.4877	22462808
0.3584	19.0	36252	0.4880	23715264
0.6004	20.0	38160	0.4880	24964664

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1