LFM2.5-2.6B TikZ LoRA
BF16 LoRA supervised fine-tuning of LiquidAI/LFM2.5-2.6B for
instruction-to-TikZ generation.
This flat repository contains the pinned base checkpoint and the separate, unmerged LoRA adapter at repository root.
Training
- Base model:
LiquidAI/LFM2.5-2.6B - Base revision:
654f9463ce32b05d0429d76fe1f580b27d4c1ac0 - Method: BF16 LoRA, rank 64, alpha 64
- Hardware: one NVIDIA RTX PRO 6000 Blackwell Server Edition
- Maximum sequence length: 8,192
- Effective batch size: 16
- Peak learning rate: 1e-4
- Scheduler: cosine
- Epochs: 1.0
- Optimizer steps: 5,959
Dataset
- Training examples: 95329
- Quarantined examples: 1171
- Prompt tokens: 19829723
- Supervised assistant tokens: 55034690
- Unpadded tokens: 74864413
- Padded input tokens processed: 106235236
Only assistant/TikZ tokens contributed to loss. Prompt tokens were masked.
Results
- Gate passed: True
- Training completed: True
- Full epoch completed: True
- Artifact complete: True
- Validation loss: 0.4656994640827179
- Trainer-reported train loss for the final resumed allocation: 0.16397293387373174
- Total FLOPs: 1.6034710974370038e+18
Validation loss is token-level cross-entropy against one reference TikZ program. Compilation and rendered-image quality require separate evaluation.
Loading
Install Transformers and PEFT, load LiquidAI/LFM2.5-2.6B at revision
654f9463ce32b05d0429d76fe1f580b27d4c1ac0, and then attach the adapter from
Praha-Labs/LFM2.5-2.6B-TikZ-LoRA using PeftModel.from_pretrained.
The package also contains the base weights, tokenizer, training evidence, dataset identity reports, environment lock, Slurm records and checksums.
- Downloads last month
- 10