GPT-1900 Drafts
Collection
Experimental and intermediate GPT-1900 checkpoints. Working artifacts, not for general use. • 49 items • Updated
GPT-1900 D34 Physics CLM (Pre-1900 Only)
3.29B parameter GPT-1900 with continued pretraining on pre-1900-only physics texts (anachronism-filtered). Trained from the pre-1900-only base model (not the standard d34-22btok which includes some post-1900 data).
This is an intermediate checkpoint in the pre-1900-only pipeline: pre-1900-only base → physics CLM (pre-1900) → v3 SFT → discovery RL.
3 checkpoints saved at steps 4800, 9600, 14399 (5 epochs over pre-1900 clean physics data).
Model Details
- Architecture: Custom GPT with RoPE, QK-norm, ReLU², value embeddings (ResFormer)
- Parameters: 3.29B
- Layers: 34, Hidden dim: 2176, Heads: 17
- Context length: 2048 tokens
- Training data:
physics_clm_data_pre1900_clean - Base model:
d34_pre1900(pre-1900 only base)
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support