scripts/README.md

Scripts

Tooling used to produce this checkpoint and the numbers in the model card. None of it is needed to use the model.

Install the dependencies first:

uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python torch transformers bitsandbytes accelerate sentencepiece

quantize.py

Rebuilds the 4-bit NF4 checkpoint from the original fp32 weights and writes a self-contained repository (weights, tokenizer files, .gitattributes) that can be uploaded directly. Optional argument: output directory, defaulting to ../models/gradient-ai-text-detector-4bit.

python scripts/quantize.py

It keeps the classifier head in fp32, because bitsandbytes' packed CPU kernel requires each quantized layer's output dimension to divide evenly by its block size and the head is [1, 1024]. It then asserts that no quantized layer would break that kernel, and prints a reload sanity value.

bench_quant.py

Compares fp32, bf16, and NF4 4-bit on CPU and Apple Silicon MPS, measuring resident memory, batch latency, and the maximum probability change against the fp32 reference.

python scripts/bench_quant.py

eval_quant.py

Measures how much quantization moves individual scores: max and mean absolute change in P(AI) against fp32, and how many verdicts flip at the 0.5 threshold.

python scripts/eval_quant.py