firetower-qwen3.5-0.8b-gguf

GGUF builds of the Yoctotta-Technologies/firetower-qwen3.5-0.8b fine-tune, ready for llama.cpp / llama-server deployment.

A vision-language fire/no-fire decision filter for forest-fire watchtower cameras, fine-tuned on Similipal Tiger Reserve (Odisha, India) imagery. See the parent repo for full training and evaluation details.

Files

File	Size	Bits/weight	Use
firetower-qwen3.5-0.8b-q4_k_m.gguf	529 MB	5.51	Recommended for production. Best size/quality trade.
firetower-qwen3.5-0.8b-f16.gguf	1.51 GB	16.0	Lossless reference. Quantize on-device if you want a different format.

Quick start (llama.cpp)

docker run --rm --gpus all -p 8080:8080 \
  -v $PWD:/models \
  ghcr.io/ggml-org/llama.cpp:server-cuda \
  -m /models/firetower-qwen3.5-0.8b-q4_k_m.gguf \
  -ngl 999 --host 0.0.0.0 --port 8080 --jinja --reasoning off --ctx-size 4096

Or with -hf to pull this repo directly:

docker run --rm --gpus all -p 8080:8080 \
  ghcr.io/ggml-org/llama.cpp:server-cuda \
  -hf Yoctotta-Technologies/firetower-qwen3.5-0.8b-gguf:Q4_K_M \
  -ngl 999 --host 0.0.0.0 --port 8080 --jinja --reasoning off --ctx-size 4096

The model needs the multimodal projector (mmproj) which is included in the GGUF for Qwen3.5 (vision is fused into the main weights, not a separate file).

Inference (OpenAI-compatible API)

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vlm",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}},
        {"type": "text", "text": "You are analyzing a forest fire tower camera image. ..."}
      ]
    }],
    "response_format": {"type": "json_object"},
    "max_tokens": 300, "temperature": 0.1
  }'

Output schema

{
  "smoke": 0.7, "haze": 0.2, "fire": 0.8, "normal": 0.0,
  "ground_anchored": true, "base_visible": true,
  "reasoning": "..."
}

Eval

	Verdict acc	Fire recall	Fires missed
Base Qwen3.5-0.8B (Q4_K_M)	72.0 %	0 %	14 / 14
This model (Q4_K_M)	74.1 %	77.9 %	15 / 68

Eval was done on the FP16 model (the post-quantization numbers should be within ~1pp of these).

See the parent repo Yoctotta-Technologies/firetower-qwen3.5-0.8b for complete training and limitations details.

License

Apache 2.0, inheriting from the base model.