Instructions to use gasschina/Spark-X2.5-4B-build-cpp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use gasschina/Spark-X2.5-4B-build-cpp with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
# Run inference directly in the terminal:
llama cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
# Run inference directly in the terminal:
llama cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Use Docker

docker model run hf.co/gasschina/Spark-X2.5-4B-build-cpp:Q8_0

LM Studio
Jan
Ollama
How to use gasschina/Spark-X2.5-4B-build-cpp with Ollama:
```
ollama run hf.co/gasschina/Spark-X2.5-4B-build-cpp:Q8_0
```
Unsloth Desktop

How to use gasschina/Spark-X2.5-4B-build-cpp with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "gasschina/Spark-X2.5-4B-build-cpp:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use gasschina/Spark-X2.5-4B-build-cpp with Docker Model Runner:
```
docker model run hf.co/gasschina/Spark-X2.5-4B-build-cpp:Q8_0
```

Lemonade

How to use gasschina/Spark-X2.5-4B-build-cpp with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Run and chat with the model

lemonade run user.Spark-X2.5-4B-build-cpp-Q8_0

List all available models

lemonade list

Hermes Agent

How to use gasschina/Spark-X2.5-4B-build-cpp with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use gasschina/Spark-X2.5-4B-build-cpp with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "gasschina/Spark-X2.5-4B-build-cpp:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Spark-X2.5-4B — Ready-to-Run llama.cpp Build & GGUF Package (Colab T4 / CUDA)

Everything needed to run iFlytek Spark-X2.5-4B on an NVIDIA T4 (16GB) with llama.cpp: a prebuilt CUDA binary (XHToken fork — the spark2_5 architecture is not supported by mainline llama.cpp), Q8_0 / Q4_K_M model weights, a 1.7B draft model (boosts the Q8_0 config by +41%, 0.89 acceptance), and battle-tested one-click scripts (256k-context OpenAI-compatible API + public tunnel).

Verified end-to-end on 2026-09-06: streaming, native tool calling (--jinja), and a real agent workload (84-tool zagent, 12 consecutive tool calls, 240k prompt tokens, zero failures).

What's in this repo

File	Size	Description
`llama.cpp-spark-build-t4-cuda.tar.gz`	242MB	Prebuilt llama.cpp (XHToken fork) for CUDA / SM 7.5: `llama-server` + all runtime libs
`Spark-X2.5-4B-Q8_0.gguf`	4.38GB	8-bit quant — default: best quality
`Spark-X2.5-4B-Q4_K_M.gguf`	2.60GB	4-bit quant — VRAM-saving option (`QUANT=Q4_K_M`), most headroom
`Spark-X2.5-1.7B-Q8_0.gguf`	1.82GB	Draft model (same tokenizer, same `spark2_5` arch) for speculative decoding
`scripts/oneclick_start.sh`	–	One-click: checks → serve → tunnel → self-test
`scripts/spark_api.sh`	–	OpenAI-compatible API manager (start / stop / status / test)
`scripts/restore_build.sh`	–	Restore the prebuilt binary from this repo (skips the 40–60 min compile)
`scripts/deploy_spark_t4.sh`	–	Full pipeline: deps / build / download / serve
`scripts/tunnel_api.sh`	–	aitun.cc public tunnel manager

Quick start (~3 minutes on a fresh T4 Colab)

# Cell 1 — mount Drive
from google.colab import drive; drive.mount('/content/drive')

# Cell 2 — restore prebuilt binary (public repo, no token needed, ~1 min)
pip install -U huggingface_hub >/dev/null
hf download gasschina/Spark-X2.5-4B-build-cpp llama.cpp-spark-build-t4-cuda.tar.gz --local-dir /content/
mkdir -p /content/drive/MyDrive/spark-t4
tar xzf /content/llama.cpp-spark-build-t4-cuda.tar.gz -C /content/drive/MyDrive/spark-t4/

# Cell 3 — fetch scripts & launch (downloads weights to local disk + starts API + tunnel + self-test)
hf download gasschina/Spark-X2.5-4B-build-cpp --include "scripts/*.sh" --local-dir /content/s4b-scripts
cp /content/s4b-scripts/scripts/*.sh /content/drive/MyDrive/spark-t4/
bash /content/drive/MyDrive/spark-t4/oneclick_start.sh

The launcher prints the local endpoint, public tunnel URL, and API key when done. Model weights live on Colab local disk (/content/spark-t4-models), never eating into the 15GB Drive quota; the build and scripts persist on Drive across sessions.

Default config (v6): Q8_0 + 1.7B draft @ 256k (reverted 2026-09-06)

Item	Value	Notes
Weights	Q8_0 (4.38GB)	best quality
Speculative decoding	draft-simple + 1.7B	Q8_0 acceptance 0.89; 52.3 tok/s on English/temp-0 factual workloads. ⚠️ on long Chinese creative generation (temp 1.0) acceptance collapses — measured ~18.5-30 tok/s; use `SPEC_TYPE=none` (≈37 tok/s) or `QUANT=Q4_K_M` (≈48.75 tok/s, 6.6GB) if that is your workload
Context	262,144 (256k)	500k available at the same speed (51.7 tok/s / 10.1GB)
KV quant	q4_0	via `-ctk/-ctv`
VRAM	11.8GB / 15.36GB	3.5GB headroom
Throughput	52.3 tok/s (EN/temp-0)	prefill ~1160 tok/s; CN temp-1.0 long-form ~18.5-30 tok/s (draft acceptance collapses)
Thinking mode	server-side off by default	⚠️ the aitun tunnel hard-kills silent non-streaming requests at 120s → 504; thinking adds 30–90s of silence. Re-enable per request with `"chat_template_kwargs": {"enable_thinking": true}`
Auth	auto-generated key → Drive `.api_key`	`API_KEY=off` to disable

Draft policy is automatic by quant tier: QUANT=Q8_0 attaches the 1.7B draft (acceptance 0.89, 52.3 tok/s, 11.8GB); 4-bit runs bare. Override with SPEC_TYPE=draft-simple / SPEC_TYPE=none.

QUANT=Q8_0 bash oneclick_start.sh      # quality + draft: 52.3 tok/s, 11.8GB
CTX=500000 bash oneclick_start.sh      # 500k long-context, same speed, 10.1GB
QUANT=Q4_K_M CTX=786432 bash oneclick_start.sh   # 768k extreme, 14.3GB

1M context physically does not fit on a T4: 1M KV@q4_0 alone needs ~13GB before weights. Single slot (-np 1): concurrent requests queue behind long agent replies. On agent platforms (aicq.me etc.) prefer streaming and keep max_tokens ≤ 2000.

Benchmarks (T4, greedy, 384-token generations, /completion timings)

Config	Context	VRAM	Throughput
Q8_0, no draft (baseline)	500k	11.8GB	37.1 tok/s
Q8_0 + 1.7B draft (v6 default)	256k	11.8GB	52.3 tok/s (EN/temp-0); 18.5-30 tok/s CN/temp-1.0
Q4_K_M, no draft	256k	6.6GB	48.75 tok/s (EN/temp-0); 33-46 tok/s CN real-world
Q4_K_M, no draft	500k	10.1GB	51.7 tok/s
Q4_K_M + draft	256k	~10.1GB	27.2 tok/s (anti-optimization! acceptance 0.37-0.61)
Q4_K_M + draft	500k	OOM	draft compute buffer ~1.5GB
ngram-simple (self-speculation)	500k	+0GB	37.5 tok/s (no gain on prose)

Takeaway: the T4 is memory-bandwidth-bound. Drafting is a distribution-matching game — with a near-lossless Q8_0 target the 1.7B sibling hits 0.89 acceptance (+41%), but a 4-bit target's argmax flips too often and drafting goes net-negative. Gateway note: the free aitun tunnel terminates non-streaming requests at exactly 120s (reproduced); streaming is immune.

MTP status (why classic drafting)

llama.cpp already ships draft-mtp / draft-eagle3 / draft-dspark speculative types, but they require matching trained heads. Spark-X2.5-4B publishes no MTP/nextn/EAGLE weights (num_nextn_predict_layers is absent from its config), and the fork's spark2_5 backend has no nextn implementation. Classic draft-model speculation with the official same-vocabulary (131,072) 1.7B sibling is the best available option: 0.891 acceptance, mean accepted length 8.13 tokens on the Q8_0 target.

API usage (OpenAI-compatible)

import requests
BASE = "http://127.0.0.1:8100"   # or your public tunnel URL
KEY  = "sk-..."

r = requests.post(f"{BASE}/v1/chat/completions",
    headers={"Authorization": f"Bearer {KEY}"},
    json={"model": "spark-x2.5-4b",
          "messages": [{"role": "user", "content": "Hello"}],
          "max_tokens": 512})
print(r.json()["choices"][0]["message"]["content"])

Streaming: add "stream": true (SSE) — strongly recommended behind tunnels/gateways
Thinking mode (server-side off by default): add "chat_template_kwargs": {"enable_thinking": true}
Tool calling: pass a standard OpenAI tools array (--jinja enables native function calling)

Why this build exists

Mainline llama.cpp does not support the spark2_5 architecture (ggml-org/llama.cpp issue #28300). This package ships a CUDA build of the XHToken fork (SM 7.5 / CUDA 12.x) so you can skip the 40–60 minute compile. Weights: Q8_0 / Q4_K_M from the community i1 quant; the 1.7B draft quantized in-house (Q8_0) from the official F16 GGUF.

中文速览

讯飞星火 Spark-X2.5-4B 在 Colab T4 上的完整 llama.cpp 运行包。默认配置 v5 = Q4_K_M 裸跑 @256k 上下文 + 思考默认关（实测 48.75 tok/s，显存仅 6.6GB）——为 aicq.me 等智能体平台调优：aitun 隧道对非流式请求有 120s 硬超时（开思考必 504），故思考默认关闭、单请求可传 enable_thinking:true 恢复；草稿投机解码只对 Q8_0 有正收益（+41%，接受率 0.89），QUANT=Q8_0 时自动挂载，4bit 档实测反而慢一半已自动关闭。API 为 OpenAI 兼容格式，模型名 spark-x2.5-4b，智能体平台建议开流式、max_tokens ≤2000。

Downloads last month: 222

GGUF

Model size

2B params

Architecture

spark2_5

Hardware compatibility

4-bit

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for gasschina/Spark-X2.5-4B-build-cpp

Base model

XHToken/Spark-X2.5-4B-Base

Finetuned

XHToken/Spark-X2.5-4B

Quantized

(21)

this model