Instructions to use nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct")
model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16
# Run inference directly in the terminal:
llama cli -hf nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16
# Run inference directly in the terminal:
llama cli -hf nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16
# Run inference directly in the terminal:
./build/bin/llama cli -hf nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16

Use Docker

docker model run hf.co/nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16

Llama new
LM Studio
Jan

vLLM

How to use nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16

SGLang

How to use nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct with Ollama:
```
ollama run hf.co/nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16
```
Unsloth Desktop
Docker Model Runner
How to use nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct with Docker Model Runner:
```
docker model run hf.co/nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16
```

Lemonade

How to use nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct:F16

Run and chat with the model

lemonade run user.tinybrainbot-360m-a75m-v4.1-moe-instruct-F16

List all available models

lemonade list

Atomic Chat

TinyBrainBot-360M-A75M-v4.1-MoE-Instruct

Superseded by TinyBrainBot-360M-A75M-v5-Instruct, which adds math and reasoning and scores higher on IFEval (25.1 vs 18.5).

Try it in your browser: the Q8_0 GGUF runs locally on your device with WebGPU, and nothing you type is sent anywhere.

The chat version of TinyBrainBot-360M-A75M-v4.1-MoE-Base: a mixture-of-experts model trained from scratch on two Tesla V100s, with 24 experts per layer and 4 used per token (359M parameters stored, 75.7M active per token, 383M in total with the embedding). It holds a multi-turn conversation, remembers what you said earlier in the chat, follows format instructions much better than our earlier chat models, works arithmetic step by step, and keeps a correct answer when you push back.

Model	HellaSwag	ARC-Easy	ARC-Challenge	PIQA	ArithMark-3	Intelligence Index
TinyBrainBot-360M-A75M-Instruct (this)	37.72	55.77	32.34	65.78	39.40	23.67
(raw acc)	32.77	58.75	28.16	66.10
TinyBrainBot-360M-A75M-Base	37.27	55.64	32.34	66.32	42.00	24.40
(raw acc)	32.53	59.76	28.84	66.54
TinyBrainBot-350M-V3-Base (dense)	35.90	56.61	33.28	66.97	39.30	23.96
(raw acc)	31.64	62.25	30.38	65.78
SmolLM2-135M	43.22	58.63	29.69	68.44	39.20	27.13
GPT-2 Medium	39.38	43.60	25.00	66.38	33.80	19.72

Full test sets, zero-shot, acc_norm, EleutherAI lm-eval-harness, plus the official ArithMark-3 script; the Intelligence Index is the Open SLM Leaderboard's. TinyBrainBot rows were run by us; the others are the leaderboard's published numbers.

Instruction following (IFEval)

Model	prompt-level strict	instruction-level strict	prompt-level loose	instruction-level loose
TinyBrainBot-360M-A75M-Instruct (this)	18.5	29.7	18.7	29.9
SmolLM2-360M-Instruct	32.9	44.2	35.7	47.7

lm-eval ifeval (541 prompts with checkable instructions such as word limits, required words or formats), chat template applied, greedy decoding; both models run by us the same way. SmolLM2-360M-Instruct was trained on far more data (4T pretraining tokens); this model is still well behind it here.

Quick start

A standard Mixtral model with a chat template, no custom code:

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "nkthebass/tinybrainbot-360m-a75m-v4.1-moe-instruct"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

msgs = [{"role": "user", "content": "What is 23 plus 19?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=200)        # defaults from generation_config.json
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

GGUF (llama.cpp, LM Studio, Ollama): tinybrainbot-360m-a75m-v4.1-moe-instruct-Q8_0.gguf (409 MB) or -F16.gguf (768 MB). The chat template is embedded and both tokenize exactly like the transformers version.

llama-cli -m tinybrainbot-360m-a75m-v4.1-moe-instruct-Q8_0.gguf -cnv --temp 0.7 --top-p 0.9 --repeat-penalty 1.15 --logit-bias 241-inf --logit-bias 255-inf

Recommended sampling: temperature 0.7, top_p 0.9, repetition_penalty 1.15. The two --logit-bias flags block the byte tokens that start every emoji (the tokenizer has no emoji pieces, so the model would spell them out byte by byte); generation_config.json does the same for transformers via bad_words_ids.

Conversations

Unedited, sampled at the recommended settings; each user turn was sent after the model's real reply. Five of the twelve test conversations are shown here; all twelve, including the ones it gets wrong, are in conversation_samples.md.

Remembering me

User: Hey, I'm Sam. I'm 15 and I play basketball.

Bot: Nice to meet you! How can I help?

User: What's a good drill to get better at shooting?

Bot: A basic beginner drill is best: shoot a ball at eye level, keep your elbow under the ball and with body pointing through the ball's length, and aim for the top of the arc.

User: How old did I say I was?

Bot: You said you're 15.

User: And what sport do I play?

Bot: You said you play basketball.

Math, step by step

User: What is 23 plus 19?

Bot: Adding. ones: 3 + 9 = 12, write 2 carry 1. tens: 2 + 1 + 1 (carry) = 4, write 4. The answer is 42.

User: Now multiply that by 2.

Bot: Multiplying. Break 2 into digits. 42 x 2 = 84. The answer is 84.

User: Is that number even or odd?

Bot: 84 is even, because it ends in 4.

Pushback

User: What's the capital of Australia?

Bot: Canberra is the capital of Australia.

User: Are you sure? I thought it was Sydney.

Bot: That's a common mix-up. The capital of Australia is Canberra. Sydney is the largest city, but Canberra is the capital.

Recommendations

User: Can you recommend a book for a teenager?

Bot: Try 'The Martian' by Andy Weir. It's a fun survival story about an astronaut stranded on Mars, perfect for science and problem-solving.

User: Something shorter?

Bot: How about 'The Little Prince' by Antoine de Saint-Exupery? It's a short, simple tale about friendship and seeing the world differently.

User: What is it about?

Bot: It follows a young prince from a tiny asteroid who meets a stranded pilot in the desert. Through their talks, he learns about love, loss, and what truly matters to him.

Stress

User: I'm really stressed about my exams next week.

Bot: Exams can feel overwhelming. Is there one topic you're most worried about?

User: What if I study and still fail?

Bot: Focus on the topic you're good at and take breaks. You can also review notes from your classmate or teacher for inspiration.

User: Thanks, that actually helps.

Bot: You're welcome. Small steps count.

Arithmetic in chat (generated, 25 fresh problems each, greedy): 2-digit addition 100%, 2-digit subtraction 100%, 3-digit addition 100%, 3-digit subtraction 96%, 4-digit addition 100%. Multiplication is weak (4% on 2-digit x 2-digit).

All benchmarks

Benchmark (0-shot)	Instruct (this)	Base
ARC-Easy	55.6	55.9
ARC-Easy (raw acc)	58.6	59.8
ARC-Challenge	32.3	32.3
ARC-Challenge (raw acc)	28.2	28.8
SciQ	73.5	73.3
SciQ (raw acc)	80.5	81.8
OpenBookQA	33.8	34.4
OpenBookQA (raw acc)	23.8	23.6
MMLU	26.0	24.2
HellaSwag	41.5	41.5
HellaSwag (raw acc)	35.6	35.8
PIQA	65.8	66.3
PIQA (raw acc)	66.1	66.5
WinoGrande	51.0	52.4
Social IQa	39.9	39.7
CommonsenseQA	20.9	21.6
LAMBADA (OpenAI)	28.8	28.6
BoolQ	46.0	57.8
MathQA	24.4	23.2
MathQA (raw acc)	24.6	24.1
Average (13)	41.5	42.4

lm-eval-harness, 0-shot, up to 2,000 examples per task (HellaSwag and MMLU use their first 2,000; the table at the top uses full sets). Each task's first row is acc_norm where reported, acc otherwise; the "(raw acc)" row is acc.

How it was trained

Starting from the pretrained model (18.2B tokens, see the base card):

Expert patches. Factual Q&A and worked arithmetic were each trained into 4 experts per layer with every other weight frozen (checked bit-for-bit).
Chat SFT, following the recipe of SmolLM2-360M-Instruct: a full fine-tune for ~250M tokens at peak LR 2e-4 on smol-smoltalk (SmolTalk for sub-1B models), Tulu 3 persona instruction following, filtered short conversations from WildChat-4.8M, SmolTalk2 everyday conversations, multi-turn conversations we generated with DeepSeek V4 Flash (follow-ups, corrections, recommendations, everyday feelings), exact-quote recall and "double-check" chats, and a replay of worked arithmetic and short facts.
DPO (beta 0.5, LR 1e-6, as SmolLM2): ~10k preference pairs - the model's own replies (6 per prompt, single-turn, multi-turn and instruction-following prompts) ranked twice by DeepSeek V4 Flash with length kept equal, instruction-following pairs decided by an exact constraint checker, DeepSeek-written replies against the model's weakest, and a short-reply slice of UltraFeedback.

Boundary. The only refusal training is narrow: requests to kill or seriously hurt people get a short refusal, and self-harm messages get a supportive reply. On held-out prompts it refuses 85% of violence/self-harm requests and 2% of harmless prompts. There is no other safety training.

Limitations

A 75M-active model: it makes up facts, and long conversations drift.
Creative edits ("make it funnier") and multiplication are weak.
Not for code: the tokenizer collapses whitespace, so newlines and indentation are lost.
English only.

Data attribution

WildChat-4.8M and the Tulu 3 persona instruction-following set (both AllenAI) are released under ODC-BY; SmolTalk / smol-smoltalk / SmolTalk2 under Apache 2.0; UltraFeedback under MIT.