antontuzovAI/MiniCPM5-2B-xLAM-60k-Production-Agent

MiniCPM5-2B xLAM 60k Production Agent (QLoRA)

This is a highly optimized, enterprise-grade QLoRA adapter for openbmb/MiniCPM5-2B, specifically fine-tuned to act as a strict Function-Calling / Tool-Use agent.

It was trained on the full 60,000 examples of the Salesforce/xlam-function-calling-60k dataset. The model has seen all 3,673 unique real-world APIs across 21 different categories.

🚀 Key Features

• 100% JSON Parse Rate: Strictly outputs valid JSON arrays of tool calls without conversational filler, markdown, or <think> reasoning tags.
• Full xLAM Coverage: Trained on the complete 60k dataset, giving it deep knowledge of complex, nested, and multi-tool API schemas.
• Lightweight & Fast: Uses 4-bit QLoRA (Rank 32, Alpha 64), making it extremely cheap and fast to deploy alongside the base 2B model.
• T4 Optimized: Training and inference were heavily optimized to run on standard 16GB VRAM GPUs (like the Tesla T4).

📊 Training Details

• Base Model: openbmb/MiniCPM5-2B
• Dataset: Salesforce/xlam-function-calling-60k (Full 60,000 samples)
• Method: QLoRA (4-bit NF4)
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• LoRA Rank (r): 32 | Alpha: 64
• Optimizer: AdamW (Cosine Schedule with Warmup)
• Max Sequence Length: 896 tokens

💻 How to Use

You need the base model and the peft library to load this adapter.

import torch
import json
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

model_id = "openbmb/MiniCPM5-2B"
adapter_id = "antontuzovAI/MiniCPM5-2B-xLAM-60k-Production-Agent"

# 1. Setup 4-bit Quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True
)

# 2. Load Base Model & Tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token

base_model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16
)

# 3. Load the Agent LoRA
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

# 4. Define Tools
tools = [
    {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "city": {"type": "string", "description": "City name", "required": True},
            "units": {"type": "string", "enum": ["celsius", "fahrenheit"]}
        }
    }
]

# 5. Build Prompt
tools_str = json.dumps(tools, indent=2)
system_msg = (
    "You are a strict tool-calling agent.\n"
    "Output ONLY a valid JSON array of tool calls.\n"
    'Format: [{"name": "tool_name", "arguments": {"arg": "val"}}]\n'
    "If no tool is needed, output: []\n\n"
    f"Available tools:\n{tools_str}"
)

messages = [
    {"role": "system", "content": system_msg},
    {"role": "user", "content": "What is the weather in Tokyo in celsius?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

# 6. Generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False, pad_token_id=tokenizer.eos_token_id)

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print("Agent Output:\n", response)

🏗️ Architecture Note: Two-Brain Agent

Because this LoRA is heavily optimized for strict JSON routing, it is recommended to use a "Two-Brain" architecture in production:

1. Brain 1 (This LoRA): Routes the query and generates the JSON tool calls.
2. Brain 2 (Base Model): Synthesizes the tool observations into a natural language response for the user.

To switch back to the base model for synthesis, simply use model.disable_adapter_layers() before generating the final answer.