Add tool calling to the chat template and a response template

#43
by Rocketknight1 HF Staff - opened

Adds tool calling in the format mistral-common uses for this tokenizer (tool definitions before the last user message, tool calls as a JSON list with ids, tool results with their call ids), and makes the default system prompt the repo's SYSTEM_PROMPT.txt with today's date, as in the model card. It should work in Transformers, llama.cpp and vLLM. Also adds a response template, so Transformers' parse_response turns the model's output back into a message (content and tool calls), matching mistral-common's own parser. Also moves the chat template from tokenizer_config.json into chat_template.jinja, where it marks the assistant's output with {% generation %} tags, so return_assistant_tokens_mask (TRL's assistant_only_loss) trains on just that; the tags don't change the rendered text.

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment