model loses its vision capabilities ?
Thank you for your work. Here is my feedback on using this model.
Comparison between the dense model and the OptiQ model
Dense = 18 t/s - OptiQ = 24,2 t/s
However, the model loses its vision capabilities.
Is this normal?
Tool used: Ka1zen MLX - https://github.com/Flor1an-B/Ka1zen
Hardware: MacBook Pro Apple M5 Max │ CPU 18c (18 log.) │ GPU 40c │ RAM 128 Go │ Disque 1.8 To
Hey, yeah we usually do multimodal stripping as part of the quantization to make the resultant weights as small as possible. if you need you can use mlx-optiq and pass --keep-unused-modalities to retain vision.
Thanks for your reply. That makes perfect sense. I'll give it a try.
Thanks again for your work.
You don't need to re-quantize. This repo already ships the vision tower at bf16 in optiq/optiq_vision.safetensors.
Ka1zen loads through stock mlx-lm, which picks weight files with glob("model*.safetensors"). optiq_vision.safetensors doesn't match that glob, so it never gets loaded and the model comes up text-only. That is by design: the same repo works as a text model under stock mlx-lm and as a vision model under OptiQ, so we don't have to publish two.
To get images, serve it with OptiQ:
pip install -U mlx-optiq
optiq serve --model mlx-community/Qwen3.6-27B-OptiQ-4bit
That gives you an OpenAI-compatible endpoint on 127.0.0.1:8080 which accepts image_url content parts.