Instructions to use mradermacher/DeepSeek-Coder-V2-Instruct-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mradermacher/DeepSeek-Coder-V2-Instruct-GGUF with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("mradermacher/DeepSeek-Coder-V2-Instruct-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
No Model files, is uploading ?
#1
by jiyintor - opened
No Model files, is uploading ?
Yup, it's uploading.
mradermacher changed discussion status to closed
btw., i-quants are arriving concurrently here: https://huggingface.co/mradermacher/DeepSeek-Coder-V2-Instruct-i1-GGUF
but of course it will take a while, the model being quite... big.
I see, Thank you for your work.
btw., Q4 is on the way ?
You can see the order in which quants are generated in the faq (https://huggingface.co/mradermacher/model_requests). There is no such quant as a Q4 - I recommend the Q4_K_S or Q4_K_M, though. If you have the ram (I don't).
yup, i mean the Q4_K_S or Q4_K_M