Instructions to use Harsha001/LFM3-1.7B-Base-ROCmFPX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Harsha001/LFM3-1.7B-Base-ROCmFPX with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP # Run inference directly in the terminal: llama cli -hf Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP # Run inference directly in the terminal: llama cli -hf Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP # Run inference directly in the terminal: ./build/bin/llama cli -hf Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
Use Docker
docker model run hf.co/Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
- Llama new
- LM Studio
- Jan
- vLLM
How to use Harsha001/LFM3-1.7B-Base-ROCmFPX with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Harsha001/LFM3-1.7B-Base-ROCmFPX" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Harsha001/LFM3-1.7B-Base-ROCmFPX", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
- Ollama
How to use Harsha001/LFM3-1.7B-Base-ROCmFPX with Ollama:
ollama run hf.co/Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
- Unsloth Desktop
- Pi
How to use Harsha001/LFM3-1.7B-Base-ROCmFPX with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Harsha001/LFM3-1.7B-Base-ROCmFPX with Docker Model Runner:
docker model run hf.co/Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
- Lemonade
How to use Harsha001/LFM3-1.7B-Base-ROCmFPX with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
Run and chat with the model
lemonade run user.LFM3-1.7B-Base-ROCmFPX-Q4_0_ROCMFP
List all available models
lemonade list
- Hermes Agent
How to use Harsha001/LFM3-1.7B-Base-ROCmFPX with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Harsha001/LFM3-1.7B-Base-ROCmFPX with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "Harsha001/LFM3-1.7B-Base-ROCmFPX:Q4_0_ROCMFP" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
LFM3-1.7B-Base, ROCmFPX GGUF
LiquidAI/LFM3-1.7B-Base, the pretrained base model of LFM3-1.7B, without chat tuning. It is meant for fine-tuning and plain text completion, not for chat, quantized for the ROCmFPX llama.cpp fork (low-bit weight types for AMD GPUs, Vulkan and HIP/ROCm).
The ROCmFPX files (8-bit, 6-bit and the 4-bit Strix file) need HarshaVardhan3002/ROCmFPX. Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds may not load it. These files were checked on the CPU and have not been tested on AMD hardware yet.
Not an official release of Liquid AI. All credit for the model goes to its authors.
Files
| File | Size | Weights | KL divergence vs BF16 (lower is better) | Same top token as BF16 |
|---|---|---|---|---|
LFM3-1.7B-Base-Q8_0_ROCMFPX.gguf |
1.80 GB | ROCmFPX 8-bit (Q8_0_ROCMFPX with tensor scales) |
0.0007* | 98.7%* |
LFM3-1.7B-Base-Q6_K_ROCMFPX.gguf (recommended) |
1.43 GB | ROCmFPX 6-bit (Q6_0_ROCMFPX with tensor scales) |
0.0031 | 96.9% |
LFM3-1.7B-Base-Q4_0_ROCMFP4_STRIX_O8.gguf |
1.06 GB | ROCmFP4-FAST 4-bit for Strix Halo, attention and output layer 8-bit | 0.040* | 89.7%* |
| stock Q8_0, for comparison (not uploaded) | 1.85 GB | stock llama.cpp Q8_0 | 0.0006 | 98.7% |
| stock Q4_K_M, for comparison (not uploaded) | 1.10 GB | stock llama.cpp Q4_K_M | 0.0267 | 92.0% |
| stock Q6_K, for comparison (not uploaded) | 1.43 GB | stock llama.cpp Q6_K | 0.0039 | 96.6% |
- Measured on: 40 chunks of 512 tokens of WikiText-2 (test split), against the BF16 GGUF, CPU.
- The ROCmFPX 6-bit file beats stock Q6_K at the same size. Its file name says
Q6_K_ROCMFPXso that Hugging Face lists it as a 6-bit quant; the weights are the ROCmFPXQ6_0_ROCMFPXtype, not a stockQ6_K. - * The 8-bit and 4-bit ROCmFPX files were added on 20 November 2026. Their numbers come from the October run of almost the same recipes (the new files also keep the output layer at 8-bit); the new files themselves passed a short answer test, and their own numbers will replace these.
- The 4-bit Strix file is built for AMD Strix Halo: ROCmFP4-FAST weights run on the fork's fast FP4 kernels, with attention and the output layer kept at 8-bit. It drifts more from BF16 than stock Q4_K_M (0.027) on this model, so pick the 6-bit or 8-bit file when quality matters more than speed and size.
- Every file passed a short greedy answer test before upload.
How these were made
- BF16 GGUF converted with the fork's
convert_hf_to_gguf.py. - Importance matrix from the BF16 file over a general calibration text (
calibration_datav3), 512-token chunks. llama-quantize --imatrixfrom the BF16 file for every file, stock ones included. The 6-bit file was made withLLAMA_ROCMFPX_TENSOR_SCALE=1, which stores a power-of-two scale per weight tensor (the runtime applies it; no setting is needed to run the file), with--token-embedding-type q6_K.
Run
git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build -j
build/bin/llama-completion -m LFM3-1.7B-Base-Q6_K_ROCMFPX.gguf -ngl 99 -p "The capital of France is" -n 64
This is a base model: give it text to continue, or fine-tune it. For chat use LFM3-1.7B instead.
LFM3 support in the fork comes from llama.cpp pull request #31252. LFM3-Omni and LFM3-Audio need further upstream llama.cpp work and will follow later.
License
LFM Open License v1.0, same as the base model. Commercial use is allowed for companies below 10 million US
dollars of annual revenue; above that it is not covered by the license. Read LICENSE before you use these files.
- Downloads last month
- 280
4-bit
6-bit
8-bit
Model tree for Harsha001/LFM3-1.7B-Base-ROCmFPX
Base model
LiquidAI/LFM3-1.7B-Base