Instructions to use hridya423/lexis-0.5b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use hridya423/lexis-0.5b-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf hridya423/lexis-0.5b-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf hridya423/lexis-0.5b-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf hridya423/lexis-0.5b-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf hridya423/lexis-0.5b-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf hridya423/lexis-0.5b-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf hridya423/lexis-0.5b-GGUF:Q4_K_M
Use Docker
docker model run hf.co/hridya423/lexis-0.5b-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use hridya423/lexis-0.5b-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "hridya423/lexis-0.5b-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hridya423/lexis-0.5b-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/hridya423/lexis-0.5b-GGUF:Q4_K_M
- Ollama
How to use hridya423/lexis-0.5b-GGUF with Ollama:
ollama run hf.co/hridya423/lexis-0.5b-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use hridya423/lexis-0.5b-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf hridya423/lexis-0.5b-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "hridya423/lexis-0.5b-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use hridya423/lexis-0.5b-GGUF with Docker Model Runner:
docker model run hf.co/hridya423/lexis-0.5b-GGUF:Q4_K_M
- Lemonade
How to use hridya423/lexis-0.5b-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull hridya423/lexis-0.5b-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.lexis-0.5b-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use hridya423/lexis-0.5b-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf hridya423/lexis-0.5b-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default hridya423/lexis-0.5b-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use hridya423/lexis-0.5b-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf hridya423/lexis-0.5b-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "hridya423/lexis-0.5b-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
lexis-0.5b
Plain English in, one shell command out, for macOS, Linux and Windows. This is the small local model for Lexis, a terminal assistant that shows every command on an approval card before it runs. Lexis picks it on machines with less than 8 GB of memory.
It is a full fine-tune of Qwen2.5-Coder-0.5B-Instruct on 147,065 requests whose commands were each checked on their target OS. The file is Q4_K_M, 398 MB, and answers in about 100 ms on an Apple M4 Pro.
| macOS | Linux | Windows | All | |
|---|---|---|---|---|
| Final held-out set, first try | 246 / 337 | 261 / 332 | 207 / 250 | 714 / 919 (77.7%) |
The untrained base model scores 188 / 919 on the same set. lexis-1.5b scores 757 / 919 at 986 MB and about 250 ms; use it if you have the memory. METHODOLOGY.md covers the data, the checks, the evaluation, and the experiments that didn't help.
Use it through Lexis
curl -fsSL https://lexis.hridya.tech/install.sh | bash # macOS, Linux
irm https://lexis.hridya.tech/win.ps1 | iex # Windows
lx "biggest 10 files in my Downloads"
Use it directly
The model expects exactly the prompt it was trained on: a fixed system prompt, then the request with a platform line appended.
llama-server -m lexis-0.5b-q4_k_m.gguf --jinja -c 4096 --port 8080
curl -s localhost:8080/v1/chat/completions -d '{
"temperature": 0,
"messages": [
{"role": "system", "content": "You are a bash command generator. Given a natural language request, output only the bash command that accomplishes it. No explanation, no markdown fences."},
{"role": "user", "content": "find all pdfs on my desktop\n(Platform: macOS (BSD), shell: zsh)"}
]}' | jq -r '.choices[0].message.content'
Platform lines used in training:
(Platform: macOS (BSD), shell: zsh)(Platform: Linux (GNU), shell: bash)(Platform: windows, shell: powershell), with this system prompt instead:You are a PowerShell command generator. Given a natural language request, output only the PowerShell command that accomplishes it. No explanation, no markdown fences.
To fix a failed command, use You fix broken bash commands. Given a failed command and its error output, output only the corrected command. No explanation. as the system prompt. The user message is the command after $ , then the error output on the following lines, with no platform line.
Limits
- One command per answer, no explanation. Multi-step tasks come back joined with
&∨. - About one first-try answer in four or five is wrong. It knows fewer rare tools than the 1.5B and is more likely to use a Linux-only flag on macOS.
- It does what it's asked, including destructive things. It has no safety judgment of its own. Don't execute its output without a review step; Lexis provides one.
- Downloads last month
- 18
4-bit
Model tree for hridya423/lexis-0.5b-GGUF
Base model
Qwen/Qwen2.5-0.5B