Instructions to use HelpingAI/HelpingAI2.5-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HelpingAI/HelpingAI2.5-2B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HelpingAI/HelpingAI2.5-2B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("HelpingAI/HelpingAI2.5-2B")
model = AutoModelForCausalLM.from_pretrained("HelpingAI/HelpingAI2.5-2B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

llama-cpp-python

How to use HelpingAI/HelpingAI2.5-2B with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="HelpingAI/HelpingAI2.5-2B",
	filename="q4_k_m.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use HelpingAI/HelpingAI2.5-2B with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf HelpingAI/HelpingAI2.5-2B:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf HelpingAI/HelpingAI2.5-2B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf HelpingAI/HelpingAI2.5-2B:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf HelpingAI/HelpingAI2.5-2B:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf HelpingAI/HelpingAI2.5-2B:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf HelpingAI/HelpingAI2.5-2B:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf HelpingAI/HelpingAI2.5-2B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf HelpingAI/HelpingAI2.5-2B:Q4_K_M

Use Docker

docker model run hf.co/HelpingAI/HelpingAI2.5-2B:Q4_K_M

LM Studio
Jan

vLLM

How to use HelpingAI/HelpingAI2.5-2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HelpingAI/HelpingAI2.5-2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HelpingAI/HelpingAI2.5-2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/HelpingAI/HelpingAI2.5-2B:Q4_K_M

SGLang

How to use HelpingAI/HelpingAI2.5-2B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HelpingAI/HelpingAI2.5-2B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HelpingAI/HelpingAI2.5-2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HelpingAI/HelpingAI2.5-2B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HelpingAI/HelpingAI2.5-2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use HelpingAI/HelpingAI2.5-2B with Ollama:
```
ollama run hf.co/HelpingAI/HelpingAI2.5-2B:Q4_K_M
```

Unsloth Studio new

How to use HelpingAI/HelpingAI2.5-2B with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for HelpingAI/HelpingAI2.5-2B to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for HelpingAI/HelpingAI2.5-2B to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for HelpingAI/HelpingAI2.5-2B to start chatting

Docker Model Runner
How to use HelpingAI/HelpingAI2.5-2B with Docker Model Runner:
```
docker model run hf.co/HelpingAI/HelpingAI2.5-2B:Q4_K_M
```

Lemonade

How to use HelpingAI/HelpingAI2.5-2B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull HelpingAI/HelpingAI2.5-2B:Q4_K_M

Run and chat with the model

lemonade run user.HelpingAI2.5-2B-Q4_K_M

List all available models

lemonade list

HelpingAI2.5-2B / LICENSE.md

Abhaykoul

Upload folder using huggingface_hub

69e2eed verified over 1 year ago

preview code

raw

history blame contribute delete

8.99 kB

HelpingAI License

Version 3.0

PREAMBLE

The HelpingAI License (the "License") is designed to govern the use, modification, and distribution of HelpingAI's proprietary assets, including but not limited to artificial intelligence models, software, algorithms, weights, documentation, datasets, and associated materials (collectively, the "Content"). This License aims to balance open innovation with the protection of HelpingAI's intellectual property rights while ensuring responsible and ethical use of AI technology.

1. DEFINITIONS

1.1. "Model" means any machine learning model, artificial intelligence system, neural network architecture, or AI-powered solution developed by HelpingAI.

1.2. "Weights" refers to the trained parameters, embeddings, model checkpoints, and other numerical values that define the behavior and capabilities of the Model.

1.3. "Source Code" means the human-readable form of software, documentation, configuration files, and associated resources.

1.4. "Derivative Work" means any work that is based upon or derived from the Content, including but not limited to: a) Fine-tuned or adapted versions of the Model b) Modified or enhanced versions of the Source Code c) Products, services, or applications that incorporate the Content d) Adaptations, transformations, or improvements of the Content e) Any output generated using the Model or Content

1.5. "Commercial Use" means any use of the Content intended for or directed toward: a) Commercial advantage or monetary compensation b) Revenue generation, either directly or indirectly c) Business operations or commercial services d) Integration into commercial products or offerings

1.6. "End User" means any individual or entity that uses the Content or Derivative Works.

2. GRANT OF RIGHTS

Subject to the terms and conditions of this License, HelpingAI hereby grants you a limited, non-exclusive, non-transferable, revocable license to:

2.1. Access and use the Content for: a) Research and evaluation purposes b) Educational and academic use c) Personal, non-commercial projects d) Open-source contributions (subject to Section 3)

2.2. Create and use Derivative Works for non-commercial purposes, provided that: a) You comply with all restrictions in Section 3 b) You maintain appropriate attribution c) You ensure responsible and ethical use

2.3. Distribute the Content or Derivative Works to third parties, provided that: a) You maintain all copyright notices and attributions b) You include a complete, unmodified copy of this License c) You clearly document any modifications made d) You comply with all restrictions in Section 3 e) You ensure recipients agree to this License

3. RESTRICTIONS AND LIMITATIONS

3.1. Commercial Use Restrictions You may not use the Content or any Derivative Works for Commercial Use without: a) Obtaining a separate commercial license from HelpingAI b) Paying applicable fees and royalties c) Receiving written authorization

3.2. Model Usage Restrictions You may not: a) Use the Model to train or develop competing AI models or services b) Extract, reverse engineer, or decompile the Model's architecture or weights c) Remove, disable, or circumvent any access controls or security measures d) Use the Model for military purposes, weapons development, or harmful applications e) Use the Model to generate harmful, discriminatory, or illegal content f) Use the Model in safety-critical applications without explicit written permission g) Use the Model in ways that violate privacy or data protection laws h) Create autonomous systems that could cause harm to individuals or society

3.3. Distribution Restrictions You may not: a) Redistribute the Model's weights or checkpoints without explicit permission b) Create hosted APIs or services that provide direct access to the Model c) Bundle or integrate the Content with commercial products or services d) Sublicense or transfer rights to the Content to third parties e) Distribute the Content in ways that could harm HelpingAI's interests

3.4. Branding and Attribution You must: a) Maintain all HelpingAI branding, logos, notices, and watermarks b) Include the following attribution in all Derivative Works: "Powered by HelpingAI technology, licensed under the HelpingAI License v3.0" c) Not use HelpingAI's name, logos, or trademarks to endorse or promote products d) Not suggest any official association with HelpingAI without permission

4. INTELLECTUAL PROPERTY

4.1. HelpingAI retains all rights, title, and interest in and to the Content, including: a) All intellectual property rights b) Patents, trademarks, and trade secrets c) Proprietary methods and algorithms d) Future improvements and modifications

4.2. This License does not grant you any rights to HelpingAI's: a) Patents or patent applications b) Trademarks or service marks c) Trade secrets or proprietary information d) Other intellectual property except as expressly stated

4.3. Derivative Works: a) You retain ownership of original portions of Derivative Works you create b) HelpingAI retains all rights to the underlying Content c) You grant HelpingAI a perpetual license to any improvements or feedback

5. DATA COLLECTION AND PRIVACY

5.1. Data Collection: a) HelpingAI may collect usage data and performance metrics b) Analytics may be used to improve the Content c) Collection will comply with applicable privacy laws

5.2. Privacy Requirements: a) You must comply with all applicable privacy laws and regulations b) You must provide appropriate privacy notices to End Users c) You must obtain necessary consents for data collection d) You must implement appropriate data security measures

6. WARRANTY DISCLAIMER

THE CONTENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO: a) WARRANTIES OF MERCHANTABILITY b) FITNESS FOR A PARTICULAR PURPOSE c) NON-INFRINGEMENT d) ACCURACY OR COMPLETENESS e) ABSENCE OF ERRORS OR DEFECTS f) COMPATIBILITY WITH OTHER SOFTWARE g) SECURITY OR PERFORMANCE

7. LIMITATION OF LIABILITY

7.1. IN NO EVENT SHALL HELPINGAI BE LIABLE FOR ANY: a) DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES b) LOSS OF PROFITS OR BUSINESS OPPORTUNITIES c) BUSINESS INTERRUPTION OR LOST DATA d) PERSONAL INJURY OR PROPERTY DAMAGE e) ANY OTHER DAMAGES OR LOSSES

7.2. THIS LIMITATION APPLIES TO: a) CONTRACT OR TORT CLAIMS b) BREACH OF WARRANTY c) ANY OTHER LEGAL THEORY d) ALL USES OF THE CONTENT

8. TERMINATION

8.1. Automatic Termination: a) This License terminates automatically upon any breach b) No notice is required for termination c) All rights granted immediately cease

8.2. Upon Termination: a) Cease all use of the Content b) Destroy all copies and installations c) Stop distribution of Derivative Works d) Certify compliance with termination terms

8.3. Survival: a) Sections 4, 6, 7, 9, and 10 survive termination b) Obligations to protect intellectual property continue c) Liability limitations remain in effect

9. GOVERNING LAW AND JURISDICTION

9.1. This License shall be governed by and construed in accordance with: a) The laws of the United States of America b) The State of California c) Without regard to conflicts of law principles

9.2. Dispute Resolution: a) Exclusive jurisdiction in Santa Clara County, California b) Agreement to personal jurisdiction c) Waiver of jury trial d) Prevailing party entitled to attorney fees

10. EXPORT COMPLIANCE

10.1. You must comply with all applicable: a) Export control laws and regulations b) Trade sanctions and embargoes c) International trade restrictions d) Anti-boycott regulations

10.2. You may not export the Content to: a) Prohibited countries or regions b) Restricted end users c) Prohibited end uses

11. MODIFICATIONS TO LICENSE

11.1. License Updates: a) HelpingAI may modify this License at any time b) Changes effective upon posting c) Continued use constitutes acceptance

11.2. Notification: a) Material changes will be announced b) Users should regularly review terms c) Opt-out requires discontinuing use

12. SUPPORT AND UPDATES

12.1. No Obligation: a) HelpingAI has no obligation to provide support b) Updates and maintenance are discretionary c) Services may be modified or discontinued

12.2. Available Support: a) Documentation and resources provided "as is" b) Community forums may be available c) Commercial support requires separate agreement

Last Updated: NOVEMBER 2024

END OF LICENSE