Instructions to use prithivMLmods/Llama-Sentient-3.2-3B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use prithivMLmods/Llama-Sentient-3.2-3B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="prithivMLmods/Llama-Sentient-3.2-3B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("prithivMLmods/Llama-Sentient-3.2-3B-Instruct")
model = AutoModelForCausalLM.from_pretrained("prithivMLmods/Llama-Sentient-3.2-3B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use prithivMLmods/Llama-Sentient-3.2-3B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "prithivMLmods/Llama-Sentient-3.2-3B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "prithivMLmods/Llama-Sentient-3.2-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/prithivMLmods/Llama-Sentient-3.2-3B-Instruct

SGLang

How to use prithivMLmods/Llama-Sentient-3.2-3B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "prithivMLmods/Llama-Sentient-3.2-3B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "prithivMLmods/Llama-Sentient-3.2-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "prithivMLmods/Llama-Sentient-3.2-3B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "prithivMLmods/Llama-Sentient-3.2-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use prithivMLmods/Llama-Sentient-3.2-3B-Instruct with Docker Model Runner:
```
docker model run hf.co/prithivMLmods/Llama-Sentient-3.2-3B-Instruct
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Llama-Sentient-3.2-3B-Instruct Modelfile

The Llama-Sentient-3.2-3B-Instruct model is a fine-tuned version of the Llama-3.2-3B-Instruct model, optimized for text generation tasks, particularly where instruction-following abilities are critical. This model is trained on the mlabonne/lmsys-arena-human-preference-55k-sharegpt dataset, which enhances its performance in conversational and advisory contexts, making it suitable for a wide range of applications.

File Name	Size	Description	Upload Status
`.gitattributes`	1.57 kB	Git attributes configuration file	Uploaded
`README.md`	42 Bytes	Initial commit README	Uploaded
`config.json`	1.04 kB	Configuration file	Uploaded
`generation_config.json`	248 Bytes	Generation configuration file	Uploaded
`pytorch_model-00001-of-00002.bin`	4.97 GB	PyTorch model file (part 1)	Uploaded (LFS)
`pytorch_model-00002-of-00002.bin`	1.46 GB	PyTorch model file (part 2)	Uploaded (LFS)
`pytorch_model.bin.index.json`	21.2 kB	Model index file	Uploaded
`special_tokens_map.json`	477 Bytes	Special tokens mapping	Uploaded
`tokenizer.json`	17.2 MB	Tokenizer JSON file	Uploaded (LFS)
`tokenizer_config.json`	57.4 kB	Tokenizer configuration file	Uploaded

Model Type	Size	Context Length	Link
GGUF	3B	-	🤗 Llama-Sentient-3.2-3B-Instruct-GGUF

Key Use Cases:

Conversational AI: Engage in intelligent dialogue, offering coherent responses and following instructions, useful for customer support and virtual assistants.
Text Generation: Generate high-quality, contextually appropriate content such as articles, summaries, explanations, and other forms of written communication based on user prompts.
Instruction Following: Follow specific instructions with accuracy, making it ideal for tasks that require structured guidance, such as technical troubleshooting or educational assistance.

The model uses a PyTorch-based architecture and includes a range of necessary files such as configuration files, tokenizer files, and model weight files for deployment.

Intended Applications:

Chatbots for virtual assistance, customer support, or as personal digital assistants.
Content Creation Tools, aiding in the generation of written materials, blog posts, or automated responses based on user inputs.
Educational and Training Systems, providing explanations and guided learning experiences in various domains.
Human-AI Interaction platforms, where the model can follow user instructions to provide personalized assistance or perform specific tasks.

With its strong foundation in instruction-following and conversational contexts, the Llama-Sentient-3.2-3B-Instruct model offers versatile applications for both general and specialized domains.

Downloads last month: 7

Model tree for prithivMLmods/Llama-Sentient-3.2-3B-Instruct

Base model

meta-llama/Llama-3.2-3B-Instruct

Finetuned

(1598)

this model

Merges

2 models

Quantizations

4 models

Dataset used to train prithivMLmods/Llama-Sentient-3.2-3B-Instruct

Collection including prithivMLmods/Llama-Sentient-3.2-3B-Instruct

Custom Build Models

Collection

Text Generation Cook with it's GGUF • 53 items • Updated 20 days ago • 2