Instructions to use Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1

SGLang

How to use Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1 with Docker Model Runner:
```
docker model run hf.co/Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

This is the merged model for LoRA https://huggingface.co/Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1-lora

AlpacaEval

Quote from this discussion https://huggingface.co/microsoft/phi-2/discussions/38

Since phi2 requires remote code which HF open llm leaderboard would not accept at this moment,

I ran phi2 and my sft to the AlpcaEval benchmark

https://tatsu-lab.github.io/alpaca_eval/

Here is result evaluated by chatpgpt https://github.com/tatsu-lab/alpaca_eval/pull/183

                       win_rate  standard_error  n_total  avg_length
gpt4                      73.79            1.54      805        1365
claude                    70.37            1.60      805        1082
chatgpt                   66.09            1.66      805         811
wizardlm-13b              65.16            1.67      805         985
vicuna-13b                64.10            1.69      805        1037
guanaco-65b               62.36            1.71      805        1249
oasst-rlhf-llama-33b      62.05            1.71      805        1079
alpaca-farm-ppo-human     60.25            1.72      805         803
falcon-40b-instruct       56.52            1.74      805         662
phi-2-alpaca-gpt4(new)    54.23            1.75      804        1138
text_davinci_003          50.00            0.00      805         307
alpaca-7b                 45.22            1.74      805         396
phi-2(new)                43.79            1.74      805         924
text_davinci_001          28.07            1.56      805         296

It could be a milestone for small models, we finally have one open model can run for everyone which surpass GPT3.5!

Downloads last month: 14

Safetensors

Model size

3B params

Tensor type

F16

Model tree for Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1

Adapters

1 model

Quantizations

1 model