Instructions to use mllmTeam/PhoneLM-0.5B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mllmTeam/PhoneLM-0.5B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mllmTeam/PhoneLM-0.5B-Instruct", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("mllmTeam/PhoneLM-0.5B-Instruct", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mllmTeam/PhoneLM-0.5B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mllmTeam/PhoneLM-0.5B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mllmTeam/PhoneLM-0.5B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mllmTeam/PhoneLM-0.5B-Instruct

SGLang

How to use mllmTeam/PhoneLM-0.5B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mllmTeam/PhoneLM-0.5B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mllmTeam/PhoneLM-0.5B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mllmTeam/PhoneLM-0.5B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mllmTeam/PhoneLM-0.5B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use mllmTeam/PhoneLM-0.5B-Instruct with Docker Model Runner:
```
docker model run hf.co/mllmTeam/PhoneLM-0.5B-Instruct
```

PhoneLM-0.5B-Instruct / README.md

mllmTeam

Update README.md

ef81075 verified almost 2 years ago

preview code

Raw

History Blame Contribute Delete

3.53 kB

metadata

license: apache-2.0
datasets:
  - stingning/ultrachat
  - TIGER-Lab/MathInstruct
  - ise-uiuc/Magicoder-Evol-Instruct-110K
  - OpenAssistant/oasst2
  - teknium/openhermes
  - bigcode/commitpackft
  - Open-Orca/SlimOrca
  - ise-uiuc/Magicoder-OSS-Instruct-75K
language:
  - en
library_name: transformers
base_model:
  - mllmTeam/PhoneLM-0.5B

PhoneLM-0.5B-Instruct is a 0.5 billion parameter decoder-only language model.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = 'mllmTeam/PhoneLM-0.5B-Instruct'
question = "Hello, who are you?"
prompt = [{"role": "user", "content": question}]

model = AutoModelForCausalLM.from_pretrained(model_name, device_map='cuda', trust_remote_code=True)

tokenizer = AutoTokenizer.from_pretrained(model_name)
input_text = tokenizer.apply_chat_template(prompt, tokenize=False, add_generation_prompt=True)

inp = tokenizer(input_text, return_tensors="pt")
inp = {k: v.to('cuda') for k, v in inp.items()}
out = model.generate(**inp, 
                     max_length=256,
                     do_sample=True,
                     temperature=0.7,
                     top_p=0.7
                     )
text = tokenizer.decode(out[0], skip_special_tokens=True)
print(text)

Model Details

Developed by: mllmTeam
Model type: PhoneLM 0.5B models are auto-regressive language models based on the transformer decoder architecture.
Language(s): English
Paper: PhoneLM Technical Report
Library: PhoneLM

Model Architecture

The model is a decoder-only transformer architecture with the following modifications:

Hidden Size	Layers	Heads	Sequence Length
1024	24	16	2048

Position Embeddings: Rotary Position Embeddings (Su et al., 2021) applied to the first 25% of head embedding dimensions for improved throughput following Black et al. (2022). PhoneLM quantized the sin and cos values in Rotary Position Embeddings to 8-bit integers.
Normalization: LayerNorm (Ba et al., 2016) with learned bias terms as opposed to RMSNorm (Zhang & Sennrich, 2019).
Biases: We remove all bias terms from the feed-forward networks and multi-head self-attention layers, except for the biases of the query, key, and value projections (Bai et al., 2023).
ReLU Activation Function: ReLU(Glorot et al., 2011) activation functions are adopted in feed-forward networks.
Tokenizer: We use the SmolLM(Allal et al., 2024)'s tokenizer with a vocabulary size of 49,152.

License

This repository is released under the Apache-2.0 License.、

Citation

@misc{yi2024phonelmanefficientcapablesmall,
      title={PhoneLM:an Efficient and Capable Small Language Model Family through Principled Pre-training}, 
      author={Rongjie Yi and Xiang Li and Weikai Xie and Zhenyan Lu and Chenghua Wang and Ao Zhou and Shangguang Wang and Xiwen Zhang and Mengwei Xu},
      year={2024},
      eprint={2411.05046},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2411.05046}, 
}