Instructions to use deepseek-ai/DeepSeek-V4-Flash-0731 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-V4-Flash-0731 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="deepseek-ai/DeepSeek-V4-Flash-0731")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V4-Flash-0731")
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V4-Flash-0731", device_map="auto")

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use deepseek-ai/DeepSeek-V4-Flash-0731 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-V4-Flash-0731"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Flash-0731",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-V4-Flash-0731

SGLang

How to use deepseek-ai/DeepSeek-V4-Flash-0731 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-V4-Flash-0731" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Flash-0731",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-V4-Flash-0731" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Flash-0731",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-V4-Flash-0731 with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-V4-Flash-0731
```

Going completely mad

by xiezirock - opened 13 days ago

Discussion

xiezirock

13 days ago

Going completely mad

alexanderacuna

13 days ago

Absolutely insane benchmarks.

qaqwebs

13 days ago

•

edited 13 days ago

True Arts!!!!!

afrehshw

13 days ago

INSANE

Aklus

13 days ago

First date, and it feels like a match made in heaven! Great job Deepseek team!

panlilu

13 days ago

The benchmark score of this model fully aligns with the hands-on coding experience, with no exaggeration whatsoever. Totally INSANE model in such a small size.

BuiDoan

13 days ago

Mind-blowing benchmark results.

rombodawg

13 days ago

Anthropic: Here is a model so powerful, the government had to shut it down. Its locked behind an expensive subscription

OpenAI: Here is a model so powerful it escaped and hacked an entire organization. We arent releasing it yet.

Deepseek-AI:
Here you go, is free

aaron-newsome

12 days ago

i'm usually not one to give much thought to benchmarks but holy crap. these benchmark numbers are absolutely WILD for a model this size. my daily driver right now is glm-5.2 on a local rig. i thought i was set for the entire year, but the glm-5.2 has literally been running 3 weeks. now I'm going to have to give this deepseek a try. i'll say the benchmarks made me do it.

Recandle

12 days ago

Is this the beginning of the small model era?

amarosnithe

12 days ago

It cannot be stressed either how this model is costing 9 CENTS on OpenRouter rn (DeepInfra, the cheapest provider serving this model). Which is schizophrenic pricing, what the hell is going on

Lolmaobruhhh

11 days ago

Is this the beginning of the small model era?

that's not an slm dpsk with 284 b parameters moe active 13 b parameters u can call it llm ...but if there was something called mlm u could call it that ...

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment