Skip to content
/ vllm Public

A high-throughput and memory-efficient inference and serving engine for LLMs

docs.vllm.ai

License

Apache-2.0 license
49.3k stars 7.9k forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

vllm-project/vllm

Error
Looks like something went wrong!

About

A high-throughput and memory-efficient inference and serving engine for LLMs

docs.vllm.ai

Topics

amd cuda inference pytorch transformer llama gpt rocm model-serving tpu hpu mlops xpu llm inferentia llmops llm-serving qwen deepseek trainium

Resources

Readme

License

Apache-2.0 license

Code of conduct

Code of conduct

Security policy

Security policy
Activity
Custom properties

Stars

49.3k stars

Watchers

390 watching

Forks

7.9k forks
Report repository

Releases 64

v0.9.1 Latest
Jun 10, 2025
+ 63 releases

Sponsor this project

Learn more about GitHub Sponsors

Contributors 1,202

+ 1,188 contributors

Languages