vLLM
High-throughput LLM inference and serving engine with an OpenAI-compatible API
Overview
vLLM is the open-source inference and serving engine that grew out of UC Berkeley's PagedAttention research, now hosted by the PyTorch Foundation. It serves 200+ Hugging Face model architectures at production throughput via continuous batching and an OpenAI-compatible server, on NVIDIA, AMD and Intel GPUs plus a widening set of accelerator plugins.
Pricing
Pricing shown for reference only. These figures reflect RECATOOLS research as of 22 Jul 2026 and may be out of date or incomplete. This is not financial or purchasing advice — always confirm the current price on the provider’s official website before making any decision.
What you can produce with vLLM
- Serve an open-weight model behind an OpenAI-compatible endpoint with a single vllm serve command, so existing SDK code works after changing only the base URL.
- Batch-generate millions of tokens offline through the Python LLM class while continuous batching and PagedAttention keep the GPU saturated.
- Split a 70B-class model across GPUs with tensor parallelism, or across nodes with pipeline, data, expert and context parallelism.
- Cut VRAM by loading FP8, INT8/INT4, GPTQ, AWQ or GGUF quantized checkpoints straight from Hugging Face with no extra configuration.
- Serve dozens of fine-tunes from one base model with multi-LoRA support instead of running a separate deployment per adapter.
- Force valid JSON or schema-conformant output with xgrammar or guidance structured decoding, plus built-in tool-calling and reasoning parsers.
- Lower generation latency with speculative decoding — n-gram, suffix, EAGLE and DFlash draft strategies are built in.
ASEAN Perspective
vLLM in Southeast Asia
ASEAN-region availability and pricing notes coming soon. Drop the editorial team a note via /contact/ if you can supply local context (Singapore/Malaysia/Indonesia/Thailand/Vietnam).
vLLM is for serving large language models at production throughput on hardware you control. Its PagedAttention algorithm manages the KV cache like paged virtual memory. Continuous batching keeps the GPU busy across concurrent requests. The built-in server speaks the OpenAI API, the Anthropic Messages API, and gRPC, so existing clients connect without changes. It runs over 200 model architectures straight from Hugging Face, including dense LLMs, mixture-of-experts, hybrid state-space, multimodal, embedding and reward models. It also supports quantization from FP8 down to INT4 and speculative decoding for low-latency work.
It's a fit for ML platform teams self-hosting open-weight models, for companies swapping per-token API bills for their own GPUs, and for researchers who need large batched offline generation. Plan for significant GPU memory and some tuning time. The defaults assume a dedicated card, so fitting a model onto smaller hardware means working through context-length, batch-size, and quantization settings. A single user wanting one model on a laptop is better served by the llama.cpp family — vLLM earns its complexity when there are concurrent requests to batch.
Free and Apache-2.0 end to end, hosted by the PyTorch Foundation since May 2025, with no hosted or paid product of its own. Install from PyPI with pip or uv (CUDA wheels by default, Python 3.10–3.14), deploy the official vllm/vllm-openai Docker image, or build from source for the AMD, Intel, TPU and CPU backends. Releases land every one to two weeks; the latest, v0.25.1, shipped on 14 July 2026.
What people say
Three years after its introduction in a SOSP 2023 paper, the PagedAttention algorithm — and the vLLM library built to demonstrate it — has become the assumed answer for teams asking how to serve an LLM themselves. vLLM came out of UC Berkeley's Sky Computing Lab. It treats the KV cache like an operating system's virtual memory and uses continuous batching to keep the GPU from idling between requests. Users moving off raw Transformers report throughput gains measured in multiples, not percentages. The OpenAI-compatible server is the other half of the appeal — existing client code points at a new base URL and keeps working. Governance matured too: the PyTorch Foundation took vLLM in as a hosted project in May 2025, and the contributor count has passed two thousand.
The operational complaints are just as consistent. vLLM preallocates 90 percent of GPU memory by default to build its KV-cache pool, so the first experience on a smaller card is often a CUDA out-of-memory error at startup — common enough that the docs keep a dedicated Conserving Memory page and the issue tracker is thick with gpu_memory_utilization threads. The flag surface is large, and settings interact. Context length, batch size, quantization, parallelism, and compilation all trade against each other. Version churn stings as well — the project is still on 0.x, ships a release every week or two, and changelogs carry deprecations and backend removals. Startup drags for big models, since each tensor-parallel rank reads the full checkpoint before CUDA graphs are captured.
Benchmark shootouts keep it honest rather than untouchable: compiled TensorRT-LLM tends to win on NVIDIA hardware once the compilation cost is paid, and SGLang posts better numbers on mid-range cards in some tests. None of that has dented adoption. Ten releases landed between late April and mid-July 2026. Hardware plugins now reach TPUs, Gaudi, Ascend and Apple Silicon alongside the native NVIDIA, AMD and Intel paths, and a CPU backend exists for development work (though nobody, including the docs, positions it as the fast path). Much of the hosted-inference market quietly runs on it.
Summary of public user & expert reviews, compiled by RECATOOLS.
About this listing
This entry was compiled from publicly available data including vLLM's official website, press releases, documentation, and reputable third-party publications. RECATOOLS is not affiliated with vLLM unless explicitly stated.
Third-party AI tools update their pricing, features, availability, and policies frequently. Information here may be outdated by the time you read this — we make reasonable efforts to keep listings current, but cannot guarantee absolute accuracy.
For the latest details, please refer to vLLM directly →
Spotted something out of date? Suggest an update →
More in Code & Dev Tools
