Text Generation Inference
Hugging Face's production server for high-performance LLM serving.
Text Generation Inference (TGI) is Hugging Face's production-grade serving engine. It offers continuous batching, optimised kernels, quantisation, and tight integration with the Hugging Face model hub.
It covers much the same ground as other modern inference engines, and is a natural pick for teams already standardised on the Hugging Face ecosystem.
Where it fits
A fit for production self-hosted serving, especially for teams already deep in the Hugging Face ecosystem.
Where it does not fit
For a single-user local model it is heavier than a simple runtime, and vLLM has broader community momentum.
Alternatives to Text Generation Inference
The closest options on the map, and where each fits instead.
-
vLLM Inference engine
The right engine when you self-host a model under real concurrency and need to use GPU capacity efficiently.
-
SGLang Inference engine
A fit for high-throughput self-hosted serving, especially with structured output or heavy shared-prefix workloads.
Unlike Text Generation Inference Maturity: growing
Pairs with
Commonly used alongside it.
