Text Generation Inference (TGI)

产品描述

Hugging Face's TGI is a toolkit for deploying and serving LLMs in production with optimizations for performance and scalability (now in maintenance mode, popular for existing setups).

核心功能

• Production serving
• Quantization support
• Multi-backend
• Easy deployment
• HF integration