Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
machine-learning transformers inference pytorch artificial-intelligence glm whisper gemma mlx kimi openai-api llm diffusers llamacpp llama-cpp vllm qwen sglang kimi-k3 glm-5-3
-
Updated
Nov 13, 2026 - Python