inference-engine

6 projects share this GitHub topic

Lines connect members that are measurably related to each other. Dot size reflects stars.

🧬 Members

Large-scale LLM inference engine

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.…

A highly optimized LLM inference acceleration engine for Llama and its variants.

Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over…

A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with…

PyTorch library for cost-effective, fast and easy serving of MoE models.

🔗 Related families

cuda 72 inference 46 deepseek 88 glm 9 deepseek-r1 19 huggingface 50 llm-inference 15

Measured from GitHub topics shared by both projects, weighted by how rare each topic is.