inference-engine

6 projetos partilham este topic do GitHub

Linhas conectam membros que estão mensuravelmente relacionados entre si. O tamanho do ponto reflete estrelas.

🧬 Membros

Large-scale LLM inference engine

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.…

A highly optimized LLM inference acceleration engine for Llama and its variants.

Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over…

A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with…

PyTorch library for cost-effective, fast and easy serving of MoE models.

🔗 Familias relacionadas

cuda 72 inference 46 deepseek 88 glm 9 deepseek-r1 19 huggingface 50 llm-inference 15

Medido a partir dos tópicos do GitHub compartilhados por ambos os projetos, ponderado pela raridade de cada tópico.