inference-engine

6 progetti condividono questo topic GitHub

Le linee collegano membri che sono misurabilmente correlati tra loro. La dimensione dei punti riflette le stelle.

🧬 Membri

Large-scale LLM inference engine

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.…

A highly optimized LLM inference acceleration engine for Llama and its variants.

Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over…

A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with…

PyTorch library for cost-effective, fast and easy serving of MoE models.

🔗 Famiglie affini

cuda 72 inference 46 deepseek 88 glm 9 deepseek-r1 19 huggingface 50 llm-inference 15

Misurato dai temi di GitHub condivisi da entrambi i progetti, ponderato in base a quanto è raro ciascun tema.