gguf

9 projects share this GitHub topic

Lines connect members that are measurably related to each other. Dot size reflects stars.

🧬 Members

Hundreds of models & providers. One command to find what runs on your hardware.

Distribute and run LLMs with a single file.

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No…

动手学Ollama，CPU玩转大模型部署，在线阅读地址：https://datawhalechina.github.io/handy-ollam…

Go manage your Ollama models

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA,…

Interface for OuteTTS models.

Llama 3+ inference in pure Java

A desktop app for running Large Language Models locally.

🔗 Related families

mlx 13 llama-cpp 5 local-ai 5 api-server 5 command-line-tool 5 developer-tools 50 langchain 77 ggml 12

Measured from GitHub topics shared by both projects, weighted by how rare each topic is.