LLMs from Scratch

Build a GPT from nothing, line by line

Learn AI & machine learningBook + codeApache-2.0Runs locallyPythonIntermediate

OSAI Pulse ⓘ ★★★★★★★★★★ /100 signals tracked

🐳 Docker pulls 📦 PyPI downloads / month 📦 npm downloads / month 🚀 Latest release ·

Visit website →View on GitHub

What is LLMs from Scratch?

Sebastian Raschka's companion repository to "Build a Large Language Model (From Scratch)": you implement attention, a transformer, pretraining and fine-tuning yourself, in plain PyTorch.

Why people choose LLMs from Scratch

You build every component yourself — no black boxes
Runs on a laptop, no cluster needed
The clearest explanation of attention anywhere

LLMs from Scratch at a glance

Category	Learn AI & machine learning
Type	Book + code
License	Apache-2.0
Runs locally	Yes
Built with	Python
Skill level	Intermediate
Best for	genuinely understanding how an LLM works

Open-source alternatives to LLMs from Scratch

Other open-source learn ai & machine learning tools worth comparing:

VirgilioA structured mentor for data science and ML

ML for BeginnersMicrosoft's classic machine learning course

AI for BeginnersFrom symbolic AI to neural networks

Generative AI for BeginnersBuild generative AI apps, lesson by lesson

Data Science for BeginnersThe data foundations before any ML

LLM CourseThe reference roadmap for learning LLMs

Neural Networks: Zero to HeroKarpathy builds backprop, then GPT, from scratch

Annotated Paper Implementations60+ papers implemented and explained side by side

Dive into Deep LearningThe textbook where every equation is runnable

fastai Book (fastbook)Deep learning for coders, top-down

Hands-On Machine LearningThe notebooks of the best-selling ML book

Hugging Face CourseMaster transformers with the actual library

Hugging Face Agents CourseBuild AI agents, properly

Made With MLFrom notebook to production system

Prompt Engineering GuideThe reference on prompting, backed by papers

Applied MLHow real companies actually ship ML

ML YouTube CoursesThe best free ML courses on YouTube, curated

OpenAI CookbookPractical recipes that work with any OpenAI-compatible API

Awesome Machine LearningThe reference index of ML libraries, by language

Awesome LLMPapers, models and tools of the LLM era

Deep Learning DrizzleUniversity lectures, from the source

ML Interviews BookWhat ML interviews actually ask

LLMs from Scratch head-to-head

LLMs from Scratch vs Virgilio LLMs from Scratch vs ML for Beginners LLMs from Scratch vs AI for Beginners LLMs from Scratch vs Generative AI for Beginners LLMs from Scratch vs Data Science for Beginners LLMs from Scratch vs LLM Course LLMs from Scratch vs Neural Networks: Zero to Hero LLMs from Scratch vs Annotated Paper Implementations LLMs from Scratch vs Dive into Deep Learning LLMs from Scratch vs fastai Book (fastbook)LLMs from Scratch vs Hands-On Machine Learning LLMs from Scratch vs Hugging Face Course LLMs from Scratch vs Hugging Face Agents Course LLMs from Scratch vs Made With ML LLMs from Scratch vs Prompt Engineering Guide LLMs from Scratch vs Applied ML LLMs from Scratch vs ML YouTube Courses LLMs from Scratch vs OpenAI Cookbook LLMs from Scratch vs Awesome Machine Learning LLMs from Scratch vs Awesome LLM LLMs from Scratch vs Deep Learning Drizzle LLMs from Scratch vs ML Interviews Book

FAQ

Is LLMs from Scratch free?

LLMs from Scratch is free and open-source (Apache-2.0 license), so you can use, self-host and modify it at no cost.

Can I run LLMs from Scratch locally?

Yes. LLMs from Scratch is designed to run on your own machine or server, keeping your data private.

What is the best alternative to LLMs from Scratch?

Popular open-source alternatives include Virgilio, ML for Beginners, AI for Beginners. See the comparisons above to choose.

Explore more open-source AI

Browse the full directory of open-source AI tools, models and projects — updated daily.

Browse all tools →