LLMs from Scratch vs 
Annotated Paper Implementations
LLMs from Scratch vs Annotated Paper Implementations compared for 2026 — features, license, ease of use, performance and which one to choose. Build a GPT from nothing, line by line vs 60+ papers implemented and explained side by side.
Updated regularly · curated by OpenSourceAI.tech
LLMs from Scratch vs Annotated Paper Implementations at a glance
| Spec | LLMs from Scratch | Annotated Paper Implementations |
|---|---|---|
| Category | Learn AI & machine learning | Learn AI & machine learning |
| Type | Book + code | Reference implementations |
| License | Apache-2.0 | MIT |
| Runs locally | Yes | Yes |
| Primary language | Python | Python |
| Ease of use | Intermediate | Advanced |
| Best for | genuinely understanding how an LLM works | reading a paper and seeing exactly how it is built |
| GitHub stars | 99k | 67.1k |
How LLMs from Scratch and Annotated Paper Implementations score
| Criterion | LLMs from Scratch | Annotated Paper Implementations |
|---|---|---|
| Popularity | 4.5 | 4.5 |
| Maintenance | 5.0 | 4.0 |
| Ease of use | 3.5 | 2.5 |
| Privacy | 5.0 | 5.0 |
| License freedom | 5.0 | 5.0 |
Scores are computed automatically from public signals — GitHub stars (popularity), recent commit activity (maintenance), license type (freedom), local-first design (privacy) and onboarding complexity (ease of use). Indicative, not a verdict.
What each one is
LLMs from Scratch
Sebastian Raschka's companion repository to "Build a Large Language Model (From Scratch)": you implement attention, a transformer, pretraining and fine-tuning yourself, in plain PyTorch.
- You build every component yourself — no black boxes
- Runs on a laptop, no cluster needed
- The clearest explanation of attention anywhere
Annotated Paper Implementations
labml.ai's collection of deep learning papers implemented in PyTorch, with the explanation printed alongside the code — transformers, diffusion, RL, optimisers and more.
- Paper and code side by side, always in sync
- 60+ architectures, all runnable
- The fastest way to understand a new paper
Key differences
LLMs from Scratch is book + code, while Annotated Paper Implementations is reference implementations. Their licenses differ (Apache-2.0 vs MIT), which matters if you ship a commercial product. LLMs from Scratch leans more intermediate-friendly, whereas Annotated Paper Implementations is more suited to advanced users. In short, LLMs from Scratch fits genuinely understanding how an LLM works, and Annotated Paper Implementations fits reading a paper and seeing exactly how it is built.
Which should you choose?
Choose LLMs from Scratch for genuinely understanding how an LLM works. Choose Annotated Paper Implementations for reading a paper and seeing exactly how it is built.
There is rarely one winner — many setups use both. The right pick depends on your hardware, your team's skills, and whether you value simplicity or control.
Frequently asked questions
Is LLMs from Scratch or Annotated Paper Implementations easier to use?
LLMs from Scratch is generally the easier of the two to get started with, while Annotated Paper Implementations rewards more setup with more control.
Are LLMs from Scratch and Annotated Paper Implementations free?
LLMs from Scratch is free and open source (Apache-2.0), and Annotated Paper Implementations is free and open source (MIT). Neither charges for the core software.
Can I run LLMs from Scratch and Annotated Paper Implementations locally?
LLMs from Scratch: yes · Annotated Paper Implementations: yes. Both can be used without sending your data to a third-party cloud where their setup allows.
LLMs from Scratch vs Annotated Paper Implementations — which should I pick in 2026?
Choose LLMs from Scratch for genuinely understanding how an LLM works. Choose Annotated Paper Implementations for reading a paper and seeing exactly how it is built.
People also compare
Explore more open-source AI
Browse thousands of open-source AI tools, models and projects — all curated in one place, updated daily.
Explore the directory →