Let’s Build a Transformer!
Cradle
A transformer, built one piece at a time in C#.
Explore the seriesThe series
0 of 18 lessons published
Start here · Prologue
Prologue: Getting the cradle ready
Set up the tools, build the starting program, and check the dependencies we’ll use in the first lesson.
Published ·
Latest: Prologue: Getting the cradle ready
- The cradle Planned
Run our first kernel on the GPU.
- Buffers and shapes Planned
Give the data a shape we can work with.
- Matrix multiplication Planned
Build the operation at the heart of the model.
- Softmax Planned
Turn scores into a stable distribution.
The approach
One program. Room to grow.
We begin with a kernel running on the GPU. Each lesson adds an operation we can inspect and test, until the same C# program loads model weights and generates text.
About the project