The AI Engineering Program: Module 5 of 13
Deep Learning & Transformers from Scratch
Train neural networks in PyTorch and build a working mini-GPT, so LLMs stop being a black box.
28guided hours
10lessons
Onlineself-paced
Lessons
Every lesson: deep notes, curated lectures, an animated explainer, a hands-on lab, practice questions and a timed exam.
- PyTorch tensors and autograd
- The training loop: batches, loss, optimiser, GPU in Colab
- Multilayer networks, overfitting and regularisation
- Convolutional networks and transfer learning
- Tokenisation and embeddings
- Attention, built by hand
- The transformer block
- Build and train a mini-GPT on Indian-language text
- Pretrained models from the Hugging Face Hub: pipelines and sentence embeddings
- Module project
Tools introduced: PyTorch, Hugging Face
Module project
A from-scratch mini-GPT trained in Colab plus a sentence-embedding semantic search demo, with a write-up of what each component does.