Under The Mango Tree

Home

English Learning Python Algorithms Machine Learning Deep Learning & AI Soft Skills AI Tools

Job Portal Projects Hackathons Team

Home

English Learning Python Algorithms Machine Learning Deep Learning & AI Soft Skills

Job Portal Team

Deep Learning

Before starting to learn Deep Learning, make sure to first learn basics of Machine Learning

ANN basics Part 1 ANN basics Part 2 ANN basics Part 3 ANN basics Part 4 Advanced ANN Part 1 Advanced ANN Part 2 Advanced ANN Part 3 Advanced ANN Part 4 Advanced ANN Playlist Advanced ANN Part 5 Advanced ANN Part 6

Boosting Algorithms Part 1 Boosting Algorithms Part 2 Boosting Algorithms Part 3 Boosting Algorithms Part 4

Basic CNN Model Part 1 Basic CNN Model Part 2 Basic CNN Model Part 3 Basic CNN Model Part 4 Backprop in CNN

CNN Architectures Part 1

Autoencoders Part 1 GANs Part 1

Practical Considerations Part 1

RNN & LSTM Theory Part 1

RNN & LSTM Practical Part 1

Transformer Architecture and LLMs Part 1

LLM Pre-Training Part 1 LLM Pre-Training Part 2 LLM Pre-Training Part 3 LLM Pre-Training Part 4 LLM Pre-Training Part 5

Fine Tuning LLMs Part 1 Fine Tuning LLMs Part 2

RAG Part 1 RAG Part 2

Image Captioning, Text to Image Part 1 Image Captioning, Text to Image Part 2

Deployment Part 1

LLM Pre-Training Part 1

Previous Next Lesson

What You Will Learn

LLM pre-training overview

Content Covered

Introduction to concepts behind LLM pre-training.

Resources

Code for LLM Inference
Tokenization using BPE, Unigram and WordPiece
LLM Notebooks by Anish
Pre-Training and Fine-Tuning (Colab Link)
BERT MLM Task Details (Colab Link)
In-Context Learning (Colab Link)
Gen AI Hyperparameters (Colab Link)
LLM Model Details (Colab Link)
BERT
GPT-3 Overview
Annotated Transformer link
Annotated Transformer Video List link
Positional Encoding Blog (Kazemnejad) link
Positional Encoding Blog (TowardsDataScience Part I) link
Positional Encoding Blog (TowardsDataScience Part II) link
WordPiece Tokenization by HuggingFace

Under The Mango Tree

Empowering students through quality education and career opportunities in a supportive, university-style environment.

Quick Links

Home
Learning Hub
Job Portal
Projects
Hackathons
Our Team

Learning Resources

English Learning
Python Programming
Algorithms
Machine Learning
Deep Learning & AI
Soft Skills
AI Tools

Contact Us

Bhopal, Madhya Pradesh.
info@pythonpal.org
+91 9891262133

© 2025 Under The Mango Tree. All rights reserved.