UTMT Logo
UTMT Learn to Build AI Products
Home Projects Learning Curiosity Park

Deep Learning

Before starting to learn Deep Learning, make sure to first learn basics of Machine Learning

ANN basics Part 1 ANN basics Part 2 ANN basics Part 3 ANN basics Part 4 Advanced ANN Part 1 Advanced ANN Part 2 Advanced ANN Part 3 Advanced ANN Part 4 Advanced ANN Playlist Advanced ANN Part 5 Advanced ANN Part 6
Boosting Algorithms Part 1 Boosting Algorithms Part 2 Boosting Algorithms Part 3 Boosting Algorithms Part 4
Basic CNN Model Part 1 Basic CNN Model Part 2 Basic CNN Model Part 3 Basic CNN Model Part 4 Backprop in CNN
CNN Architectures Part 1
Autoencoders Part 1 GANs Part 1
Practical Considerations Part 1
RNN & LSTM Theory Part 1
RNN & LSTM Practical Part 1
Transformer Architecture and LLMs Part 1
LLM Pre-Training Part 1 LLM Pre-Training Part 2 LLM Pre-Training Part 3 LLM Pre-Training Part 4 LLM Pre-Training Part 5
Fine Tuning LLMs Part 1 Fine Tuning LLMs Part 2
RAG Part 1 RAG Part 2
Image Captioning, Text to Image Part 1 Image Captioning, Text to Image Part 2
Deployment Part 1

LLM Pre-Training Part 1

What You Will Learn

  • LLM pre-training overview

Content Covered

Introduction to concepts behind LLM pre-training.

Resources

  • Code for LLM Inference
  • Tokenization using BPE, Unigram and WordPiece
  • LLM Notebooks by Anish
  • Pre-Training and Fine-Tuning (Colab Link)
  • BERT MLM Task Details (Colab Link)
  • In-Context Learning (Colab Link)
  • Gen AI Hyperparameters (Colab Link)
  • LLM Model Details (Colab Link)
  • BERT
  • GPT-3 Overview
  • Annotated Transformer link
  • Annotated Transformer Video List link
  • Positional Encoding Blog (Kazemnejad) link
  • Positional Encoding Blog (TowardsDataScience Part I) link
  • Positional Encoding Blog (TowardsDataScience Part II) link
  • WordPiece Tokenization by HuggingFace

"Empowering builders to architect the neural future."

UTMT

An institution dedicated to the fusion of ancient wisdom and modern artificial intelligence, empowering the next generation of builders.

  • Contact Us
© 2024 UTMT SYSTEMS. ALL RIGHTS RESERVED.
Built for the Neural Age Version 4.0.2