AI Disruption

AI Disruption

Share this post

AI Disruption
AI Disruption
Pre-Training Large Models: How Pre-Training Makes Models Smarter

Pre-Training Large Models: How Pre-Training Makes Models Smarter

Learn how pre-training, parameter initialization, forward propagation, and loss functions optimize neural networks for efficient data classification.

Meng Li's avatar
Meng Li
Aug 01, 2024
∙ Paid

Share this post

AI Disruption
AI Disruption
Pre-Training Large Models: How Pre-Training Makes Models Smarter
1
Share

Welcome to the "Practical Application of AI Large Language Model Systems" Series

Table of Contents

Table of Contents

Meng Li
·
June 7, 2024
Read full story

Last class, I introduced the internal structure of the model. To understand it better, we reviewed the model's implementation principles. I mentioned that the training process involves continually adjusting weights. More precisely, it also includes adjusting biases. So, training involves constantly adjusting weights and biases using backpropagation, loss functions, etc.

We didn't dive into details before, but in this class, we'll go through the pre-training process with a simple example.

We'll use a three-layer neural network model for data classification.

This model takes two input variables: study time and sleep time, and predicts whether a student will pass an exam.

We'll follow the usual model training steps, but won't repeat previously covered content.

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Meng Li
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share