Skip to main content
Crowdfunding
Python + AI for Geeks
Practice

Differences Between Fine-tuning and General Training

Fine-tuning is the process of adjusting an already trained AI model to fit new data.

Instead of training an AI model from scratch, you use what the existing model has already learned and make slight adjustments for a new purpose.


Why Use Fine-tuning?​

Training an AI model from scratch requires vast amounts of data, lengthy periods of time, and significant computing resources.

However, if you have a well-trained model already, it's much more efficient to quickly adapt it to a new task using fine-tuning.

For instance, a smart model like GPT, which understands various languages, can be easily transformed into an AI capable of solving legal issues by training it on various case laws.


How is General Training Different from Fine-tuning?​

General training starts from a completely uninitialized state.

All weights and biases are set randomly, and the model is trained by gradually adjusting these values using data.

Note: Weights and biases together are called the model's parameters.


In contrast, fine-tuning begins with a pre-trained model that already has learned weights and biases.

It retains the existing knowledge and makes selective or slight adjustments to the weights and biases to fit new data.


The Technical Principle of Fine-tuning​

Internally, the model is composed of several layers, each containing weights and bias values.

These values are modified through training and have a direct impact on prediction accuracy.

Fine-tuning works in the following way:

  1. Load an Existing Model: Use a pre-trained model that has been trained on a large dataset.

  2. Adjust Some or All Layers: Keep some layers intact while adjusting the parameters of a few, or make slight adjustments to all layers.

  3. Train on New Data: Update the existing model using new data from fields like healthcare or law.


Fine-tuning is one of the key techniques, alongside prompt engineering, to use AI more efficiently.

In the next lesson, we will explore the differences between fine-tuning and prompt engineering.

Want to learn more?

Join CodeFriends Plus membership or enroll in a course to start your journey.