Supervised Fine-Tuning (SFT)

Supervised Fine-Tuning (SFT)

Definition

Supervised fine-tuning (SFT) is the process of training a pre-trained model on labeled data for a specific task, adjusting all or part of its parameters.

Purpose

The purpose is to adapt general-purpose models to specialized tasks with improved accuracy.

Importance

  • Core technique in NLP and vision tasks.
  • Requires high-quality labeled data.
  • Risks overfitting with small datasets.
  • Often a precursor to RLHF.

How It Works

  1. Select a pre-trained model.
  2. Collect labeled data for the target task.
  3. Train the model with supervised learning.
  4. Validate on a held-out test set.
  5. Deploy and monitor for performance.

Examples (Real World)

  • GPT fine-tuned on customer service conversations.
  • BERT fine-tuned for named entity recognition.
  • Vision transformers fine-tuned on medical image classification.

References / Further Reading

Tell us how we can help with your next AI initiative.