Supervised Fine-Tuning (SFT)

Supervised Fine-Tuning (SFT)

Definition

Supervised fine-tuning (SFT) is the process of training a pre-trained model on labeled data for a specific task, adjusting all or part of its parameters.

Purpose

The purpose is to adapt general-purpose models to specialized tasks with improved accuracy.

Importance

  • Core technique in NLP and vision tasks.
  • Requires high-quality labeled data.
  • Risks overfitting with small datasets.
  • Often a precursor to RLHF.

How It Works

  1. Select a pre-trained model.
  2. Collect labeled data for the target task.
  3. Train the model with supervised learning.
  4. Validate on a held-out test set.
  5. Deploy and monitor for performance.

Examples (Real World)

  • GPT fine-tuned on customer service conversations.
  • BERT fine-tuned for named entity recognition.
  • Vision transformers fine-tuned on medical image classification.

References / Further Reading

  • Devlin et al. “BERT: Pre-training of Deep Bidirectional Transformers.” NAACL 2019.
  • Hugging Face Transformers Documentation.
  • Stanford CS224N: NLP with Deep Learning.