Supervised Fine-Tuning (SFT)

Definition

Supervised fine-tuning (SFT) is the process of training a pre-trained model on labeled data for a specific task, adjusting all or part of its parameters.

Purpose

The purpose is to adapt general-purpose models to specialized tasks with improved accuracy.

Importance

• Core technique in NLP and vision tasks.
• Requires high-quality labeled data.
• Risks overfitting with small datasets.
• Often a precursor to RLHF.

How It Works

1. Select a pre-trained model.
2. Collect labeled data for the target task.
3. Train the model with supervised learning.
4. Validate on a held-out test set.
5. Deploy and monitor for performance.

Examples (Real World)

• GPT fine-tuned on customer service conversations.
• BERT fine-tuned for named entity recognition.
• Vision transformers fine-tuned on medical image classification.

References / Further Reading

• Devlin et al. “BERT: Pre-training of Deep Bidirectional Transformers.” NAACL 2019.
• Hugging Face Transformers Documentation.
• Stanford CS224N: NLP with Deep Learning.
• What is SFT? Why is It Important?