Fine-tuning LLMs Guide (Unsloth Documentation)
Unsloth AI
End-to-end documentation for fine-tuning open models: choosing a base model, LoRA versus QLoRA, dataset formatting, hyperparameter selection, running training in free Colab or Kaggle notebooks, evaluating results, and exporting to GGUF or vLLM.
More resources on Fine-Tuning LLMs
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Derives a closed-form mapping between reward models and optimal policies, replacing the reward-model-plus-PPO pipeline of RLHF with a single classification loss over preference pairs. The reference behind TRL's DPOTrainer and its ORPO and KTO successors.
QLoRA: Efficient Finetuning of Quantized LLMs
Dettmers and colleagues combine 4-bit NormalFloat quantisation, double quantisation and paged optimisers with LoRA adapters to fine-tune a 65B model on one 48GB GPU. The technical basis for most consumer-hardware fine-tuning stacks in use today. Explains why a 4-bit base model can be fine-tuned without quality collapse, which is the assumption every consumer-GPU tutorial silently relies on. Also contains a still-useful data-quality result: small curated datasets beat large scraped ones.
LoRA: Low-Rank Adaptation of Large Language Models
Foundational and short. Reading it once removes the guesswork behind rank, alpha and target-module settings that the tutorials present as folklore. Age is irrelevant here — the field's own notation is downstream of this paper. The paper introducing low-rank adaptation: freezing pretrained weights and training rank-decomposition matrices injected into each layer. Reduces GPT-3 175B trainable parameters roughly ten-thousand-fold and adds no inference latency, unlike earlier adapter methods.
TRL: Transformer Reinforcement Learning Documentation
Reference documentation for the library implementing SFT, DPO, KTO, ORPO, GRPO, reward modelling and distillation trainers. Includes dataset format specifications, memory-reduction and distributed-training how-tos, and PEFT, vLLM and DeepSpeed integration guides.
Hugging Face smol-course: Fine-Tuning Language Models
Hands-on post-training course built around SmolLM3. Units cover instruction tuning, evaluation, preference alignment, vision-language models and reinforcement learning, each with runnable TRL notebooks sized for a single consumer GPU or free Colab. The only free, structured, start-to-finish course dedicated specifically to fine-tuning rather than to LLMs generally. Actively maintained with new units, uses current TRL APIs, and every exercise runs on hardware a learner already has.