Deep-Dive FTDD-05 — Axolotl: Config-Driven Production Fine-Tuning

Course: Course 3 — LLM Fine-Tuning Masterclass Module: FTDD-05 — Axolotl Duration: 45 minutes Level: Senior Engineer and above Prerequisites: FT00 (The Steering Stack), FT11 (The Training Loop), FTDD-04 (TRL)

Learning Objectives

After completing this module, you will be able to:

  1. Explain the declarative YAML-config pattern and why config-as-source-of-truth is the production advantage over hand-written training scripts.
  2. Describe how Axolotl relates to TRL (it wraps the same trainers) and what the wrapper adds: opinionated defaults and multi-GPU orchestration.
  3. Choose between Axolotl, Unsloth, and raw TRL given a concrete constraint (multi-GPU production vs single-GPU speed vs full control/research).
  4. Read and write an Axolotl YAML config, identifying the model, data, PEFT, distributed, and eval sections.

FTDD-05.1 — The Config-Driven Pattern

The real lesson of Axolotl is not the tool — it is the pattern. The training recipe becomes a declarative file. Everything good about Axolotl flows from that.

Config as the source of truth

Here is the core idea, and it is older than Axolotl: the training recipe should be a declarative file, not a program. A YAML (or JSON) that says what to train, not how to write the loop. Axolotl's contribution is to apply this pattern thoroughly and with opinionated, cross-family defaults that work — so thoroughly that for most practitioners "an Axolotl config" is synonymous with "a reproducible fine-tuning run."

The payoff is three properties that hand-written training scripts cannot match:

  1. Reproducibility is a file. The entire recipe — base model, dataset and split, LoRA rank and targets, learning rate and scheduler, batch size and gradient accumulation, precision, distributed strategy, eval cadence — lives in one YAML. "Reproduce my run" is "run this file." A colleague (or your future self, in six months) does not need to reverse-engineer a Python script.
  2. Recipes are diff-able and version-able. Because the recipe is a text file, it goes in git, and a git diff shows exactly what changed between two runs. "We raised the learning rate and added DeepSpeed ZeRO-3" is a two-line diff, not a code review of a 300-line script.
  3. CI-validatable. A YAML can be schema-validated, linted, and dry-run-checked in CI before it ever touches a GPU. A Python training script cannot. This is the same reason Kubernetes manifests are YAML, not Python: declarative config is auditable in a way imperative code is not.

Why this pattern recurs

You will see this pattern three times in these deep-dives: Axolotl's YAML, the TRL CLI's YAML (trl sft --config), and the broader "infra-as-config" movement. They are the same idea at different altitudes. Axolotl was the tool that proved it for LLM fine-tuning; the TRL v1.0 CLI adopted it; and now the expectation across the ecosystem is that a fine-tuning job is described by a file.

The anti-pattern it replaces: the 200-line train.py with hardcoded paths, ad-hoc hyperparameters, and a comment that says # TODO: clean up. That script runs once, on one machine, and is unreproducible a quarter later. The YAML outlives the script every time.

FTDD-05.2 — Under the Hood: Axolotl Wraps TRL

Axolotl is not a competing training engine. It is a config layer and orchestration layer over TRL. Knowing this makes the whole stack legible.

The same trainers, configured

This is the fact that prevents Axolotl from being a black box: underneath the YAML, Axolotl calls TRL's trainers. When you write an SFT config in Axolotl, the resulting run uses SFTTrainer. A DPO config uses DPOTrainer. A GRPO config uses GRPOTrainer. The optimizer, the loss function, the checkpointing — all TRL's, which is to say all the HuggingFace Trainer's (FTDD-04's thin-wrapper principle).

This matters for two reasons. First, it means your TRL knowledge transfers directly: the hyperparameters and concepts are the same, Axolotl just names and organizes them in YAML. Second, it means Axolotl inherits TRL's distributed story (DDP, DeepSpeed ZeRO, FSDP) for free — and then adds curation on top.

What the wrapper actually adds

If Axolotl is "just" TRL under the hood, what justifies the wrapper? Two things, and they are substantial:

  1. Opinionated, cross-family defaults. TRL gives you the trainer and the knobs; it does not tell you which knobs to turn for a Llama 3.1 vs a Mistral vs a Qwen. Axolotl ships defaults that work across model families — sensible sequence lengths, gradient accumulation, precision, and PEFT configs that are known-good. You spend your time on your data and your goal, not on re-deriving a working baseline.
  2. Battle-tested multi-GPU orchestration. This is the decisive one. Raw TRL gives you the ability to use FSDP or DeepSpeed, but it hands you the responsibility of writing the FSDP/DeepSpeed config — and those configs are notoriously fiddly (sharding dim, CPU offload, the interaction with gradient checkpointing, ZeRO stage selection). Axolotl curates these into known-good recipes. For a 2-, 4-, or 8-GPU job on a 70B model, Axolotl's multi-GPU path is the path of least resistance in the open-weights world.

The cost of the abstraction

Axolotl is a layer of indirection, and indirection has a price. The first is latency on the bleeding edge: when a new trainer lands in TRL, it takes time to surface cleanly in Axolotl's config schema. If you need a method released in TRL yesterday, raw TRL gets you there first. The second is debuggability: when something breaks, you are debugging through Axolotl → TRL → Transformers, and the error may surface far from its cause. For standard recipes this is rare; for exotic setups it is real.

FTDD-05.3 — Axolotl vs Unsloth vs Raw TRL

The three-way decision every fine-tuning practitioner makes. It is a decision by constraint, not by preference.

The three tools, by what they optimize

You now have three tools that overlap heavily. The decision is governed by your constraint, not by which tool is "best":

Axolotl Unsloth Raw TRL
Relationship to TRL Wraps it (config + orchestration) Replaces kernels, TRL-compatible API Is the substrate
Best at Production, multi-GPU, reproducibility Single-GPU speed, sub-30B, low VRAM Full control, research, freshest methods
Multi-GPU Strong (curated FSDP/DeepSpeed) Limited, OOM-prone Possible but you write the config
Config YAML (declarative) Python (code) YAML CLI or Python API
Speed Solid; abstraction overhead ~2x faster, ~half VRAM (Triton kernels) Baseline
Freshness Trails TRL slightly Trails TRL more Newest methods first

The decision rule

The hybrid pattern many practitioners use

A common real-world setup: Unsloth as the default backend for sub-30B single-GPU work, Axolotl for anything multi-GPU or above 30B. This is not indecision — it is using each tool for its strength. Unsloth's TRL-compatible API means the transition is mostly a config change, not a rewrite.

The honest summary: Axolotl, Unsloth, and TRL are not competitors in the way their marketing sometimes implies. They are a stack. TRL is the engine; Unsloth is a faster engine for one regime; Axolotl is the cockpit for multi-GPU production. Knowing which to reach for is the skill this module teaches.

FTDD-05.4 — Reading and Writing an Axolotl Config

The pattern made concrete. A real YAML, annotated.

Anatomy of an Axolotl SFT config

Here is a representative Axolotl config for a LoRA SFT job. Read it as five sections: model, data, PEFT, hyperparameters, distributed.

# --- 1. MODEL ---
base_model: meta-llama/Llama-3.1-8B
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

# --- 2. DATA ---
datasets:
  - path: HuggingFaceTB/smoltalk
    type: chat_template
    split: train
val_set_size: 0.02            # hold out 2% for eval

# --- 3. PEFT (adapter settings) ---
adapter: qlora                # qlora | lora | none (full)
lora_r: 16
lora_alpha: 32
lora_dropout: 0.05
lora_target_modules:          # which weights to steer
  - q_proj
  - k_proj
  - v_proj
  - o_proj

# --- 4. TRAINING HYPERPARAMETERS (PIN THESE) ---
sequence_len: 4096
sample_packing: true          # pack short sequences for throughput
micro_batch_size: 2           # per-device
gradient_accumulation_steps: 8  # effective batch = 2 * 8 * num_gpus
num_epochs: 3
learning_rate: 2e-4           # PINNED — never rely on a default
lr_scheduler: cosine
warmup_steps: 50
bf16: auto                    # auto-detect; A100/H100 yes, older no

# --- 5. DISTRIBUTED + OUTPUT ---
deepspeed: deepspeed_configs/zero2.json   # curated; or fsdp: [...]
output_dir: ./out/llama31-8b-smoltalk
save_steps: 200
eval_steps: 200

What to notice

The reproducibility checklist

Before you treat a config as production-ready:

If you can answer yes to all six, your run is reproducible. That is the bar Axolotl was built to clear.

Anti-Patterns

Treating the config as documentation, not source

Writing a YAML, then "tweaking it live" by passing CLI overrides or editing the loaded model in Python, so the on-disk config no longer matches what ran. The config must be the complete source of truth, or it loses its reproducibility guarantee. If you override, write the override into the YAML and commit it.

Reaching for Axolotl when you need a custom reward

Axolotl is configured, not programmed. A GRPO job with a custom reward function (Python callable) cannot be fully expressed in YAML — you will fight the tool. Drop to raw TRL for non-standard loops. Use Axolotl for the standard recipes it excels at.

Choosing the tool by familiarity, not constraint

"I know Axolotl, so I'll use Axolotl for this single-GPU 7B job" leaves throughput on the table — Unsloth would be ~2x faster. "I know Unsloth, so I'll force this 8-GPU 70B job onto Unsloth" leads to OOM pain. The tool is chosen by the constraint (multi-GPU vs single-GPU speed vs control), not by what you happen to know today.

Assuming the wrapper removes the need to understand TRL

Because Axolotl wraps TRL, a TRL-level problem (a bad learning rate, the wrong loss, a misunderstood trainer) is still your problem. The wrapper configures the trainer; it does not absolve you of knowing what the trainer does. Learn TRL first (FTDD-04); Axolotl is the layer above.

Key Terms

Term Definition
Axolotl A config-driven, multi-GPU-capable production wrapper over TRL; YAML configs are the source of truth
Config-as-source-of-truth The pattern: the training recipe is a declarative YAML, not a program — making it reproducible, diff-able, and CI-validatable
Opinionated defaults Axolotl's known-good settings across model families (sequence lengths, grad accumulation, PEFT configs) that save you re-deriving a baseline
Multi-GPU orchestration Axolotl's curated FSDP / DeepSpeed ZeRO configs — the decisive advantage over raw TRL for cluster jobs
The hybrid pattern Unsloth for sub-30B single-GPU speed, Axolotl for multi-GPU / >30B — using each tool for its strength
adapter: qlora / lora / none The Axolotl config switch between QLoRA, LoRA, and full fine-tuning — one line, not a rewrite

Lab Exercise

See 07-lab-spec.md. The "YAML to trained model" lab: write an Axolotl SFT config for a 1B model, run it, then mutate the config (LoRA on/off, batch size) and observe how a single declarative change propagates through the run. Feel the config-as-source-of-truth pattern.

References

  1. Axolotl — OpenAXolotl project documentation and config reference. github.com/OpenAccess-AI-Collective/axolotl
  2. Course 3, FTDD-04 — TRL. The substrate Axolotl wraps; read this first.
  3. Course 3, FTDD-03 — Unsloth. The single-GPU kernel-replacement alternative.
  4. Course 3, FT11 — The Training Loop. Where the role of TRL/Axolotl in the stack is introduced.
  5. Course 3, FT08–FT09 — LoRA / QLoRA / DoRA. The PEFT methods the adapter: config selects.
  6. DeepSpeed — ZeRO Optimizer. microsoft.github.io/DeepSpeed. The sharding strategy Axolotl curates.
  7. PyTorch FSDP — Fully Sharded Data Parallel. The alternative sharding strategy Axolotl exposes.
# Deep-Dive FTDD-05 — Axolotl: Config-Driven Production Fine-Tuning

**Course**: Course 3 — LLM Fine-Tuning Masterclass
**Module**: FTDD-05 — Axolotl
**Duration**: 45 minutes
**Level**: Senior Engineer and above
**Prerequisites**: FT00 (The Steering Stack), FT11 (The Training Loop), FTDD-04 (TRL)

---

## Learning Objectives

After completing this module, you will be able to:

1. Explain the declarative YAML-config pattern and why config-as-source-of-truth is the production advantage over hand-written training scripts.
2. Describe how Axolotl relates to TRL (it wraps the same trainers) and what the wrapper adds: opinionated defaults and multi-GPU orchestration.
3. Choose between Axolotl, Unsloth, and raw TRL given a concrete constraint (multi-GPU production vs single-GPU speed vs full control/research).
4. Read and write an Axolotl YAML config, identifying the model, data, PEFT, distributed, and eval sections.

---

# FTDD-05.1 — The Config-Driven Pattern

*The real lesson of Axolotl is not the tool — it is the pattern. The training recipe becomes a declarative file. Everything good about Axolotl flows from that.*

## Config as the source of truth

Here is the core idea, and it is older than Axolotl: **the training recipe should be a declarative file, not a program.** A YAML (or JSON) that says *what* to train, not *how* to write the loop. Axolotl's contribution is to apply this pattern thoroughly and with opinionated, cross-family defaults that work — so thoroughly that for most practitioners "an Axolotl config" is synonymous with "a reproducible fine-tuning run."

The payoff is three properties that hand-written training scripts cannot match:

1. **Reproducibility is a file.** The entire recipe — base model, dataset and split, LoRA rank and targets, learning rate and scheduler, batch size and gradient accumulation, precision, distributed strategy, eval cadence — lives in one YAML. "Reproduce my run" is "run this file." A colleague (or your future self, in six months) does not need to reverse-engineer a Python script.
2. **Recipes are diff-able and version-able.** Because the recipe is a text file, it goes in git, and a `git diff` shows exactly what changed between two runs. "We raised the learning rate and added DeepSpeed ZeRO-3" is a two-line diff, not a code review of a 300-line script.
3. **CI-validatable.** A YAML can be schema-validated, linted, and dry-run-checked in CI before it ever touches a GPU. A Python training script cannot. This is the same reason Kubernetes manifests are YAML, not Python: declarative config is auditable in a way imperative code is not.

### Why this pattern recurs

You will see this pattern three times in these deep-dives: Axolotl's YAML, the TRL CLI's YAML (`trl sft --config`), and the broader "infra-as-config" movement. They are the same idea at different altitudes. Axolotl was the tool that proved it for LLM fine-tuning; the TRL v1.0 CLI adopted it; and now the expectation across the ecosystem is that a fine-tuning job is described by a file.

> **The anti-pattern it replaces:** the 200-line `train.py` with hardcoded paths, ad-hoc hyperparameters, and a comment that says `# TODO: clean up`. That script runs once, on one machine, and is unreproducible a quarter later. The YAML outlives the script every time.

---

# FTDD-05.2 — Under the Hood: Axolotl Wraps TRL

*Axolotl is not a competing training engine. It is a config layer and orchestration layer over TRL. Knowing this makes the whole stack legible.*

## The same trainers, configured

This is the fact that prevents Axolotl from being a black box: **underneath the YAML, Axolotl calls TRL's trainers.** When you write an SFT config in Axolotl, the resulting run uses `SFTTrainer`. A DPO config uses `DPOTrainer`. A GRPO config uses `GRPOTrainer`. The optimizer, the loss function, the checkpointing — all TRL's, which is to say all the HuggingFace `Trainer`'s (FTDD-04's thin-wrapper principle).

This matters for two reasons. First, it means your TRL knowledge transfers directly: the hyperparameters and concepts are the same, Axolotl just names and organizes them in YAML. Second, it means Axolotl inherits TRL's distributed story (DDP, DeepSpeed ZeRO, FSDP) for free — and then adds curation on top.

### What the wrapper actually adds

If Axolotl is "just" TRL under the hood, what justifies the wrapper? Two things, and they are substantial:

1. **Opinionated, cross-family defaults.** TRL gives you the trainer and the knobs; it does not tell you which knobs to turn for a Llama 3.1 vs a Mistral vs a Qwen. Axolotl ships defaults that work across model families — sensible sequence lengths, gradient accumulation, precision, and PEFT configs that are known-good. You spend your time on *your* data and *your* goal, not on re-deriving a working baseline.
2. **Battle-tested multi-GPU orchestration.** This is the decisive one. Raw TRL gives you the *ability* to use FSDP or DeepSpeed, but it hands you the *responsibility* of writing the FSDP/DeepSpeed config — and those configs are notoriously fiddly (sharding dim, CPU offload, the interaction with gradient checkpointing, ZeRO stage selection). Axolotl curates these into known-good recipes. For a 2-, 4-, or 8-GPU job on a 70B model, Axolotl's multi-GPU path is the path of least resistance in the open-weights world.

### The cost of the abstraction

Axolotl is a layer of indirection, and indirection has a price. The first is latency on the bleeding edge: when a new trainer lands in TRL, it takes time to surface cleanly in Axolotl's config schema. If you need a method released in TRL yesterday, raw TRL gets you there first. The second is debuggability: when something breaks, you are debugging through Axolotl → TRL → Transformers, and the error may surface far from its cause. For standard recipes this is rare; for exotic setups it is real.

---

# FTDD-05.3 — Axolotl vs Unsloth vs Raw TRL

*The three-way decision every fine-tuning practitioner makes. It is a decision by constraint, not by preference.*

## The three tools, by what they optimize

You now have three tools that overlap heavily. The decision is governed by your constraint, not by which tool is "best":

| | **Axolotl** | **Unsloth** | **Raw TRL** |
| --- | --- | --- | --- |
| Relationship to TRL | Wraps it (config + orchestration) | Replaces kernels, TRL-compatible API | Is the substrate |
| Best at | Production, **multi-GPU**, reproducibility | **Single-GPU speed**, sub-30B, low VRAM | **Full control**, research, freshest methods |
| Multi-GPU | Strong (curated FSDP/DeepSpeed) | Limited, OOM-prone | Possible but you write the config |
| Config | YAML (declarative) | Python (code) | YAML CLI or Python API |
| Speed | Solid; abstraction overhead | ~2x faster, ~half VRAM (Triton kernels) | Baseline |
| Freshness | Trails TRL slightly | Trails TRL more | Newest methods first |

### The decision rule

- **Multi-GPU production job, must reproduce, standard recipe (SFT/DPO/GRPO)?** Axolotl. The multi-GPU orchestration and config-as-source-of-truth are decisive.
- **Single GPU, sub-30B, speed or VRAM constrained?** Unsloth. The Triton kernels give you roughly double the throughput at half the memory — often the difference between "fits on my 4090" and "doesn't."
- **Custom reward function, non-standard loop, brand-new method, or full control?** Raw TRL (Python API). The wrappers are configured, not programmed; the moment your loop is non-standard you are in raw TRL anyway.
- **Standard recipe, zero code, no wrapper dependency?** Raw TRL CLI (`trl sft --config`). The v1.0 Stability Contract means this is increasingly viable for production without Axolotl's orchestration layer.

### The hybrid pattern many practitioners use

A common real-world setup: **Unsloth as the default backend for sub-30B single-GPU work, Axolotl for anything multi-GPU or above 30B.** This is not indecision — it is using each tool for its strength. Unsloth's TRL-compatible API means the transition is mostly a config change, not a rewrite.

> **The honest summary:** Axolotl, Unsloth, and TRL are not competitors in the way their marketing sometimes implies. They are a stack. TRL is the engine; Unsloth is a faster engine for one regime; Axolotl is the cockpit for multi-GPU production. Knowing which to reach for is the skill this module teaches.

---

# FTDD-05.4 — Reading and Writing an Axolotl Config

*The pattern made concrete. A real YAML, annotated.*

## Anatomy of an Axolotl SFT config

Here is a representative Axolotl config for a LoRA SFT job. Read it as five sections: model, data, PEFT, hyperparameters, distributed.

```yaml
# --- 1. MODEL ---
base_model: meta-llama/Llama-3.1-8B
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

# --- 2. DATA ---
datasets:
  - path: HuggingFaceTB/smoltalk
    type: chat_template
    split: train
val_set_size: 0.02            # hold out 2% for eval

# --- 3. PEFT (adapter settings) ---
adapter: qlora                # qlora | lora | none (full)
lora_r: 16
lora_alpha: 32
lora_dropout: 0.05
lora_target_modules:          # which weights to steer
  - q_proj
  - k_proj
  - v_proj
  - o_proj

# --- 4. TRAINING HYPERPARAMETERS (PIN THESE) ---
sequence_len: 4096
sample_packing: true          # pack short sequences for throughput
micro_batch_size: 2           # per-device
gradient_accumulation_steps: 8  # effective batch = 2 * 8 * num_gpus
num_epochs: 3
learning_rate: 2e-4           # PINNED — never rely on a default
lr_scheduler: cosine
warmup_steps: 50
bf16: auto                    # auto-detect; A100/H100 yes, older no

# --- 5. DISTRIBUTED + OUTPUT ---
deepspeed: deepspeed_configs/zero2.json   # curated; or fsdp: [...]
output_dir: ./out/llama31-8b-smoltalk
save_steps: 200
eval_steps: 200
```

### What to notice

- **Everything is explicit.** There are no "magic" defaults you are trusting blind. `learning_rate`, `lora_r`, `sequence_len`, `gradient_accumulation_steps` — all pinned. This is the reproducibility property in action.
- **PEFT is a section, not a script.** Switching from QLoRA to full fine-tuning is changing `adapter: qlora` to `adapter: none` — one line. Switching LoRA rank is changing `lora_r`. The declarative pattern means *the change is localized to the config*, not scattered through code.
- **Multi-GPU is a config value.** `deepspeed: deepspeed_configs/zero2.json` (or an `fsdp:` block) is how you express "shard this across GPUs." Axolotl's curated configs are the value-add over raw TRL — you are not hand-writing the ZeRO config.
- **Eval is first-class.** `val_set_size`, `eval_steps` — the recipe declares evaluation, it does not bolt it on. This is part of why Axolotl configs are trusted in production: the eval cadence is in the source of truth.

### The reproducibility checklist

Before you treat a config as production-ready:

- [ ] Is every hyperparameter pinned (no reliance on an unspecified default)?
- [ ] Are the dataset path and split explicit (not "whatever the default split is")?
- [ ] Is the distributed strategy declared (single-GPU, FSDP, or a named DeepSpeed config)?
- [ ] Is the precision explicit (`bf16: auto` is fine, but know what it resolved to)?
- [ ] Is eval declared (`val_set_size` / `eval_steps`)?
- [ ] Could a colleague reproduce this from the YAML alone, with no access to you?

If you can answer yes to all six, your run is reproducible. That is the bar Axolotl was built to clear.

---

## Anti-Patterns

### Treating the config as documentation, not source

Writing a YAML, then "tweaking it live" by passing CLI overrides or editing the loaded model in Python, so the on-disk config no longer matches what ran. The config must be the *complete* source of truth, or it loses its reproducibility guarantee. If you override, write the override into the YAML and commit it.

### Reaching for Axolotl when you need a custom reward

Axolotl is configured, not programmed. A GRPO job with a custom reward function (Python callable) cannot be fully expressed in YAML — you will fight the tool. Drop to raw TRL for non-standard loops. Use Axolotl for the standard recipes it excels at.

### Choosing the tool by familiarity, not constraint

"I know Axolotl, so I'll use Axolotl for this single-GPU 7B job" leaves throughput on the table — Unsloth would be ~2x faster. "I know Unsloth, so I'll force this 8-GPU 70B job onto Unsloth" leads to OOM pain. The tool is chosen by the constraint (multi-GPU vs single-GPU speed vs control), not by what you happen to know today.

### Assuming the wrapper removes the need to understand TRL

Because Axolotl wraps TRL, a TRL-level problem (a bad learning rate, the wrong loss, a misunderstood trainer) is still your problem. The wrapper configures the trainer; it does not absolve you of knowing what the trainer does. Learn TRL first (FTDD-04); Axolotl is the layer above.

---

## Key Terms

| Term | Definition |
| --- | --- |
| **Axolotl** | A config-driven, multi-GPU-capable production wrapper over TRL; YAML configs are the source of truth |
| **Config-as-source-of-truth** | The pattern: the training recipe is a declarative YAML, not a program — making it reproducible, diff-able, and CI-validatable |
| **Opinionated defaults** | Axolotl's known-good settings across model families (sequence lengths, grad accumulation, PEFT configs) that save you re-deriving a baseline |
| **Multi-GPU orchestration** | Axolotl's curated FSDP / DeepSpeed ZeRO configs — the decisive advantage over raw TRL for cluster jobs |
| **The hybrid pattern** | Unsloth for sub-30B single-GPU speed, Axolotl for multi-GPU / >30B — using each tool for its strength |
| **`adapter: qlora / lora / none`** | The Axolotl config switch between QLoRA, LoRA, and full fine-tuning — one line, not a rewrite |

---

## Lab Exercise

See `07-lab-spec.md`. The "YAML to trained model" lab: write an Axolotl SFT config for a 1B model, run it, then mutate the config (LoRA on/off, batch size) and observe how a single declarative change propagates through the run. Feel the config-as-source-of-truth pattern.

---

## References

1. **Axolotl** — OpenAXolotl project documentation and config reference. github.com/OpenAccess-AI-Collective/axolotl
2. **Course 3, FTDD-04** — *TRL*. The substrate Axolotl wraps; read this first.
3. **Course 3, FTDD-03** — *Unsloth*. The single-GPU kernel-replacement alternative.
4. **Course 3, FT11** — *The Training Loop*. Where the role of TRL/Axolotl in the stack is introduced.
5. **Course 3, FT08–FT09** — *LoRA / QLoRA / DoRA*. The PEFT methods the `adapter:` config selects.
6. **DeepSpeed** — *ZeRO Optimizer*. microsoft.github.io/DeepSpeed. The sharding strategy Axolotl curates.
7. **PyTorch FSDP** — Fully Sharded Data Parallel. The alternative sharding strategy Axolotl exposes.