Fine-Tuning Small Models: When and Why to Fine-Tune vs. Prompt Engineer

The phrase “fine-tune your model” sounds like the kind of thing that requires a PhD, a cluster of A100s, and six months of runway. In practice, you can meaningfully fine-tune a 7B or 8B parameter model on a consumer GPU in an afternoon, using open-source tooling, and deploy the result on the same laptop you use for everything else.

The harder question isn’t “how do I fine-tune?” — it’s “should I fine-tune, and what for?” Fine-tuning is frequently the wrong tool. It’s also sometimes the only tool that works. Understanding the difference saves a lot of wasted effort.

This guide covers the decision framework first, then the mechanics: LoRA/QLoRA, dataset preparation, training with Unsloth and Axolotl, evaluation, and deployment with Ollama.

Fine-Tuning vs. Prompt Engineering: The Decision

What Prompt Engineering Can Do

Modern large models — Llama 3, Qwen 2.5, Mistral, Gemma 2 — are remarkably capable with nothing but careful prompting. Before considering fine-tuning, exhaust these approaches:

System prompts: A detailed system prompt that defines the persona, constraints, output format, and examples covers the majority of customization needs.

Few-shot examples: Including 3–10 examples of desired input/output pairs in the prompt often achieves near-fine-tuned quality for structured tasks.

Structured output / JSON mode: Most modern models with tool use or JSON mode reliably produce structured output from a schema.

RAG (Retrieval-Augmented Generation): For injecting domain knowledge, retrieving relevant documents at query time is almost always better than baking knowledge into weights.

If the above approaches get you 80% of the way there with acceptable latency and cost, stop. You don’t need fine-tuning.

When Fine-Tuning Actually Wins

Fine-tuning is the right choice in a specific set of scenarios:

1. Style and voice consistency at scale You need every output to sound like it was written by the same author — a specific brand voice, a company’s documentation style, a particular coding style guide. Few-shot examples in context drift over long conversations; a fine-tuned model carries the style into every response by default.

2. Highly structured domain-specific output Your application requires a very specific output format that vanilla models get wrong even with prompting — a custom JSON schema with dozens of fields, a domain-specific markup language, a particular code style with project-specific conventions. Fine-tuning teaches the format once rather than re-teaching it every call.

3. Specialization for a narrow task on a small model You need a 1B or 3B parameter model to perform a task that normally requires a 7B+ model. Fine-tuning a small model on a focused dataset often closes most of the gap, at a fraction of the inference cost.

4. Removing context overhead If you’re spending 2,000 tokens on system prompt and few-shot examples for every call, fine-tuning bakes that in and reduces token usage by 90%. At scale this matters a lot.

5. Teaching genuinely new knowledge (carefully) Models don’t know your internal APIs, your proprietary data formats, your company’s terminology. Fine-tuning can teach this — but RAG is usually better for facts that change, while fine-tuning is better for reasoning patterns and structural knowledge that’s stable.

6. Latency-sensitive edge deployments You need a model running on a Raspberry Pi, a phone, or an embedded device. Fine-tuning a 1B or 3B model to specialize in your task is the path to acceptable quality at those sizes.

When Fine-Tuning Doesn’t Help (and Hurts)

General reasoning and knowledge: Fine-tuning a 7B model on 500 customer support tickets will not teach it to reason better. It might make it more likely to produce responses that look like your customer support tickets. Those are different things.

Hallucination reduction: Fine-tuning often makes hallucinations different, not fewer. If factual accuracy is the goal, RAG or structured tool use is more reliable.

Tasks requiring diverse capabilities: Fine-tuning for a narrow task degrades performance on other tasks. A heavily fine-tuned coding model loses general conversation ability. If you need both, you’re probably better served by a larger base model.

Small datasets (< 50–100 examples): Fine-tuning on very small datasets tends to overfit badly. You’ll get a model that memorized your examples rather than learned from them. With < 100 examples, few-shot prompting almost always wins.

How LoRA and QLoRA Work

Training all the weights of even a 7B model requires more GPU RAM than most people have access to. A 7B model in full precision (fp32) is 28 GB of weights alone, before you account for gradients and optimizer state.

LoRA (Low-Rank Adaptation) solves this by only training a small set of additional weights rather than modifying the original model weights at all.

The insight: weight updates during fine-tuning have low intrinsic rank. Instead of updating a full weight matrix W (shape: d×k, potentially millions of parameters), LoRA decomposes the update as two small matrices: W + ΔW = W + BA, where B is d×r and A is r×k, and r (the rank) is much smaller than d or k. For r=8 and a 4096×4096 weight matrix, you’re training 65,536 parameters instead of 16,777,216 — a 256× reduction.

The original model weights are frozen. You only train the LoRA adapter matrices. At inference time, you can either merge them into the original weights (for deployment) or load them separately (for flexibility, switching between adapters).

QLoRA adds quantization: the frozen base model is loaded in 4-bit precision using bitsandbytes NF4 quantization, while the LoRA adapters train in bf16. A 7B model in 4-bit takes ~5–6 GB of GPU RAM instead of 14+ GB in fp16. This makes fine-tuning practical on a single 8–12 GB consumer GPU (RTX 3080, RTX 4070, etc.) and on Apple Silicon Macs with 16+ GB unified memory.

LoRA Hyperparameters

For most practical fine-tuning, r=16, lora_alpha=32, target_modules="all-linear" is a solid starting point. Higher rank gives more capacity at the cost of memory and risk of overfitting.

Preparing Your Dataset

Dataset quality is the single most important factor in fine-tuning success. A good 200-example dataset beats a mediocre 2,000-example one.

Format: Instruction Tuning

The standard format for fine-tuning chat/instruction models is a list of conversations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

[
  {
    "conversations": [
      {"role": "system", "content": "You are a helpful DevOps assistant..."},
      {"role": "user", "content": "How do I check which process is using port 8080?"},
      {"role": "assistant", "content": "Use `lsof` or `ss`:\n\n```bash\nlsof -i :8080\n# or\nss -tlnp | grep 8080\n```\n\nThe output shows the PID and process name holding the port."}
    ]
  },
  ...
]

For simpler instruction-response tasks, a flat format works:

1
2
3
4
5
6
7

[
  {
    "instruction": "Convert this Python function to use async/await",
    "input": "def fetch_data(url):\n    response = requests.get(url)\n    return response.json()",
    "output": "async def fetch_data(url):\n    async with aiohttp.ClientSession() as session:\n        async with session.get(url) as response:\n            return await response.json()"
  }
]

Generating Training Data

Writing hundreds of examples by hand is tedious. Better approaches:

Mine your existing corpus: If you have Slack exports, documentation, code review comments, or customer support tickets, these are your training data. Extract input/output pairs that demonstrate the behavior you want.

Use a large model to generate synthetic data: Prompt GPT-4, Claude, or a large local model to generate training examples from a smaller set of seeds. This works surprisingly well — you’re essentially distilling the larger model’s behavior into your smaller one.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

# generate_training_data.py — use Ollama to create synthetic training examples
import json
import ollama

SEED_EXAMPLES = [
    {"task": "explain a kubernetes concept", "topic": "Pod"},
    {"task": "explain a kubernetes concept", "topic": "Service"},
    {"task": "explain a kubernetes concept", "topic": "Ingress"},
    # add more seeds...
]

GENERATION_PROMPT = """Generate a realistic question someone might ask about {topic} in the context of {task}.
Then provide a thorough, accurate answer a senior DevOps engineer would give.

Output valid JSON with this structure:
{{"question": "...", "answer": "..."}}

Only output the JSON, nothing else."""

examples = []
for seed in SEED_EXAMPLES:
    response = ollama.chat(
        model="qwen2.5:14b",
        messages=[{
            "role": "user",
            "content": GENERATION_PROMPT.format(**seed)
        }]
    )
    try:
        data = json.loads(response["message"]["content"])
        examples.append({
            "conversations": [
                {"role": "user", "content": data["question"]},
                {"role": "assistant", "content": data["answer"]}
            ]
        })
    except json.JSONDecodeError:
        pass  # skip malformed outputs

with open("training_data.json", "w") as f:
    json.dump(examples, f, indent=2)

print(f"Generated {len(examples)} examples")

Quality filtering: After generation, filter aggressively. Remove examples shorter than ~100 characters in the output, examples where the model clearly hallucinated, and near-duplicates. A clean dataset of 300 examples outperforms a noisy dataset of 3,000.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# filter_dataset.py
import json

def is_quality_example(example: dict) -> bool:
    convs = example.get("conversations", [])
    if len(convs) < 2:
        return False
    assistant_turn = next((c for c in convs if c["role"] == "assistant"), None)
    if not assistant_turn:
        return False
    output = assistant_turn["content"]
    if len(output) < 100:          # too short
        return False
    if output.count("I cannot") > 2:  # refusal-heavy
        return False
    return True

with open("training_data.json") as f:
    raw = json.load(f)

filtered = [ex for ex in raw if is_quality_example(ex)]
print(f"Kept {len(filtered)}/{len(raw)} examples after filtering")

with open("training_data_filtered.json", "w") as f:
    json.dump(filtered, f, indent=2)

Dataset Size Guidelines

For most homelab/personal use cases, 200–1,000 high-quality examples is plenty.

Fine-Tuning with Unsloth

Unsloth is the fastest open-source fine-tuning library, achieving 2–5× speedup over raw Hugging Face training with 60–80% less GPU memory usage. It’s the recommended starting point for most fine-tuning work.

Installation

1
2
3
4
5
6

# Create a fresh environment
python -m venv .venv && source .venv/bin/activate

# Install Unsloth (CUDA 12.1 example)
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install --no-deps "trl<0.9.0" peft accelerate bitsandbytes

For CPU-only or Apple Silicon:

1
2

pip install unsloth
pip install trl peft accelerate

Training Script

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105

#!/usr/bin/env python3
# train.py — fine-tune with Unsloth + QLoRA

from unsloth import FastLanguageModel
from trl import SFTTrainer
from transformers import TrainingArguments
from datasets import Dataset
import json
import torch

# ── Configuration ──────────────────────────────────────────
MODEL_NAME   = "unsloth/llama-3.2-3b-instruct-bnb-4bit"  # or llama-3.1-8b, qwen2.5-7b, etc.
OUTPUT_DIR   = "./outputs/my-fine-tune"
DATA_PATH    = "./training_data_filtered.json"
MAX_SEQ_LEN  = 2048
LORA_RANK    = 16
BATCH_SIZE   = 2
GRAD_ACCUM   = 4   # effective batch = BATCH_SIZE * GRAD_ACCUM = 8
EPOCHS       = 3
LR           = 2e-4

# ── Load base model ────────────────────────────────────────
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=MODEL_NAME,
    max_seq_length=MAX_SEQ_LEN,
    dtype=None,        # auto-detect (bf16 if supported)
    load_in_4bit=True, # QLoRA
)

# ── Apply LoRA ─────────────────────────────────────────────
model = FastLanguageModel.get_peft_model(
    model,
    r=LORA_RANK,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=LORA_RANK * 2,
    lora_dropout=0.05,
    bias="none",
    use_gradient_checkpointing="unsloth",  # memory-efficient
    random_state=42,
)

print(f"Trainable parameters: {model.num_parameters(only_trainable=True):,}")
print(f"Total parameters:     {model.num_parameters():,}")

# ── Load dataset ───────────────────────────────────────────
with open(DATA_PATH) as f:
    raw_data = json.load(f)

def format_example(example):
    """Format conversation as a single training string."""
    messages = example["conversations"]
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=False
    )
    return {"text": text}

dataset = Dataset.from_list(raw_data).map(format_example)
print(f"Training on {len(dataset)} examples")

# ── Training arguments ─────────────────────────────────────
training_args = TrainingArguments(
    output_dir=OUTPUT_DIR,
    num_train_epochs=EPOCHS,
    per_device_train_batch_size=BATCH_SIZE,
    gradient_accumulation_steps=GRAD_ACCUM,
    warmup_ratio=0.05,
    learning_rate=LR,
    lr_scheduler_type="cosine",
    fp16=not torch.cuda.is_bf16_supported(),
    bf16=torch.cuda.is_bf16_supported(),
    logging_steps=10,
    save_strategy="epoch",
    optim="adamw_8bit",
    weight_decay=0.01,
    report_to="none",  # set to "wandb" if you want tracking
    seed=42,
)

# ── Train ──────────────────────────────────────────────────
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=MAX_SEQ_LEN,
    args=training_args,
)

trainer.train()

# ── Save ───────────────────────────────────────────────────
model.save_pretrained(OUTPUT_DIR + "/lora")
tokenizer.save_pretrained(OUTPUT_DIR + "/lora")

# Merge and save as full model (for Ollama deployment)
model.save_pretrained_merged(
    OUTPUT_DIR + "/merged",
    tokenizer,
    save_method="merged_16bit",  # or "merged_4bit" for smaller size
)

print(f"\nTraining complete. Model saved to {OUTPUT_DIR}")

Run it:

1

python train.py

Training 3 epochs on 500 examples with Llama 3.2 3B on an RTX 3080 (10 GB VRAM) takes about 20 minutes. On an M2 MacBook Pro (16 GB), expect 45–60 minutes via Metal.

Memory Requirements by Model

For homelab fine-tuning, 3B and 7B/8B models are the sweet spot. The 3B models train fast and are usable on very modest hardware; 7B/8B models offer significantly better quality and still fit in consumer GPUs.

Fine-Tuning with Axolotl

Axolotl is more configurable than Unsloth and better for production pipelines. It’s YAML-driven and supports more dataset formats, training strategies, and model architectures out of the box.

Installation

1
2
3

git clone https://github.com/OpenAccess-AI-Collective/axolotl
cd axolotl
pip install -e ".[flash-attn,deepspeed]"

Configuration File

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

# config.yaml
base_model: meta-llama/Llama-3.2-3B-Instruct

model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

load_in_4bit: true
strict: false

datasets:
  - path: training_data_filtered.json
    type: sharegpt          # use 'alpaca' for instruction/input/output format
    conversation: chatml

dataset_prepared_path: ./data_prepared
val_set_size: 0.05          # 5% for validation

output_dir: ./outputs/axolotl-run

sequence_len: 2048
sample_packing: true        # pack short examples together for efficiency

# LoRA config
adapter: lora
lora_r: 16
lora_alpha: 32
lora_dropout: 0.05
lora_target_modules:
  - q_proj
  - v_proj
  - k_proj
  - o_proj
  - gate_proj
  - up_proj
  - down_proj

# Training
num_epochs: 3
micro_batch_size: 2
gradient_accumulation_steps: 4
learning_rate: 0.0002
lr_scheduler: cosine
warmup_ratio: 0.05
optimizer: adamw_bnb_8bit
weight_decay: 0.01

# Logging
logging_steps: 10
eval_steps: 0.1             # evaluate every 10% of training
save_strategy: epoch

# Precision
bf16: auto
fp16: false
tf32: false

Run training:

1

accelerate launch -m axolotl.cli.train config.yaml

Monitor training:

1
2

# In another terminal
tail -f outputs/axolotl-run/trainer_log.jsonl | jq .loss

Evaluating Your Fine-Tuned Model

Before deploying, verify the fine-tuning actually helped and didn’t break things.

Quick Manual Evaluation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

# evaluate.py — compare base model vs fine-tuned
from unsloth import FastLanguageModel
import torch

TEST_PROMPTS = [
    "Explain the difference between a Kubernetes Deployment and a StatefulSet.",
    "Write a bash function that retries a command up to 3 times with exponential backoff.",
    "What's the best way to handle secrets in a Docker Compose file?",
    # Add prompts specific to your fine-tuning domain
]

def load_and_test(model_path: str, label: str):
    model, tokenizer = FastLanguageModel.from_pretrained(
        model_name=model_path,
        max_seq_length=2048,
        load_in_4bit=True,
    )
    FastLanguageModel.for_inference(model)

    print(f"\n{'='*60}")
    print(f"Model: {label}")
    print('='*60)

    for prompt in TEST_PROMPTS:
        messages = [{"role": "user", "content": prompt}]
        text = tokenizer.apply_chat_template(
            messages, tokenize=False, add_generation_prompt=True
        )
        inputs = tokenizer([text], return_tensors="pt").to("cuda")

        with torch.inference_mode():
            outputs = model.generate(
                **inputs,
                max_new_tokens=300,
                temperature=0.1,
                do_sample=True,
            )

        response = tokenizer.decode(
            outputs[0][inputs["input_ids"].shape[1]:],
            skip_special_tokens=True
        )
        print(f"\nPrompt: {prompt[:80]}...")
        print(f"Response: {response[:400]}...")

# Compare
load_and_test("unsloth/llama-3.2-3b-instruct-bnb-4bit", "Base model")
load_and_test("./outputs/my-fine-tune/lora", "Fine-tuned")

Watching for Overfitting

In your training logs, watch the validation loss alongside training loss:

Epoch 1: train_loss=1.42, eval_loss=1.38
Epoch 2: train_loss=1.21, eval_loss=1.25  ← eval_loss higher than train: early sign
Epoch 3: train_loss=1.05, eval_loss=1.43  ← overfitting — stop here or use epoch 2

If eval loss diverges upward while train loss keeps falling, you’re overfitting. Use the checkpoint from the epoch with the lowest eval loss. This is why save_strategy: "epoch" matters.

For small datasets (< 500 examples), 1–2 epochs often produces better generalization than 3–5.

Task-Specific Metrics

For structured output tasks, write a simple precision/recall evaluator:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

import json
from pathlib import Path

def evaluate_json_output(model, tokenizer, test_file: str):
    with open(test_file) as f:
        test_cases = json.load(f)

    correct = 0
    for case in test_cases:
        # Generate output
        output = generate(model, tokenizer, case["input"])
        try:
            parsed = json.loads(output)
            if parsed == case["expected"]:
                correct += 1
        except json.JSONDecodeError:
            pass  # malformed JSON = wrong

    accuracy = correct / len(test_cases)
    print(f"JSON accuracy: {accuracy:.1%} ({correct}/{len(test_cases)})")
    return accuracy

Deploying with Ollama

Once you have a merged model (the base weights + LoRA adapter combined), you can run it locally with Ollama using a Modelfile.

Convert to GGUF

Ollama uses GGUF format. Convert your merged model:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# Install llama.cpp for conversion
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
pip install -r requirements.txt

# Convert merged model to GGUF
python convert_hf_to_gguf.py \
  /path/to/outputs/my-fine-tune/merged \
  --outfile /path/to/my-model.gguf \
  --outtype q8_0  # 8-bit quantization — good balance of size/quality

# Or quantize more aggressively
./llama-quantize /path/to/my-model.gguf /path/to/my-model-q4.gguf Q4_K_M

Quantization levels:

• Q8_0 — best quality, larger file (~7 GB for 7B)
• Q5_K_M — excellent quality, smaller (~5 GB)
• Q4_K_M — good quality, smaller still (~4 GB) — recommended default
• Q3_K_M — acceptable quality, very small (~3 GB)

Create an Ollama Modelfile

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# Modelfile
FROM /path/to/my-model-q4.gguf

# System prompt baked in — no need to repeat it every call
SYSTEM """
You are a DevOps assistant specializing in Kubernetes, Linux, and cloud infrastructure.
You provide concise, accurate, practical answers with working code examples.
When writing shell commands, use modern syntax and add brief inline comments.
"""

PARAMETER temperature 0.1
PARAMETER top_p 0.9
PARAMETER num_ctx 4096

Register with Ollama:

1
2

ollama create my-devops-assistant -f Modelfile
ollama run my-devops-assistant

Test it:

1

ollama run my-devops-assistant "How do I debug a CrashLoopBackOff?"

Serving via API

Once registered, Ollama exposes the model via its standard API:

1
2
3
4
5

curl http://localhost:11434/api/chat -d '{
  "model": "my-devops-assistant",
  "messages": [{"role": "user", "content": "Explain PersistentVolumeClaims"}],
  "stream": false
}'

Any app already using Ollama can swap in your fine-tuned model by name.

Real-World Use Cases and Realistic Expectations

Use Case 1: Internal Documentation Assistant

Goal: A model that answers questions using your company’s internal APIs, idioms, and terminology — with the right tone and style.

Approach: Fine-tune a 7B model on 500–1,000 examples extracted from your internal docs, Confluence pages, and Slack threads where experts answered questions. The model learns both the vocabulary and the preferred answer style.

What you get: Faster, more accurate answers for common questions. Still hallucinates on edge cases — pair with RAG for factual accuracy.

Use Case 2: Code Style Enforcement

Goal: A coding assistant that generates code matching your team’s exact style (naming conventions, error handling patterns, logging format).

Approach: Extract 200–500 before/after pairs from your code reviews — what was submitted vs. the reviewed version. Fine-tune on these pairs.

What you get: Code completions that fit your codebase. Works best for consistent, rule-based style rather than architectural judgment.

Use Case 3: Structured Data Extraction

Goal: A model that reads unstructured text (log lines, support tickets, emails) and outputs a specific JSON schema.

Approach: Create 300–500 input/output examples with your target schema. A fine-tuned 3B model often matches or exceeds a prompted 7B model for this task.

What you get: Fast, cheap, accurate extraction. Very reliable JSON output. Fragile to inputs that are very different from training examples.

What to Expect Honestly

Fine-tuning a small model on a specialized task produces a model that:

• Is noticeably better than the base model on in-distribution examples
• Behaves more consistently and predictably
• Is faster and cheaper to run than larger prompted models
• Still fails on out-of-distribution inputs
• Can degrade on tasks far from its training distribution

It does not produce a model that:

• Is smarter or more capable in any fundamental sense
• Reliably handles situations it hasn’t seen
• Is immune to hallucination
• Approaches GPT-4/Claude-class quality on complex reasoning

Fine-tuning specializes; it doesn’t upgrade. Keep that expectation clear and you’ll find the right problems to apply it to.

The Full Workflow in Summary

1. Define the task precisely
   └── "I need a model that converts SQL queries to our
        internal query builder syntax"

2. Exhaust prompting first
   └── Can a system prompt + few-shot examples do it well enough?
   └── If yes: stop here

3. Build dataset
   ├── Mine existing examples from your codebase/docs
   ├── Generate synthetic data with a large model
   └── Filter aggressively for quality

4. Fine-tune with Unsloth
   ├── Start with 3B model + QLoRA + r=16
   ├── Train 2–3 epochs, watch eval loss
   └── Save best checkpoint (lowest eval loss)

5. Evaluate
   ├── Manual comparison with base model on test prompts
   └── Automated accuracy on held-out test set

6. Deploy with Ollama
   ├── Convert to GGUF (Q4_K_M for balance)
   ├── Register with Modelfile
   └── Serve via Ollama API

7. Iterate
   └── More data, better data, or different hyperparameters
       if quality isn't there

The whole cycle from dataset to deployed model can be done in a weekend. The bottleneck is almost always dataset quality, not the training process.