The Decision Flowchart & License Reality

The decision tree, from the top

Did you build a strong prompt baseline? If no — go do that first. Measure success on at least 50 representative inputs.
Does the task need fresh or external knowledge? If yes — RAG. Stop here.
Do you need consistent behavior, format, or style? If no — keep iterating on the prompt and few-shot examples. Stop here.
Do you have at least 50 high-quality examples? If no — collect or synthesize them first (Track 2). Stop here until you do.
Yes to all of the above? Fine-tune. Pick OpenAI managed (Track 3) for speed, or open-source PEFT (Tracks 4–5) for control.

Evaluation gates before training

Defined success metrics. Pick at most three.
Tested the base model with careful prompting. Record the baseline number.
Tested with few-shot examples. Adding 3–5 examples often closes the gap; if so, do not fine-tune.
Documented the gap. "Baseline 78%, few-shot 84%, target 95% → 11-point gap to close." Now you have a brief.

Licensing reality (2025–2026)

Open-weights does not mean fully unrestricted. Read the license before you ship.

Family	License	Practical commercial reality
Llama 3 / 4 (Meta)	Llama Community License	Free for commercial use under 700M MAU. Above that, contact Meta.
Mistral / Mixtral	Apache 2.0	Fully open. No restrictions.
Gemma 2 / 3 (Google)	Gemma Terms	Permissive with a usage policy on harmful content. Gemma 4 moved to Apache 2.0.
Qwen 3 (Alibaba)	Apache 2.0	Fully open. No restrictions.
Phi (Microsoft)	MIT	Fully open. No restrictions.

And ask: is your training data licensed for this use? Privacy concerns? Could the model reproduce copyrighted material at inference? Fine-tuning does not magically launder data rights.

Code

A baseline-vs-few-shot evaluation harness·python

import json
from openai import OpenAI

client = OpenAI()

def run(model: str, system: str, user: str) -> str:
    r = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": system},
            {"role": "user", "content": user},
        ],
        temperature=0,
    )
    return r.choices[0].message.content

def score(test_set: list, system: str) -> float:
    correct = 0
    for ex in test_set:
        out = run("gpt-4.1-mini", system, ex["input"])
        if out.strip().lower() == ex["expected"].strip().lower():
            correct += 1
    return correct / len(test_set)

with open("test.jsonl") as f:
    tests = [json.loads(line) for line in f]

base_system = "You classify support tickets into [billing, technical, feature_request]."
fewshot_system = base_system + (
    "\n\nExamples:\n"
    "- 'My card was charged twice' -> billing\n"
    "- 'Login button is broken' -> technical\n"
    "- 'Please add dark mode' -> feature_request"
)

print(f"Baseline: {score(tests, base_system):.1%}")
print(f"Few-shot: {score(tests, fewshot_system):.1%}")
# If few-shot already hits the target, do NOT fine-tune.

Exercise

Walk a real candidate task through the 5-step decision tree out loud. At each step, write the literal answer you have evidence for, not a guess. If at any step you cannot answer with evidence, stop and collect that evidence before continuing. Most projects fail at step 1 (no baseline) or step 4 (no real data) — find your project's true blocker.