RAG or Fine-Tuning? A Plain-English Decision Tree

The short version

• RAG (Retrieval-Augmented Generation): the model looks up your documents at answer time
• Fine-tuning: you teach the model to behave a certain way

For 90% of SME use cases, the answer is RAG.

Use RAG when

• Your knowledge changes often (policies, products, prices)
• You need to cite sources
• You want to add or remove documents without retraining

Use fine-tuning when

• You need a very specific output format or tone consistently
• You have thousands of high-quality examples of the desired behaviour
• Latency or cost at scale matters more than freshness

Often the right answer: both

Fine-tune for tone and format. Use RAG for the facts. This is how most production assistants are built today.

What to skip

Don't fine-tune to "teach the model your domain" with a few hundred PDFs. It rarely works the way demos suggest. Use RAG.