Prompting vs. RAG vs. Finetuning
Which one is best for you?
Build incredibly robust RAG systems with EyeLevel
Vanilla RAGs work as long as your external docs. look like the image on the left, but real-world documents are like the image on the right:
They have images, text, tables, flowcharts, and whatnot!
No vanilla RAG system can handle this complexity.
EyeLevel's GroundX is solving this.
They are developing systems that can intuitively chunk relevant content and understand what’s inside each chunk, whether it's text, images, or diagrams as shown below:
As depicted above, the system takes an unstructured (text, tables, images, flow charts) input and parses it into a JSON format that LLMs can easily process to build RAGs over.
Try EyeLevel's GroundX to build real-world RAG systems:
Thanks to EyeLevel for sponsoring today's issue.
Prompting vs. RAG vs. Finetuning?
Continuing the discussion on RAGs from EyeLevel...
If you are building real-world LLM-based apps, it is unlikely you can start using the model right away without adjustments. To maintain high utility, you either need:
Prompt engineering
Fine-tuning
RAG
Or a hybrid approach (RAG + fine-tuning)
The following visual will help you decide which one is best for you:
Two important parameters guide this decision:
The amount of external knowledge required for your task.
The amount of adaptation you need. Adaptation, in this case, means changing the behavior of the model, its vocabulary, writing style, etc.
For instance, an LLM might find it challenging to summarize the transcripts of company meetings because speakers might be using some internal vocabulary in their discussions.
So here's the simple takeaway:
Use RAGs to generate outputs based on a custom knowledge base if the vocabulary & writing style of the LLM remains the same.
Use fine-tuning to change the structure (behaviour) of the model than knowledge.
Prompt engineering is sufficient if you don't have a custom knowledge base and don't want to change the behavior.
And finally, if your application demands a custom knowledge base and a change in the model's behavior, use a hybrid (RAG + Fine-tuning) approach.
That's it!
If RAG is your solution, check out EyeLevel's GroundX for building robust RAG systems on complex real-world documents that don’t break in production.
👉 Over to you: How do you decide between prompting, RAG, and fine-tuning?
P.S. For those wanting to develop “Industry ML” expertise:
At the end of the day, all businesses care about impact. That’s it!
Can you reduce costs?
Drive revenue?
Can you scale ML models?
Predict trends before they happen?
We have discussed several other topics (with implementations) in the past that align with such topics.
Here are some of them:
Learn sophisticated graph architectures and how to train them on graph data: A Crash Course on Graph Neural Networks – Part 1.
So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here: Bi-encoders and Cross-encoders for Sentence Pair Similarity Scoring – Part 1.
Learn techniques to run large models on small devices: Quantization: Optimize ML Models to Run Them on Tiny Hardware.
Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust: Conformal Predictions: Build Confidence in Your ML Model’s Predictions.
Learn how to identify causal relationships and answer business questions: A Crash Course on Causality – Part 1
Learn how to scale ML model training: A Practical Guide to Scaling ML Model Training.
Learn techniques to reliably roll out new models in production: 5 Must-Know Ways to Test ML Models in Production (Implementation Included)
Learn how to build privacy-first ML systems: Federated Learning: A Critical Step Towards Privacy-Preserving Machine Learning.
Learn how to compress ML models and reduce costs: Model Compression: A Critical Step Towards Efficient Machine Learning.
All these resources will help you cultivate key skills that businesses and companies care about the most.
SPONSOR US
Get your product in front of 450k+ data scientists and other tech professionals.
Our newsletter puts your products and services directly in front of an audience that matters — thousands of leaders, senior data scientists, machine learning engineers, data analysts, etc., who have influence over significant tech decisions and big purchases.
To ensure your product reaches this influential audience, reserve your space here or reply to this email to ensure your product reaches this influential audience.