Last week, we shared a demo of a Llama-OCR app that someone else built.
This week, we created our own OCR app using the Llama-3.2-vision model, and today, we are sharing how we built it.
You can upload an image, and it converts it into a structured markdown using the Llama-3.2 multimodal model, as shown in the video above.
Here’s what we'll use:
Ollama for serving Llama 3.2 vision locally
Streamlit for the UI
The entire code is available here: Llama OCR Demo GitHub.
Now, let’s build the app.
For simplicity, we are not going to show the Streamlit part. We will just show how the LLM is utilized in this demo.
Also, talking of LLMs, we started a beginner-friendly crash course on RAGs recently with implementations. Read the first four parts below:
Step 1) Download Ollama
Ollama provides a platform to run LLMs locally, giving you control over your data and model usage.
Go to Ollama.com, select your operating system, and follow the instructions.
Step 2) Download Llama3.2-vision
Llama3.2-vision is a multimodal LLM for visual recognition, image reasoning, captioning, and answering general questions about an image.
Download it as follows:
Step 3) Download Ollama Python package
Finally, install the Ollama Python package as follows:
Step 4) Prompt Llama3.2-vision
Almost done!
Finally, prompting Llama3.2-vision with Ollama is as simple as this code:
Done!
Of course, the full Streamlit app will require more code to build, but everything is still just 50 lines of code!
Also, as mentioned above, building the full Streamlit app is the focus of today's issue.
Instead, the goal is to demonstrate how simple it is to use frameworks like Ollama to build LLM apps.
In the app we built, you can upload an image, and it converts it into a structured markdown using the Llama-3.2 multimodal model, as shown in this demo:
The entire code (along with the code for Streamlit) is available here: Llama OCR Demo GitHub.
👉 Over to you: What other demos do you want us to cover next?
P.S. For those wanting to develop “Industry ML” expertise:
At the end of the day, all businesses care about impact. That’s it!
Can you reduce costs?
Drive revenue?
Can you scale ML models?
Predict trends before they happen?
We have discussed several other topics (with implementations) in the past that align with such topics.
Here are some of them:
Learn sophisticated graph architectures and how to train them on graph data: A Crash Course on Graph Neural Networks – Part 1.
So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here: Bi-encoders and Cross-encoders for Sentence Pair Similarity Scoring – Part 1.
Learn techniques to run large models on small devices: Quantization: Optimize ML Models to Run Them on Tiny Hardware.
Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust: Conformal Predictions: Build Confidence in Your ML Model’s Predictions.
Learn how to identify causal relationships and answer business questions: A Crash Course on Causality – Part 1
Learn how to scale ML model training: A Practical Guide to Scaling ML Model Training.
Learn techniques to reliably roll out new models in production: 5 Must-Know Ways to Test ML Models in Production (Implementation Included)
Learn how to build privacy-first ML systems: Federated Learning: A Critical Step Towards Privacy-Preserving Machine Learning.
Learn how to compress ML models and reduce costs: Model Compression: A Critical Step Towards Efficient Machine Learning.
All these resources will help you cultivate key skills that businesses and companies care about the most.
SPONSOR US
Get your product in front of 115,000+ data scientists and machine learning professionals.
Our newsletter puts your products and services directly in front of an audience that matters — thousands of leaders, senior data scientists, machine learning engineers, data analysts, etc., who have influence over significant tech decisions and big purchases.
To ensure your product reaches this influential audience, reserve your space here or reply to this email to ensure your product reaches this influential audience.
Share this post