Local LLM Lab

Ollama · Granite 4 3B · Python · Hermes automation

A real-world experiment in local AI, practical limitations, and turning troubleshooting into a useful automation.

A desktop computer running a local language model, with data kept on the device

Project overview

I spent time testing whether a small language model running locally could reliably help with AWS Solutions Architect Associate (SAA-C03) practice questions. The experiment used Granite 4 3B through Ollama on my own computer, rather than relying entirely on a hosted model.

The goal was to explore what a local model could do, understand its limits, and learn how to connect it to an existing automation agent. It was a learning and engineering exercise, not a claim that the model can guarantee correct exam answers.

What I worked through

1. Run locally

Installed and tested Granite 4 3B through Ollama, then checked that a local prompt could produce a captured response.

2. Test the limits

Evaluated the idea of using a local model for AWS exam practice. Its accuracy was not consistent enough to trust as an answer key.

3. Troubleshoot Hermes

Worked through several issues trying to connect the model directly into Hermes. The direct integration route was more difficult than expected.

4. Build a practical script

A Python script eventually provided a working way to run the local test and capture the output for reporting.

5. Report scheduled jobs

Connected the reporting workflow with Hermes cron-job monitoring and AgentMail delivery to a private email inbox.

Hermes daily cron report

The useful outcome was broader than the LLM test itself. The reporting workflow is intended to summarise which scheduled Hermes jobs ran during the day and identify successful runs and failures, so problems are easier to spot without manually checking every job.

Honest result: no tested local model was demonstrated to be 100% accurate on SAA-C03 questions. I continue to use structured practice and explanations to build my own AWS knowledge; the local model remains an experiment, not an exam authority.

What are Ollama, Granite and Hermes?

Ollama is the local model runner used in this experiment. It makes it possible to download and run compatible language models on a personal computer and send them prompts through a local interface. It is the tool running the model, rather than the model itself. Running locally can reduce dependence on a hosted AI service, but performance depends on the model, available memory and hardware.

Granite 4 3B is the language model tested here. “3B” refers to roughly three billion parameters. It was small enough to try locally and was useful for basic prompt handling, structured output and some reasoning tasks. Hermes is the automation agent in this setup: it runs scheduled jobs (cron), coordinates tools and produces reports. AgentMail was used to deliver the daily report to my private inbox.

Models tested: what worked and what did not

I compared several small models through Ollama against AWS Solutions Architect Associate (SAA-C03)-style tasks. These were practical experiments, not a statistically rigorous benchmark or an official exam score. The same broad lesson kept appearing: a response can sound confident and still contain a technical error.

ModelWhat I observedAssessment for AWS practice
MiniCPM-V 4.7 1BRepeated parts of the prompt and exposed awkward raw reasoning markers such as </think>. Its VPC explanations were clumsy and did not consistently explain the networking path clearly.Not reliable enough
Output quality and clarity needed too much checking.
Qwen3 1.7BCould complete some simple checks, including returning the expected DAILY REPORT READY marker and some route-selection reasoning. But some AWS practice questions were flawed, and structured JSON output became corrupted.Mixed results
Useful for simple experiments, but not dependable for generating or grading exam questions unattended.
Granite 4 3BProduced the expected structured-output marker and handled some route-selection reasoning. However, a VPC question had ambiguous options and an incorrect NAT detail; a traffic-path explanation misstated source/destination translation; and an RDS question introduced Aurora Global Database even though the correct choice was B.Best practical fit tested, with caveats
Promising for controlled local tasks and reports, but still needs verification for AWS technical answers.

Examples of the AWS-answer problems

VPC and NAT

Granite's VPC response included an incorrect NAT detail and the answer options were not cleanly distinguishable. That matters because AWS questions often test exactly which subnet initiates traffic and how a route table directs it.

Traffic flow

In a route/path question, the model was mostly on the right track but described source and destination address translation inaccurately. A nearly correct networking explanation can still teach the wrong mental model.

RDS architecture

In an RDS question, Granite introduced Aurora Global Database even though that was not the correct answer (the expected choice was B). Adding a plausible AWS service does not make a response correct.

Question and JSON quality

Qwen3 1.7B sometimes generated flawed AWS questions and corrupted JSON, while MiniCPM-V 4.7 1B repeated prompt content and exposed raw reasoning markers. Those failures made automation and dependable marking difficult.

What Granite was useful for

Granite is not an AWS-specific expert model, and a small model does not automatically know the latest service behaviour or interpret every exam scenario correctly. Its useful role here was as a local component for bounded tasks and experimentation—not as an authoritative answer key.

From model testing to an operational report

After spending many hours trying to connect the local model directly with Hermes, I switched to a simpler Python-script approach. That was the turning point: instead of making the whole workflow depend on a fragile direct integration, the script ran the local test and captured the result. The Hermes reporting workflow could then focus on summarising the day's scheduled jobs.

Local Ollama model ↓ Python test / capture script ↓ Hermes scheduled-job results ↓ Daily success-and-failure summary ↓ AgentMail → private email inbox

The email report is useful because it gives me a daily view of which Hermes cron jobs ran and which failed, rather than leaving me to inspect each job manually. The model experiment and the cron-monitoring workflow are related, but they are separate responsibilities: the local LLM is tested locally, while Hermes handles scheduled automation and reporting.

Important distinction: I did not achieve 100% accuracy on AWS questions, and this page does not claim that. The meaningful result was learning where small models fail, verifying output rather than trusting confidence, and finding a working script-based route to useful automation.

The hardware I was working with

This experiment was run on my own everyday Windows PC, not an AI workstation: an AMD Ryzen 5 3400G, around 14 GB of usable RAM, and no dedicated high-end graphics card. That matters. Large language models can use a lot of memory, and without a powerful GPU the CPU may have to do most of the work. A model can technically start and still feel too slow to use comfortably.

I have tried different local-AI experiments over time and hoped there might be a simple route to a capable private assistant. The reality is that hardware, model size, quantisation, context length and the job being asked all matter. I have not found a magic model that turns this PC into a high-end AI machine, so the sensible approach is to use small models for bounded tasks rather than chase the biggest models.

Why not just use the biggest model? A model download size is not the same as its total RAM requirement. Windows, the model's working context and other processes need memory too. On this PC, a quantised model in the roughly 1B–4B range is a more realistic starting point; larger models may be possible in some configurations but can be slow or leave little headroom.

Small models worth considering on a 14 GB RAM PC

These are practical candidates to test through Ollama, not promises of perfect answers or guaranteed speed. Actual performance depends on the model variant, quantisation, context settings and what else is running.

ModelWhy try itGood usesLimitations
Granite 4 3B
granite4:3b
Already tested in this project; handled a simple structured-output check and some routing reasoning.Summaries, extraction, simple classifications, controlled tool/workflow experiments.My AWS tests still exposed incorrect networking and database details. Never treat its confident answer as proof.
Qwen3 1.7B
qwen3:1.7b
Small download and responsive enough to experiment with on modest hardware.Simple questions, short drafts, labels, basic reasoning tests.In my tests it generated flawed AWS practice questions and malformed JSON, so unattended output needs validation.
LFM2.5 1.2B
lfm2.5-thinking:1.2b
A lightweight on-device model worth trying when memory and CPU speed are priorities.Short text tasks, simple reasoning experiments, small automation steps.Small does not mean universally accurate; test the exact task and allow for slower CPU inference.
Gemma 3 1B
gemma3:1b
A compact general-purpose option for comparison.Basic chat, rewriting, short summaries and classification.Expect weaker performance on multi-step technical reasoning than from larger models; verify facts.
Gemma 3 4B
gemma3:4b
A more capable-size experiment if the smaller models are not enough.Longer summaries, more involved prompts, some coding assistance.More demanding on RAM and likely slower on a CPU-only setup. Try a quantised build and short context first.
Qwen3 4B
qwen3:4b
Another step up in size for comparing answer quality against speed.General assistance, code explanations and more complex reasoning tests.May be noticeably slower on this PC; a bigger model can still make mistakes.

Official model listings and download variants change over time: Granite 4, Qwen3, LFM2.5 Thinking and Gemma 3. Check the current variant and download size before installing.

What can small models do reliably?

For my Hermes reporting experiment, the useful job was not asking a tiny model to understand everything. It was capturing scheduled-job results and reporting successes and failures. A predictable Python script and explicit checks are a better foundation for that than hoping a language model always interprets output correctly.

MiniCPM-V, phones and PC expectations

A YouTube video by Julian Goldie brought MiniCPM-V to my attention. The important distinction is that MiniCPM-V is a vision-language model family, and different model variants and device builds can target different hardware. A demo on an iPhone does not automatically mean that the same model will run better on every PC: phones may use their own optimised runtimes and acceleration, while my Windows PC has different hardware constraints. The model version, quantisation, runtime and task all need to match the device.

My own MiniCPM-V 4.7 1B test on this PC repeated parts of prompts and exposed raw reasoning markers, and its VPC explanations were not clear enough. That is evidence about the version and setup I tested—not proof that every MiniCPM variant is bad, or that an iPhone is always more powerful than a PC.

My conclusion: I am not giving up on AI. I am being realistic about local models on this hardware. Small models can be useful helpers for narrow, checkable tasks; they are not a drop-in replacement for a strong cloud model or a dependable AWS expert. The most valuable outcome so far has been learning to test, verify and simplify the workflow.

How to test a local model from Windows

This is a small, repeatable test bench for Ollama. It checks whether the model answers basic facts, follows a simple format, explains an AWS concept, and handles a small scenario. Run the same prompts against different models and compare accuracy, clarity and speed. These are manual smoke tests, not a scientific benchmark.

Step 1 — Check Ollama and your installed models

ollama --version ollama list ollama ps

ollama list shows downloaded models; ollama ps shows models currently loaded in memory.

Step 2 — Try a small Llama model

For example, Llama 3.2 1B is a compact text model. Its Ollama library listing gives the current variant and download details. It is a test candidate, not a guarantee that it will be the best model for this PC.

ollama pull llama3.2:1b ollama run llama3.2:1b

At the interactive prompt, enter one question at a time:

What is the capital of France? Answer in one sentence. Explain what an Amazon VPC is in two simple sentences. A private subnet needs outbound IPv4 internet access initiated by instances, but no inbound internet connections. Which AWS component is commonly used, and where should it be placed? Explain briefly.

Exit the chat with /bye. A small model may know the capital but still get AWS details wrong. Check technical answers against trusted AWS documentation or your study materials.

Step 3 — Run a prompt directly from PowerShell

You do not have to enter interactive chat each time. This runs one prompt and returns to PowerShell:

ollama run llama3.2:1b "What is the capital of France? Answer in one sentence." ollama run llama3.2:1b "Explain Amazon S3 in two simple sentences." ollama run llama3.2:1b "Name three checks you would make before troubleshooting an EC2 instance that cannot reach the internet."

Swap llama3.2:1b for a model you already have, such as granite4:3b or qwen3:1.7b (use the exact tag shown by ollama list).

Step 4 — Capture the answer and elapsed time

This PowerShell example saves a response to a text file and reports how long the command took. Run it in a normal PowerShell window, not inside Ollama's interactive chat:

$model = "llama3.2:1b" $prompt = "What is the capital of France? Answer in one sentence." $timer = [System.Diagnostics.Stopwatch]::StartNew() $answer = ollama run $model $prompt 2>&1 | Out-String $timer.Stop() "Model: $model`nPrompt: $prompt`nElapsed: $($timer.Elapsed.TotalSeconds) seconds`n`nAnswer:`n$answer" | Tee-Object -FilePath ".\llm-test-result.txt"

Repeat with the same prompt and a different model, changing only the $model value. Timing is approximate and can vary with first-load overhead, background applications and model settings.

Step 5 — Test format-following, not just knowledge

Small models are often useful for bounded tasks, but they can also break simple instructions. Try this prompt:

Return exactly one line of JSON with keys "capital" and "country" for the capital of France. Do not add Markdown or commentary.

Check whether the response is valid JSON and contains the expected values. A script can validate the format; that does not prove the model will be accurate on harder questions.

Step 6 — Keep a simple comparison record

TestExpected resultWhat to record
Basic factParisCorrect / incorrect; response time
Instruction followingOne sentence or two sentences as requestedFollowed length and format?
JSON outputValid JSON with requested keys and valuesValid syntax and correct values?
AWS conceptConsistent with trusted AWS documentationTechnical accuracy, missing caveats, invented details
AWS scenarioReasoned answer with assumptions made clearCorrect choice and explanation, not just confidence
Important: the capital-of-France test only checks a basic fact. A model passing it does not show that it is safe or accurate for AWS architecture, security, or exam answers. Score a set of questions with known answers, keep the prompts identical across models, and verify technical claims independently.

References: Llama 3.2 model library · Ollama CLI reference.

What this demonstrates

This project reflects a practical approach to learning: try the integration, investigate what fails, simplify the workflow, and keep the working result useful and honest.


Beginner’s guide: try your own local AI

Curious about running a language model on your own computer? You can try a small model with Ollama, a tool that downloads and runs supported models locally. You do not need to build an AI application to start: install Ollama, open PowerShell, download a model, and ask it a question.

Is it free?

Ollama is free to download and use. Many models can be downloaded without paying, but each model has its own licence and terms, so check those before commercial use. You will need an internet connection to download the model. Once downloaded, a local model can usually answer prompts without sending them to a hosted AI service, provided you use the local Ollama interface and your other software does not forward the data elsewhere.

1. Install Ollama

Download it from the official Ollama for Windows page and follow the installer. Then open a new PowerShell window and check it is available:

ollama --version ollama list

The first command checks the installation. The second lists models you have already downloaded.

2. Download a small model and start chatting

Here are a few beginner-friendly starting points. They are not ranked guarantees: choose one first, then try others if it is too slow or the answers are weak.

For example, to try Llama 3.2 1B, run:

ollama run llama3.2:1b

The first run downloads the model if needed. When its prompt appears, try these examples:

What is the capital of France? Answer in one sentence. Explain photosynthesis as if I were 12 years old. Give me three ideas for a simple weekend meal. Summarise this paragraph in three bullet points: [paste a short paragraph here]

To leave the interactive chat, type /bye. For the other models, check their official Ollama library pages for the current model tags and instructions: Llama 3.2, Qwen3 and Gemma 3.

3. Ask a single question straight from PowerShell

You can also run one prompt without entering the interactive chat:

ollama run llama3.2:1b "What is the capital of France?" ollama run llama3.2:1b "Explain what a password manager does in two sentences." ollama run llama3.2:1b "Give me five beginner tips for learning Python."

Replace llama3.2:1b with the exact tag shown by the model's Ollama library page or by ollama list.

4. What kind of computer do you need?

Small models are generally the easiest place to start on an ordinary PC. Bigger models usually need more memory and may run slowly without a suitable GPU. As a rough rule, try a 1B–4B quantised model first on a machine with around 14 GB RAM; actual requirements vary by model, context length and other programs running. If it feels slow, close memory-heavy apps or try a smaller model.

5. What can a small local model do?

6. Does “local” mean private?

Running prompts through the local Ollama service can keep them on your own computer instead of sending them to a hosted model. But privacy also depends on the application you use, plugins, telemetry, network settings and any tools connected to the model. Avoid entering passwords, API keys or sensitive personal information, and check the software's documentation if privacy is important.

Beginner tip: start with one small model and three questions whose answers you already know. Try the same prompts with another model and compare correctness, clarity and speed. Passing a simple test like “What is the capital of France?” does not prove a model is reliable at harder tasks.

Watch: run Llama from Windows Command Prompt

If you prefer typing commands instead of using a browser-based chat interface, start here. This Windows tutorial shows how to install Ollama and run small Llama models from your own computer.

▶ Watch on YouTube: Download, Install, and Run Llama 3.2 1B and 3B in Windows Using Ollama — Aleksandar Haber PhD.

After installing Ollama, open PowerShell or Command Prompt and try:

ollama --version
ollama list
ollama run llama3.2:1b

The first run downloads the model. When the chat prompt appears, type a question and press Enter. Type /bye to leave the chat. You do not need Open WebUI for this workflow.

Official installer: Ollama for Windows. The model runs locally once downloaded; allow time and disk space for the initial download.