Project overview
I spent time testing whether a small language model running locally could reliably help with AWS Solutions Architect Associate (SAA-C03) practice questions. The experiment used Granite 4 3B through Ollama on my own computer, rather than relying entirely on a hosted model.
The goal was to explore what a local model could do, understand its limits, and learn how to connect it to an existing automation agent. It was a learning and engineering exercise, not a claim that the model can guarantee correct exam answers.
What I worked through
1. Run locally
Installed and tested Granite 4 3B through Ollama, then checked that a local prompt could produce a captured response.
2. Test the limits
Evaluated the idea of using a local model for AWS exam practice. Its accuracy was not consistent enough to trust as an answer key.
3. Troubleshoot Hermes
Worked through several issues trying to connect the model directly into Hermes. The direct integration route was more difficult than expected.
4. Build a practical script
A Python script eventually provided a working way to run the local test and capture the output for reporting.
5. Report scheduled jobs
Connected the reporting workflow with Hermes cron-job monitoring and AgentMail delivery to a private email inbox.
Hermes daily cron report
The useful outcome was broader than the LLM test itself. The reporting workflow is intended to summarise which scheduled Hermes jobs ran during the day and identify successful runs and failures, so problems are easier to spot without manually checking every job.
What are Ollama, Granite and Hermes?
Ollama is the local model runner used in this experiment. It makes it possible to download and run compatible language models on a personal computer and send them prompts through a local interface. It is the tool running the model, rather than the model itself. Running locally can reduce dependence on a hosted AI service, but performance depends on the model, available memory and hardware.
Granite 4 3B is the language model tested here. “3B” refers to roughly three billion parameters. It was small enough to try locally and was useful for basic prompt handling, structured output and some reasoning tasks. Hermes is the automation agent in this setup: it runs scheduled jobs (cron), coordinates tools and produces reports. AgentMail was used to deliver the daily report to my private inbox.
Models tested: what worked and what did not
I compared several small models through Ollama against AWS Solutions Architect Associate (SAA-C03)-style tasks. These were practical experiments, not a statistically rigorous benchmark or an official exam score. The same broad lesson kept appearing: a response can sound confident and still contain a technical error.
| Model | What I observed | Assessment for AWS practice |
|---|---|---|
| MiniCPM-V 4.7 1B | Repeated parts of the prompt and exposed awkward raw reasoning markers such as </think>. Its VPC explanations were clumsy and did not consistently explain the networking path clearly. | Not reliable enough Output quality and clarity needed too much checking. |
| Qwen3 1.7B | Could complete some simple checks, including returning the expected DAILY REPORT READY marker and some route-selection reasoning. But some AWS practice questions were flawed, and structured JSON output became corrupted. | Mixed results Useful for simple experiments, but not dependable for generating or grading exam questions unattended. |
| Granite 4 3B | Produced the expected structured-output marker and handled some route-selection reasoning. However, a VPC question had ambiguous options and an incorrect NAT detail; a traffic-path explanation misstated source/destination translation; and an RDS question introduced Aurora Global Database even though the correct choice was B. | Best practical fit tested, with caveats Promising for controlled local tasks and reports, but still needs verification for AWS technical answers. |
Examples of the AWS-answer problems
VPC and NAT
Granite's VPC response included an incorrect NAT detail and the answer options were not cleanly distinguishable. That matters because AWS questions often test exactly which subnet initiates traffic and how a route table directs it.
Traffic flow
In a route/path question, the model was mostly on the right track but described source and destination address translation inaccurately. A nearly correct networking explanation can still teach the wrong mental model.
RDS architecture
In an RDS question, Granite introduced Aurora Global Database even though that was not the correct answer (the expected choice was B). Adding a plausible AWS service does not make a response correct.
Question and JSON quality
Qwen3 1.7B sometimes generated flawed AWS questions and corrupted JSON, while MiniCPM-V 4.7 1B repeated prompt content and exposed raw reasoning markers. Those failures made automation and dependable marking difficult.
What Granite was useful for
- Local inference: run prompts on the computer through Ollama, rather than depending on a remote model for every test.
- Structured responses: it passed a simple expected-output check used during the reporting tests.
- Basic reasoning: some route-selection and workflow tasks were handled reasonably, which made it worth investigating further.
- Automation experiments: the local run could be wrapped in a Python script so output could be captured and included in a repeatable workflow.
Granite is not an AWS-specific expert model, and a small model does not automatically know the latest service behaviour or interpret every exam scenario correctly. Its useful role here was as a local component for bounded tasks and experimentation—not as an authoritative answer key.
From model testing to an operational report
After spending many hours trying to connect the local model directly with Hermes, I switched to a simpler Python-script approach. That was the turning point: instead of making the whole workflow depend on a fragile direct integration, the script ran the local test and captured the result. The Hermes reporting workflow could then focus on summarising the day's scheduled jobs.
The email report is useful because it gives me a daily view of which Hermes cron jobs ran and which failed, rather than leaving me to inspect each job manually. The model experiment and the cron-monitoring workflow are related, but they are separate responsibilities: the local LLM is tested locally, while Hermes handles scheduled automation and reporting.
The hardware I was working with
This experiment was run on my own everyday Windows PC, not an AI workstation: an AMD Ryzen 5 3400G, around 14 GB of usable RAM, and no dedicated high-end graphics card. That matters. Large language models can use a lot of memory, and without a powerful GPU the CPU may have to do most of the work. A model can technically start and still feel too slow to use comfortably.
I have tried different local-AI experiments over time and hoped there might be a simple route to a capable private assistant. The reality is that hardware, model size, quantisation, context length and the job being asked all matter. I have not found a magic model that turns this PC into a high-end AI machine, so the sensible approach is to use small models for bounded tasks rather than chase the biggest models.
Small models worth considering on a 14 GB RAM PC
These are practical candidates to test through Ollama, not promises of perfect answers or guaranteed speed. Actual performance depends on the model variant, quantisation, context settings and what else is running.
| Model | Why try it | Good uses | Limitations |
|---|---|---|---|
Granite 4 3Bgranite4:3b | Already tested in this project; handled a simple structured-output check and some routing reasoning. | Summaries, extraction, simple classifications, controlled tool/workflow experiments. | My AWS tests still exposed incorrect networking and database details. Never treat its confident answer as proof. |
Qwen3 1.7Bqwen3:1.7b | Small download and responsive enough to experiment with on modest hardware. | Simple questions, short drafts, labels, basic reasoning tests. | In my tests it generated flawed AWS practice questions and malformed JSON, so unattended output needs validation. |
LFM2.5 1.2Blfm2.5-thinking:1.2b | A lightweight on-device model worth trying when memory and CPU speed are priorities. | Short text tasks, simple reasoning experiments, small automation steps. | Small does not mean universally accurate; test the exact task and allow for slower CPU inference. |
Gemma 3 1Bgemma3:1b | A compact general-purpose option for comparison. | Basic chat, rewriting, short summaries and classification. | Expect weaker performance on multi-step technical reasoning than from larger models; verify facts. |
Gemma 3 4Bgemma3:4b | A more capable-size experiment if the smaller models are not enough. | Longer summaries, more involved prompts, some coding assistance. | More demanding on RAM and likely slower on a CPU-only setup. Try a quantised build and short context first. |
Qwen3 4Bqwen3:4b | Another step up in size for comparing answer quality against speed. | General assistance, code explanations and more complex reasoning tests. | May be noticeably slower on this PC; a bigger model can still make mistakes. |
Official model listings and download variants change over time: Granite 4, Qwen3, LFM2.5 Thinking and Gemma 3. Check the current variant and download size before installing.
What can small models do reliably?
- Often a good fit: summarising a short log, extracting named fields from predictable text, sorting messages into a few categories, rewriting a paragraph, drafting a report from known results, or producing a first-pass answer for a person to review.
- Use guardrails: outputting JSON, choosing an automation action, or answering a technical question. Validate the format, constrain the choices, and check important fields in code.
- Do not trust them alone: AWS architecture decisions, exam answer keys, current service limits/prices, security-sensitive changes, or any task where a plausible technical error could cause harm.
For my Hermes reporting experiment, the useful job was not asking a tiny model to understand everything. It was capturing scheduled-job results and reporting successes and failures. A predictable Python script and explicit checks are a better foundation for that than hoping a language model always interprets output correctly.
MiniCPM-V, phones and PC expectations
A YouTube video by Julian Goldie brought MiniCPM-V to my attention. The important distinction is that MiniCPM-V is a vision-language model family, and different model variants and device builds can target different hardware. A demo on an iPhone does not automatically mean that the same model will run better on every PC: phones may use their own optimised runtimes and acceleration, while my Windows PC has different hardware constraints. The model version, quantisation, runtime and task all need to match the device.
My own MiniCPM-V 4.7 1B test on this PC repeated parts of prompts and exposed raw reasoning markers, and its VPC explanations were not clear enough. That is evidence about the version and setup I tested—not proof that every MiniCPM variant is bad, or that an iPhone is always more powerful than a PC.
How to test a local model from Windows
This is a small, repeatable test bench for Ollama. It checks whether the model answers basic facts, follows a simple format, explains an AWS concept, and handles a small scenario. Run the same prompts against different models and compare accuracy, clarity and speed. These are manual smoke tests, not a scientific benchmark.
Step 1 — Check Ollama and your installed models
ollama list shows downloaded models; ollama ps shows models currently loaded in memory.
Step 2 — Try a small Llama model
For example, Llama 3.2 1B is a compact text model. Its Ollama library listing gives the current variant and download details. It is a test candidate, not a guarantee that it will be the best model for this PC.
At the interactive prompt, enter one question at a time:
Exit the chat with /bye. A small model may know the capital but still get AWS details wrong. Check technical answers against trusted AWS documentation or your study materials.
Step 3 — Run a prompt directly from PowerShell
You do not have to enter interactive chat each time. This runs one prompt and returns to PowerShell:
Swap llama3.2:1b for a model you already have, such as granite4:3b or qwen3:1.7b (use the exact tag shown by ollama list).
Step 4 — Capture the answer and elapsed time
This PowerShell example saves a response to a text file and reports how long the command took. Run it in a normal PowerShell window, not inside Ollama's interactive chat:
Repeat with the same prompt and a different model, changing only the $model value. Timing is approximate and can vary with first-load overhead, background applications and model settings.
Step 5 — Test format-following, not just knowledge
Small models are often useful for bounded tasks, but they can also break simple instructions. Try this prompt:
Check whether the response is valid JSON and contains the expected values. A script can validate the format; that does not prove the model will be accurate on harder questions.
Step 6 — Keep a simple comparison record
| Test | Expected result | What to record |
|---|---|---|
| Basic fact | Paris | Correct / incorrect; response time |
| Instruction following | One sentence or two sentences as requested | Followed length and format? |
| JSON output | Valid JSON with requested keys and values | Valid syntax and correct values? |
| AWS concept | Consistent with trusted AWS documentation | Technical accuracy, missing caveats, invented details |
| AWS scenario | Reasoned answer with assumptions made clear | Correct choice and explanation, not just confidence |
References: Llama 3.2 model library · Ollama CLI reference.
What this demonstrates
- Running and testing a local LLM using Ollama.
- Evaluating model output instead of assuming it is correct.
- Python scripting to make a repeatable local test and report.
- Troubleshooting integration boundaries between tools.
- Connecting scheduled automation results to email reporting.
This project reflects a practical approach to learning: try the integration, investigate what fails, simplify the workflow, and keep the working result useful and honest.
Beginner’s guide: try your own local AI
Curious about running a language model on your own computer? You can try a small model with Ollama, a tool that downloads and runs supported models locally. You do not need to build an AI application to start: install Ollama, open PowerShell, download a model, and ask it a question.
Is it free?
Ollama is free to download and use. Many models can be downloaded without paying, but each model has its own licence and terms, so check those before commercial use. You will need an internet connection to download the model. Once downloaded, a local model can usually answer prompts without sending them to a hosted AI service, provided you use the local Ollama interface and your other software does not forward the data elsewhere.
1. Install Ollama
Download it from the official Ollama for Windows page and follow the installer. Then open a new PowerShell window and check it is available:
The first command checks the installation. The second lists models you have already downloaded.
2. Download a small model and start chatting
Here are a few beginner-friendly starting points. They are not ranked guarantees: choose one first, then try others if it is too slow or the answers are weak.
- Llama 3.2 1B — a compact text model for basic chat and simple tasks.
- Qwen3 1.7B — another small model to compare for short questions and basic reasoning.
- Gemma 3 1B — a compact general-purpose model for simple prompts and summaries.
For example, to try Llama 3.2 1B, run:
The first run downloads the model if needed. When its prompt appears, try these examples:
To leave the interactive chat, type /bye. For the other models, check their official Ollama library pages for the current model tags and instructions: Llama 3.2, Qwen3 and Gemma 3.
3. Ask a single question straight from PowerShell
You can also run one prompt without entering the interactive chat:
Replace llama3.2:1b with the exact tag shown by the model's Ollama library page or by ollama list.
4. What kind of computer do you need?
Small models are generally the easiest place to start on an ordinary PC. Bigger models usually need more memory and may run slowly without a suitable GPU. As a rough rule, try a 1B–4B quantised model first on a machine with around 14 GB RAM; actual requirements vary by model, context length and other programs running. If it feels slow, close memory-heavy apps or try a smaller model.
5. What can a small local model do?
- Often useful for: drafting, rewriting, simple summaries, brainstorming, explaining everyday concepts and sorting short pieces of text.
- Check its work for: calculations, code, structured data, factual claims and technical instructions.
- Do not rely on it alone for: medical, legal, financial, security or important cloud architecture decisions. Small models can sound certain while being wrong.
6. Does “local” mean private?
Running prompts through the local Ollama service can keep them on your own computer instead of sending them to a hosted model. But privacy also depends on the application you use, plugins, telemetry, network settings and any tools connected to the model. Avoid entering passwords, API keys or sensitive personal information, and check the software's documentation if privacy is important.
Watch: run Llama from Windows Command Prompt
If you prefer typing commands instead of using a browser-based chat interface, start here. This Windows tutorial shows how to install Ollama and run small Llama models from your own computer.
▶ Watch on YouTube: Download, Install, and Run Llama 3.2 1B and 3B in Windows Using Ollama — Aleksandar Haber PhD.
After installing Ollama, open PowerShell or Command Prompt and try:
ollama --version
ollama list
ollama run llama3.2:1b
The first run downloads the model. When the chat prompt appears, type a question and press Enter. Type /bye to leave the chat. You do not need Open WebUI for this workflow.
Official installer: Ollama for Windows. The model runs locally once downloaded; allow time and disk space for the initial download.