Project overview
Granny is an experimental voice assistant running locally on a Windows PC. The goal is to speak naturally, have the assistant interpret a request, save simple to-do items, read them back, and respond aloud. This is a working prototype, not a finished product or a hosted web application.
Architecture and tools
- Speech-to-text: Faster-Whisper using the
basemodel on CPU with int8 computation. - Language model: Granite
granite4:3bthrough the local Ollama setup to interpret requests. - Text-to-speech: Piper-generated speech, played through the PC's audio output using Python audio playback.
- Task storage: a plain text file at
%USERPROFILE%\GrannyAssistant odos.txt. - Runtime: a Python script,
granite_task_assistant.py, running locally on Windows.
The design keeps the core interaction local on the PC. That does not by itself guarantee that every model component or dependency is fully offline in every configuration; this prototype's privacy and dependency assumptions should be verified before broader use.
What worked in today's build session
- Whisper captured speech and printed the transcription for inspection.
- Granite interpreted a task request and saved Buy bread to the to-do file.
- Granny read back the saved item and identified the file where it was stored.
- Spoken responses and visible console responses worked together.
- A Python compilation check was used to catch a tab/space indentation error before running the script again.
What went wrong — and what it taught us
When the spoken request was “Add a task, buy milk”, Whisper transcribed it as “Add a task by milk”. The assistant then saved the misheard words. The distinction between buy and by is difficult because the words sound alike; this is a speech-recognition/context problem, not simply a matter of adding more code.
A small phrase-specific correction was attempted, but the following test still showed the same transcription and response. Therefore, that issue is not yet verified as fixed. Adding many one-off substitutions would make the code brittle and would not deliver perfect recognition.
An indentation error also occurred after a code edit mixed tabs and spaces. Removing the temporary workaround and restoring a consistently indented main() got the script compiling and running again. The lesson is to make small, targeted changes, compile-check after edits, and preserve working audio code.
Current status
Known limitation: similar-sounding words can be transcribed incorrectly; the “buy milk” case remains unresolved.
Not yet demonstrated: a packaged installer, robust confirmation flow, broad error recovery, automated tests, or a browser-hosted version.
Practical next steps
- Keep a known-good copy of the working script and avoid changing the audio playback path.
- Try a small set of representative spoken commands and record both the transcription and saved result.
- For ambiguous task wording, consider asking for confirmation before saving instead of silently guessing.
- Keep phrase correction rules narrow and tested; prefer contextual confirmation over an ever-growing list of substitutions.
- Before publishing the actual source code, review it for machine-specific paths, local configuration, credentials, and other private details.
Why this project matters
This project demonstrates practical integration and debugging across Python, speech recognition, a locally hosted language model, text-to-speech, file I/O, and a real microphone/audio loop. It also records a realistic engineering lesson: a prototype can be genuinely useful without being perfect, and documenting a failed fix is better than claiming a problem has been solved when the test says otherwise.