Granny — Local AI Voice Assistant

A practical Windows voice-assistant prototype built, tested and debugged on 26 October 2026.

PythonWhisperGranite 4:3bPiper TTSLocal-first

← Back to all projects

Project overview

Granny is an experimental voice assistant running locally on a Windows PC. The goal is to speak naturally, have the assistant interpret a request, save simple to-do items, read them back, and respond aloud. This is a working prototype, not a finished product or a hosted web application.

Architecture and tools

The design keeps the core interaction local on the PC. That does not by itself guarantee that every model component or dependency is fully offline in every configuration; this prototype's privacy and dependency assumptions should be verified before broader use.

What worked in today's build session

What went wrong — and what it taught us

When the spoken request was “Add a task, buy milk”, Whisper transcribed it as “Add a task by milk”. The assistant then saved the misheard words. The distinction between buy and by is difficult because the words sound alike; this is a speech-recognition/context problem, not simply a matter of adding more code.

A small phrase-specific correction was attempted, but the following test still showed the same transcription and response. Therefore, that issue is not yet verified as fixed. Adding many one-off substitutions would make the code brittle and would not deliver perfect recognition.

An indentation error also occurred after a code edit mixed tabs and spaces. Removing the temporary workaround and restoring a consistently indented main() got the script compiling and running again. The lesson is to make small, targeted changes, compile-check after edits, and preserve working audio code.

Current status

Working: listen/transcribe loop, visible output, spoken replies, basic task saving, and reading the last saved task back.
Known limitation: similar-sounding words can be transcribed incorrectly; the “buy milk” case remains unresolved.
Not yet demonstrated: a packaged installer, robust confirmation flow, broad error recovery, automated tests, or a browser-hosted version.

Practical next steps

  1. Keep a known-good copy of the working script and avoid changing the audio playback path.
  2. Try a small set of representative spoken commands and record both the transcription and saved result.
  3. For ambiguous task wording, consider asking for confirmation before saving instead of silently guessing.
  4. Keep phrase correction rules narrow and tested; prefer contextual confirmation over an ever-growing list of substitutions.
  5. Before publishing the actual source code, review it for machine-specific paths, local configuration, credentials, and other private details.

Why this project matters

This project demonstrates practical integration and debugging across Python, speech recognition, a locally hosted language model, text-to-speech, file I/O, and a real microphone/audio loop. It also records a realistic engineering lesson: a prototype can be genuinely useful without being perfect, and documenting a failed fix is better than claiming a problem has been solved when the test says otherwise.

Portfolio repository on GitHub