What I engineered.
I engineered every layer of Muse, from the iOS app a storyteller holds to the audio their grandchildren hear. Here it is by discipline.
It listens like
a great interviewer.
Storytellers don’t want a chatbot. Muse lets them talk, keeps them going with small acknowledgments, and saves its questions for when a story is finished. It moves through a conversation the way a skilled oral historian would, warming up, finding a thread, listening, then filling the gaps.
- runtime
- Node/WebSocket voice runtime on Railway, streaming to GPT Realtime Mini
- modes
- 5 conversational states, from warming up to filling in the gaps, each with its own prompt overlay and tool set
- prompts
- compiled from versioned sources, and CI fails the build on an uncompiled or drifted prompt
- side trips
- asking for help with the app is a detour, not a story state: the runtime goes back to the mode it left rather than moving forward
- interface
- provider-neutral
ConversationProviderport, so the speech model can change without touching the modes - evaluation
- simulated conversations, runtime benchmarks and per-turn cost regression reports
Instant replies.
Deliberate memory.
Talking to Muse feels immediate, with no awkward pause while a model thinks. But nothing a storyteller says is saved, shared or deleted on impulse. Every change to their archive goes through a separate, deliberate decision, and they only ever hear one Muse.
It feels like a conversation
Replies come in real time, and Muse handles interruptions and turn-taking naturally.
It knows what matters
Muse keeps track of the thread a storyteller is on and what’s appropriate to bring up next, across sessions.
It never acts on a whim
A quick, off-the-cuff reply can’t change or corrupt anyone’s archive.
- voice
- GPT Realtime Mini handles every turn: audio in, speech out, turn-taking and interruption
- controller
- Railway service backed by System 1 decision models, off the response path, that never speaks
- write authority
- only the controller can write stories, memories and deletions
- delegation
- EdDSA-signed JWTs, 60 s max lifetime, bound to audience, environment and session, unique
jti, verified by the tool gateway - secrets
- self-hosted Infisical, scoped per environment, so the runtime never holds a master key
Built to be trusted
with a life.
These are the promises Muse makes to every family, each followed by the engineering that keeps it.
Every memory has a receipt
When Muse says “last time you mentioned your daughter,” it can point to the moment you said it.
- provenance
- each extracted fact carries a bounds-checked source span resolved to a voice clip ID
- storage
- pgvector memory cards, consolidated by clustering and never by deleting
Your words, never invented
Muse shapes and arranges a story, but it never puts words in the storyteller’s mouth.
- structure
- story decisions operate on span indices into real clips, not generated text
- quotes
- every verbatim phrase is re-validated as a real substring before it’s used
Only family sees finished stories
Descendants see finished stories only, and only from storytellers they’re connected to.
- authn
- Supabase Auth JWTs, with user-scoped database clients
- rbac
- storyteller and descendant roles enforced by 79 RLS policies, with sensitive tables restricted to the service role
- ci
- RLS integration test run against all 245 migrations applied fresh
Forget anything, just by asking
A storyteller can ask Muse out loud to forget something. Muse confirms first, and they have a day to change their mind.
- flow
- scope is read back verbatim and confirmed, with a 24-hour undo window
- safety
- deletion queue is service-role only, and it fails closed: an error means no deletion
Nothing leaks out the side door
Personal stories don’t end up in logs, error reports or anyone’s inbox.
- telemetry
- Sentry events are rebuilt from an allow-list of fields, and production logs are metadata-only
- secrets
- self-hosted Infisical, plus AWS SSM on GPU hosts, with least-privilege IAM
- ci
- session-redaction coverage test runs on every build
An app you can use
at arm’s length.
Muse is made for storytellers in their seventies and beyond. The type is large, the paths are short, and the main screen asks for one thing only: talk. Family members get the same app with their own accent color.
- stack
- React Native and Expo on a native dev client, in TypeScript, shipping to iOS
- audio
- custom native Expo module (Swift/Kotlin) that runs mic capture and playback on one audio engine, streams raw PCM, and tracks playback drain so the app knows when Muse has actually finished speaking
- auth
- passwordless one-time codes by phone or email through Supabase Auth
- navigation
- native liquid-glass tab bar, falling back to a classic tab bar during the tour so each tab can be measured and highlighted
- legibility
- nothing on first-run screens is set in muted grey or smaller than body text, for phones held outdoors at arm’s length
- roles
- storytellers and descendants share one design system, and only the accent changes (coral or turquoise)
- quality
- 20 UI test suites, plus Sentry telemetry rebuilt from an allow-list of fields