AI Product / PoC
Japanese-English live captions & interpretation
A web app that turns browser microphone input into live Japanese-English captions and interpretation, displayed side by side in fixed English and Japanese panes. Silero VAD detects utterance boundaries, and Groq's low-latency inference (Whisper + Llama 3.3 70B) handles recognition and translation. A two-stage design — provisional captions on short silences, commit-and-translate on longer ones — balances fast source-text display with stable translations.
Background
In online meetings and face-to-face multilingual communication, waiting for a translation breaks the rhythm of conversation. This PoC explores whether software can deliver an experience where source captions appear while you are still speaking and a stable translation is committed each time an utterance ends — instead of the consecutive style that only translates after you finish.
How it works
In the browser, Silero VAD detects utterance boundaries while audio accumulates in a PCM buffer. After roughly 0.4 seconds of silence (a soft boundary) a provisional transcription is shown; after roughly 1 second (a hard boundary) the source text is committed and translated into the other language. STT runs on Groq's whisper-large-v3-turbo and translation on Llama 3.3 70B. The language of each utterance is detected automatically, and output goes to fixed panes — English always on the left, Japanese always on the right. To keep the API key off the client, all Groq requests go through a single Cloudflare Worker that also serves the static assets.
Key design points
The crux is the balance between committing partial output and re-translating. The earlier text appears, the more misrecognition and flicker; the longer you wait, the higher the latency. Instead of merging text fragments, this PoC re-submits the entire audio since the last commit to Whisper on every provisional update — so even if VAD cuts off mid-sentence, the next pass re-reads a longer span and naturally overwrites the earlier guess. Translation runs only on commit to keep the translated pane from flickering, and generation counters prevent late responses from clobbering committed text. STT and translation latencies are recorded in a tuning log exportable as CSV, used to tune parameters such as the silence thresholds.
Demo & write-up
A demo video and write-up are in preparation and will be published on this page.