MouthSample
Face landmarks and blendshapes, ~30 times a second, into the ring buffer above.
Everything is divided by the inter-ocular distance, so leaning toward the camera
should move nothing. Marks on the aperture bar are the spec's acceptance
thresholds: under 0.1 at rest, over 0.7 on an open "ah".
Width wants a few seconds of talking before it means anything — it calibrates
against the range your own face actually covers.
Transcript
Each word is bound to the moment it was spoken, not the moment the recogniser reported it — so its characters carry the mouth state from ~300 ms ago. The bar under each character is its aperture; that is what becomes weight in Phase 3.
Say "and then I scream" and let it revise to "ice cream". The prefix must keep its bars exactly — the diff log shows how many words each result kept, added and dropped. This is the phase's acceptance test.
Channel A sends your microphone to Google. Everything else in this project stays on the device; transcription cannot.
diff log — newest first
bindings — spoken time, relative
Session recorder
Records every MouthSample the landmarker produces, plus the derived
intermediates the thresholds are tuned against. Mark segments while you record —
a labelled recording is a fixture, and Phases 5 and 6 will want fixtures rather
than another afternoon in front of the camera.
Every recording is sent to R2 when you press stop. Download is there for when the upload fails, not as the normal path. Nothing is kept in browser storage.
marks 0
sessions — this browser
RingBuffer.sampleAt
The acceptance ramp from the spec: 100 samples at 10 ms spacing,
value = i. Marks are stored samples; the line is
sampleAt swept continuously across a range wider than the data, so
the flat shoulders are the clamp — queries outside the retained window return the
nearest end rather than extrapolating a value that was never observed.
Timeline
Every event is pushed with the time it was noticed and how far behind reality its detector runs. The timeline subtracts that latency to recover the time it actually happened, then holds everything back by LAG. Fire the burst and watch the fast detector lose the race it should never have been in.