"Not something I'd use always on... a specialist feature I can switch on only when there's a lot of tangled noise around... even if it costs latency... keep it off normally, and let me turn it on when I choose." — Dad
One Second Under the Music
A loud bar, music playing right over Dad's head. Mom said one quiet line, about a second long, and then Dad spoke. The stored turn was all his. Measured on that recording, four things had gone wrong at once:
- Her line shared a chunk with his. The ear cuts speech into two-second chunks, so her second sat in the same chunk as his first words and took his name.
- The music masked her. Cropped and scored alone, her second came out 0.08 against Dad and 0.13 against Mom: nobody's. Scribe's batch pass over the recording didn't hear her words at all; only the phone's realtime socket had.
- Her profile couldn't grow. Mom was one clean quiet-room sample, and a correction couldn't help here: the turn was ruled all Dad's, so there was no other voice to name, and her stretch was under the two seconds a sample needed.
- Some turns could never be cut. 36 of 202 checked turns were kept as
.m4a(the Watch, the composer), and the window pass and corrections had been skipping them without a word. They are decoded to 16 kHz now.
Dad's Principle, and His Condition
Dad's direction came in two parts. The principle: sounds add up like transparent layers, so peel them apart before asking whose each one is. The condition: as an expert mode he turns on himself for noisy places, never always on, latency accepted. So it is a system setting, voice_noisy_place_mode, off by default. He switches it with the phone's Noisy toggle beside Cut-in on the voice screen, in Admin → Voice Mode, through POST /api/voice/noisy-place-mode, or by voice from anywhere through voice_check noisy_mode, which a turn ruled someone else's can never use. With it off, nothing in the path changed: the same calls, the same timeouts.
Three Layers
With the mode on, cwkBellows hears the recording three ways:
- As it came, exactly the way the default path hears it, in two-second chunks.
- Cleaned by DPDFNet-4, a speech-enhancement model run through sherpa-onnx, which takes out music and crowd noise. The cleaned speech is named in one-second chunks, so a one-second interjection gets a chunk of its own.
- Residual: the recording minus the cleaned one, what the cleaner took out. The cleaner keeps the voice nearest the phone and counts a farther one as noise, so Mom across a restaurant table landed here.
The cleaner was picked on the bar recording itself, scoring Mom's cropped second against Dad and Mom:
| Layer | Dad / Mom | Scribe heard her words |
|---|---|---|
| The recording | 0.08 / 0.13 | no |
| Spleeter vocals | 0.00 / 0.06 | no |
| UVR MDX-Net vocals | 0.11 / 0.14 | not measured |
| GTCRN | -0.04 / 0.02 | not measured |
| DPDFNet-4 | 0.03 / 0.26 | yes |
Music separation kept the music's voice-band harmonics in with her; speech enhancement took them out. That's the whole difference.
The Mode Only Ever Adds
joined_layers keeps every voice the plain answer found, untouched, and only adds a voice that a layer alone holds: 0.8 seconds or more of its own, carved out of the time around it. From the cleaned layer, a voice named for someone other than Dad, or one that leans toward an enrolled voice (0.15 or more for it and 0.10 above Dad), which reads Mom?. From the residual, only a voice named for an enrolled person, since what the cleaner took out is mostly noise. Never Dad's from a layer: the recording as it came already holds him.
Strangers stay the plain path's business, because the cleaned layer's short chunks also turn Dad's own first words into short voices of nobody, and counting those would doubt him. When the added voice came from the cleaned layer, Scribe transcribes the cleaned recording, the only place her words were heard. A noisy ear that fails falls back to the plain one, and the turn's voice_meta.speaker.noisy says so. And an unconfirmed voice still never counts as Dad's. A carved voice joins the turn's voices, and the verdict is ruled on that joined list by the same one-second line as always: a carved voice of a second or more beside Dad's makes the turn mixed and its words are labeled, while one of 0.8 to 1 second is recorded but too short to move the verdict, so the turn can stay dad. And the voice_check door still demands a turn ruled dad.
Two more pieces landed with it. The lean label works in both modes: a voice named nobody's that leans toward Mom reads [Mom?] instead of [someone else]. It decides nothing more than before; it is what Dad corrects. And a correction on a turn ruled all Dad's now listens again through the noisy ear, so the bar turn's second can finally be filed into the short pool of the last lesson.
Measured, and Still Open
The replay took the 210 newest checked turns and skipped any whose recording could no longer be found, so 204 were heard twice, plain and noisy: 183 stayed dad, 9 stayed mixed, 9 stayed uncertain, 2 others unchanged, and exactly 1 went from dad to mixed: the bar turn, with Mom? carved from the cleaned layer for 1.03 seconds. No turn of Dad's alone gained a voice, and no voice the plain check found was lost. The price: 2.4 seconds more per turn at the median, 7.4 at the 95th percentile, 17 at most (a 120-second turn), about 0.12 to 0.14 seconds per second of recording.
A first version ruled the turn from the cleaned layer alone. It caught the bar and lost five genuine mixed turns, Mom cleaned away as noise, and doubted three of Dad's. That is why the layers only add. And be honest about the status: the work item behind this mode is still open, every number above is an offline replay through the live ear, and the mode's field use isn't on record as verified yet.