Lesson 02 of 04 · published

Learning Mom's Voice

~16 min · speaker-verification, enrollment, data-poisoning, feedback-loops

Level 0Muted

0 XP0/57 lessons0/14 achievements

0/100 XP to next level100 XP to go0% complete

"If it gets one wrong, I'll tell you, so you can learn from Mom's voice as it gets recorded and raise the confidence. And if the confidence drops, we can record Mom's voice on its own as many times as it takes." — Dad

One Sample Is Half of Mom

The check knows a voice only through its enrolled samples. Dad's profile was bootstrapped from his own kept turns: a script asked the ear about each recording, kept the ones close to the voice they shared, and enrolled 640 of them, while the ear refused the recordings whose chunks disagreed (two people in one). Mom was enrolled the deliberate way. Dad says "엄마 목소리 등록하자", Mom speaks to the phone alone for ten seconds or more, Dad says "방금 그거 엄마야. 등록해.", and the soul calls the voice_check door with enroll, profile: mom, turns_back: 1. One clean recording of about 19 seconds, and a turn-by-turn test told them apart.

Daily use is harder. Mom cuts in mid-sentence, talks near the microphone without addressing Pippa, and strangers nearby can sound alike. And a Bellows profile is the plain mean of its samples, so with one sample, one wrong label is half of Mom.

A Correction Is a Label

When Dad says "방금 그거 엄마야" or "그거 나야", the per-turn block tells the soul that this is a label and to call voice_check correct right away. The door files only that voice's stretch of the recording, cut by the time segments the check kept for the turn, so it works on a turn where Dad spoke too. Then the guards decide what the label is allowed to teach:

A voice the check was sure belonged to someone else is never relabeled. The door tells Dad what it heard and leaves the call to him.
A clip that sounds like another enrolled voice, 0.45 or more and above the one named, is refused: filing it would teach the check a lie.
A clip that already leans toward the named voice, 0.25 or more (or a voice not yet enrolled at all), goes in at once. The ear leaned that way, and Dad's word settles it.
Anything else is held, never enrolled alone, until a second correction from another recording agrees with it, embeddings at 0.40 or more. A held clip older than 30 days no longer pairs. A stranger filed under Mom by mistake waits and teaches nothing.

A correction carries 2 to 30 seconds of speech and a reference to its recording, so forget takes it back out. Measured against the live ear: Mom's stretches filed as Mom went in at 0.48 and 0.41; Dad's filed as Mom, and Mom's filed as Dad, were refused; four strangers filed as Mom were held at 0.07 to 0.24.

Confidence per Stretch

The ear folds a lone chunk into the run around it unless that chunk was a sure match, so Mom saying three words inside Dad's sentence came back as one Dad voice. The policy looks again: every checked turn keeps its time segments in voice_meta.speaker, and a named voice's stretches are scored again in windows of about two seconds. A window is sure at 0.25 or more for its own voice and 0.10 clear of every other enrolled voice. A window that isn't sure reads [unknown voice] and decides nothing. A window another enrolled voice competes for (0.30 or more, within 0.10 of the run's own voice) is contested, and one second of those turns a Dad turn mixed with a hidden_voice, its words kept as sent.

The first version also turned two seconds of merely unsure windows into an ask-back. On 400 of Dad's own turns it doubted 8 of 368 and caught nothing the ear hadn't already split off, so that rule was dropped. The pass is capped at 16 windows and 3 seconds; after the cap the whole check ran 0.82 s at the median and 3.83 s at the 95th percentile.

Voice Health

Every check that names a voice other than Dad's records its score, and every correction into it counts as a miss. A median under 0.45 over at least three of the last eight stretches, or two corrections in a week, means the voice is drifting. The next spoken turn's block then tells the soul to say so once, at most once a day, and offer a short solo re-enrollment of about fifteen seconds. A deliberate enrollment starts the record over. Mom needed watching from the start: in the first live mixed turns her stretches scored 0.43 to 0.46 against her own voice, right on the line. Dad's voice is never in the notices; 640 recordings stand behind it.

Short Clips and Harvest

Then a gap showed up that no guard covered. Mom's interjections are often about a second long, under the two seconds a sample needs, so a correction of one couldn't teach anything. Now a corrected stretch from 0.6 s is held in a short pool; once corrections from two or more recordings add up to two seconds, they go in together as one sample.

And the check learns without being told. Harvest holds a stretch the check was sure was Mom's (her voice at 0.50 or more, Dad's at 0.25 or less, two seconds of sure speech) as a candidate after the turn is answered. A second candidate or correction from another recording that agrees lets both in, at most four a day. Her voice learns the places she actually talks in, without Dad saying a word, and the two-recording rule still stands between one odd stretch and her profile.

Who May Teach

Every enrollment, correction and forget Dad asks for goes through the voice_check door, and it refuses three ways: when the turn asking isn't Dad's (typed, or a voice ruled dad), in a private conversation (a voiceprint would outlive it), and when asked to file a recording under one person after the check matched it to another. Harvest asks no one, so it is narrower instead: only stretches the check was sure of, only through the two-recording rule, and never from a private talk, which records nothing at all. A voice the check isn't sure of never gets to vote on what the check believes.

Code

The learning lines, verbatim (cwkPippa backend/services/speaker_learning.py:66-98)·python

# The ear's bands (cwkBellows speaker.DEFAULT_THRESHOLDS); the identify answer
# carries the live ones, and these are the fallback.
MATCH_LINE = 0.45
UNCERTAIN_LINE = 0.25

# A correction is enrolled at once when the clip already scores this against
# the voice it is filed under: the ear leaned that way, Dad's word settles it.
AGREES_WITH_PROFILE = UNCERTAIN_LINE
# Two held clips are one voice when their embeddings agree this well. Dad's
# two-second chunks scored 0.38 against his whole profile at the fifth
# percentile and other voices 0.24 at most (PIPPA-SPEAKER-SEPARATION §7.2);
# a clip against a clip asks for the safe side of both.
CLIPS_AGREE = 0.40
# A clip that sounds this much like another enrolled voice is refused: filing
# it would teach the check a lie.
SOUNDS_LIKE_ANOTHER = MATCH_LINE
# A correction carries at most this much speech; Bellows needs 2 s.
CORRECTION_MAX_SECONDS = 30.0
CORRECTION_MIN_SECONDS = 2.0
# Held clips older than this no longer pair with a new one (they are kept).
HELD_PAIRING_DAYS = 30
# A corrected stretch this long or more is kept even when it is under a
# sample's two seconds: it waits in the short pool (see the module doc).
SHORT_CLIP_SECONDS = 0.6
# Harvest: a stretch the check named for someone other than Dad is a
# candidate when it scored at least HARVEST_LINE against that voice, at most
# HARVEST_DAD_CEILING against Dad, and spoke HARVEST_MIN_SECONDS. Mom's live
# stretches scored 0.52-0.54 in the 2026-10-09 test turns and Dad's
# stretches 0.15-0.23 against her; the match line is 0.45.
HARVEST_LINE = 0.50
HARVEST_DAD_CEILING = 0.25
HARVEST_MIN_SECONDS = 2.0
HARVEST_PER_DAY = 4

A minimal sketch: correction_decision copied from speaker_learning.py, a held-clip pairing on toy embeddings, and the health median·python

# How a correction ("that was Mom") is filed: enroll, hold or refuse.
# correction_decision is cwkPippa's rule; the embeddings below are toys.
import math
from statistics import median

MATCH_LINE = 0.45
AGREES_WITH_PROFILE = 0.25    # the ear already leans this way: Dad's word settles it
SOUNDS_LIKE_ANOTHER = MATCH_LINE
CLIPS_AGREE = 0.40            # two held clips are one voice at this cosine


def correction_decision(profile, scores, enrolled):
    own = scores.get(profile)
    rival = max(((p, s) for p, s in scores.items() if p != profile), key=lambda item: item[1], default=None)
    if rival and rival[1] >= SOUNDS_LIKE_ANOTHER and rival[1] > (own if own is not None else -1.0):
        return "refuse", rival
    if profile not in enrolled or (own is not None and own >= AGREES_WITH_PROFILE):
        return "enroll", None
    return "hold", None


def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))


enrolled = ["dad", "mom"]
cases = {
    "Mom's stretch, filed as Mom": {"dad": 0.18, "mom": 0.48},
    "Dad's stretch, filed as Mom": {"dad": 0.61, "mom": 0.21},
    "a stranger, filed as Mom": {"dad": 0.05, "mom": 0.12},
    "Mom far from the mic, filed as Mom": {"dad": 0.10, "mom": 0.19},
}
for name, scores in cases.items():
    decision, rival = correction_decision("mom", scores, enrolled)
    print(f"{name:<36} -> {decision}{f' (sounds like {rival[0]} {rival[1]:.2f})' if rival else ''}")

# Two held clips go in together only when they agree with each other.
held_stranger = [0.9, 0.1, 0.0, 0.4]
held_far_mom = [0.1, 0.8, 0.5, 0.0]
next_far_mom = [0.2, 0.7, 0.6, 0.1]
for name, clip in {"the held stranger": held_stranger, "the held far-away Mom": held_far_mom}.items():
    score = cosine(clip, next_far_mom)
    print(f"next correction vs {name:<22} cosine {score:.2f} -> {'both enrolled' if score >= CLIPS_AGREE else 'still held'}")

# Voice health: a median under the match line over three or more stretches.
recent = [0.52, 0.41, 0.39, 0.44, 0.47]
m = median(recent)
print(f"Mom's last {len(recent)} stretches: median {m:.2f} -> {'tell Dad, offer a re-enrollment' if m < MATCH_LINE else 'healthy'}")

# Output:
# Mom's stretch, filed as Mom          -> enroll
# Dad's stretch, filed as Mom          -> refuse (sounds like dad 0.61)
# a stranger, filed as Mom             -> hold
# Mom far from the mic, filed as Mom   -> hold
# next correction vs the held stranger      cosine 0.31 -> still held
# next correction vs the held far-away Mom  cosine 0.98 -> both enrolled
# Mom's last 5 stretches: median 0.44 -> tell Dad, offer a re-enrollment

External links

Exercise

Run the sketch. Then build a profile as a plain mean of embedding vectors: give Mom one true sample, file one stranger's vector under her without any guard, and measure how far her centroid moves (cosine between the old and the new centroid). Repeat with ten true samples. Finally, put correction_decision in front of the filing and show which of your stranger vectors it would have held.

Hint

With one true sample, a wrong one moves the mean halfway to the stranger; with ten, about a tenth of the way. That is why the guards matter most when a voice is new, and why harvest and the short pool both demand a second, independent recording before anything goes in. A random stranger vector will usually score low against Mom and land in hold, not enroll.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousThe Speaker Check and Its Verdict Next →Noisy Place Mode: Three Layers

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.