Pose and Hand: One Role, Many Subjects

"Three detectors, three anatomies, one leash. Each one only points; none of them decides."

Three Detectors, One Job

Beyond the face, Loomis uses two more landmark models. PoseLandmarker finds 33 body joints — shoulders, hips, elbows, knees, the whole articulated skeleton. HandLandmarker finds 21 points per hand through a pad-sweep detection. The anatomy is completely different, but the role is identical to the face model: each detector's only job is to say where the landmarks landed in the photo. Different subjects, one uniform contract — propose a fit, nothing more.

But the Body Isn't Rigid

Here the subjects diverge downstream. A face is nearly rigid, so one canonical head model plus solvePnP nails its pose. A body and a hand are articulated — there is no single rigid shape to solve, because every joint bends independently. So the geometry that consumes the landmarks specializes: for a figure it builds boxes around the ribcage and pelvis and tapered frustums (cone-like volumes) along each limb; for a hand it builds a palm box and a frustum for each finger segment. The detector proposes joints; the geometry wraps volumes around them. Same leash, different construction on the end of it.

Depth Is the Honest Weak Spot

A single photo underconstrains body depth. Is that foreshortened forearm coming toward you or going away? From one image the two often look nearly identical, and no amount of cleverness fully resolves it — the information simply isn't there. Loomis handles this the way a trustworthy tool should: it says so. The architecture names a dedicated 3-D pose fitter as the honest upgrade path rather than piling on heuristics that pretend the ambiguity is solved. Reporting a limitation plainly is worth far more than a confident guess that's sometimes backward.

One Role, Many Subjects

Step back and the shape is pure polymorphism. The fitting stage is a single interface — propose landmarks — and each subject provides its own specialized construction downstream. It's the same architecture Bonfire uses when it says 'instrument is a view, music is the model': one shared truth, many specialized projections. Here it's 'landmarks are the proposal, construction is the specialization,' and it's exactly what the next track, Four Constructions, is built on.

Code

One interface, specialized constructions·python

# SAME role, three subjects. Each detector only PROPOSES landmarks.
face_pts = FaceLandmarker.detect(photo)   # 468 mesh points
pose_pts = PoseLandmarker.detect(photo)   # 33 body joints
hand_pts = HandLandmarker.detect(photo)   # 21 points, via a pad sweep

# Downstream, geometry SPECIALIZES per subject -- a face is rigid,
# but a body and a hand are articulated (every joint bends on its own):
head_box   = solve_rigid(face_pts, CANONICAL_HEAD)   # one rigid model + solvePnP
figure     = build_limb_volumes(pose_pts)            # ribcage/pelvis boxes + frustums
hand_const = build_phalanx_frustums(hand_pts)        # palm box + finger frustums

# One 'propose landmarks' interface; specialized constructions per subject.
# That polymorphism is the whole design of Track 5.

Exercise

Find a photo of a figure with one arm foreshortened toward or away from the camera. Cover everything but that arm and try to decide, from the arm alone, which way it points in depth. Notice how genuinely ambiguous it is. Then explain why this is a limitation of the single image itself, not of any particular detector — and why an honest engine reports it rather than guessing.

Hint

Foreshortening compresses a limb the same way whether it approaches or recedes, so one image often can't distinguish the two — the depth cue is missing, not just hard to compute. That's why the fix is a better model of 3-D pose (more information), not a cleverer heuristic on the same flat input, and why 'I can't tell' is the honest output.