No Mirror Flip, Ever

"A near-symmetric face and a 'keep the lower error' patch will build you a machine that randomly turns heads backward."

A Symmetric Face Is a Trap

A human face is close to mirror-symmetric, and that innocent fact is a landmine for the camera solve. Because the left and right sides nearly match, a fit where the head faces one way and its mirror image — facing the opposite way — reproject onto the landmarks with almost identical error. The two answers are geometrically opposite (yaw differs by roughly 180 degrees) but numerically nearly tied. The data barely distinguishes them.

Don't Choose Chirality by Error

The tempting patch writes itself: solve both the normal and the mirrored version, then keep whichever has the lower reprojection error. It looks like robustness. It is a coin toss. When two fits are separated by numerical noise, 'pick the lower error' is a random handedness generator — most photos land correctly, and then one shifts a landmark by a pixel and the head snaps around to face backward. This exact bug already happened in Loomis, which is why the rule is stated so bluntly: never select a mirror flip by comparing error.

Fix It in the Setup, Not the Selection

The real fix isn't a smarter selection between two candidates — it's making sure only the correct candidate is ever produced. The canonical head is defined y-up with z toward the viewer; it reaches OpenCV's camera frame through a proper 180-degree rotation about the x-axis. Build that transform correctly once, and solvePnP recovers the handedness unaided — there is no second candidate to choose between and no flip test to get wrong. This is the same discipline the whole family keeps: eliminate an ambiguity at its source rather than patching the symptom downstream.

Read Angles from Vectors, Not Euler

There is a second, sibling trap in turning the recovered rotation into readable yaw/pitch/roll. The obvious move — decompose the rotation matrix into Euler angles — wraps and gimbal-locks, producing garbage precisely on near-frontal faces where you most want a clean readout. That bug also happened. The fix is to skip the decomposition entirely and read the angles off the head's own forward and up vectors (columns of the rotation matrix), which stay stable and unambiguous through frontal views. Two traps, one lesson: when a face is symmetric or frontal, naive numerics lie — so build the geometry so the lie is never even representable.

Code

The coin-toss patch vs the structural fix·python

# WRONG (a real bug we hit): pick chirality by comparing reprojection error.
# Faces are near-symmetric, so both fits score almost the same -- this is a
# coin toss that randomly flips yaw by ~180 degrees on noisy detections.
fit_a = solve(landmarks, canonical)
fit_b = solve(landmarks, mirror(canonical))
best  = fit_a if fit_a.err < fit_b.err else fit_b   # <-- DON'T. noise decides.

# RIGHT: build the canonical -> camera transform correctly ONCE (a proper
# 180-degree rotation about x). Then solvePnP recovers handedness unaided --
# there is no second candidate and no flip test.
R, t = solve(landmarks, CANONICAL_IN_CAMERA_FRAME)

# And read angles from the head's OWN axes, never an Euler decomposition:
fwd, up = R[:, 2], R[:, 1]     # forward + up vectors, stable on frontal faces
yaw, pitch, roll = angles_from_vectors(fwd, up)   # NOT RQDecomp3x3 (it wraps)

Exercise

Find a symmetric ambiguity in your own field — a sign you can't recover from magnitude alone, two roots of a quadratic, a phase that's defined only up to 180 degrees. For your example, describe the tempting 'compute both and keep the better-scoring one' patch, argue why it's secretly random when the two are near-tied, and sketch a structural fix that makes only the correct answer representable.

Hint

The pattern is always the same: a downstream chooser between two near-identical candidates is at the mercy of noise. The structural fix injects the missing information earlier — a convention, a constraint, an orientation of the coordinate frame — so the ambiguous pair never both exist. Loomis pushed the fix into the transform; your fix pushes it into the setup too.