Benchmarking

Speed and quality both matter

A 'fast' encode that looks bad is useless. A 'high quality' encode that takes 30× real time is unusable for daily work. The right answer is on the Pareto frontier of speed × quality, and the way to find it is to measure.

Three metrics

Wall-clock encode time — time ffmpeg .... Compare to source duration; ratio is your 'realtime multiplier'.
File size — ls -lh. Same source at different presets/CRF/encoders.
VMAF — Netflix's perceptual quality score (0–100). 95+ ≈ visually lossless. libvmaf filter compares encoded output to source.

Code

Time + size + VMAF score·bash

# Encode
time ffmpeg -y -i src.mp4 \
  -c:v libx264 -crf 22 -preset slow -c:a copy \
  test.mp4

ls -lh test.mp4

# VMAF score (need libvmaf compiled in — Homebrew default has it)
ffmpeg -i test.mp4 -i src.mp4 -lavfi libvmaf=log_path=vmaf.json:log_fmt=json \
  -f null -

# Read the result:
jq '.pooled_metrics.vmaf' vmaf.json
# 96.5 = visually lossless
# 90+ = excellent
# 80+ = good for daily use
# below 70 = visible compression

Sweep — find the Pareto frontier·bash

# Sweep CRF for libx264 and dump time/size/vmaf
for crf in 18 20 22 24 26 28; do
  start=$(date +%s)
  ffmpeg -y -i src.mp4 -c:v libx264 -crf $crf -preset slow -c:a copy \
    test_${crf}.mp4 >/dev/null 2>&1
  end=$(date +%s)
  size=$(ls -l test_${crf}.mp4 | awk '{print $5}')
  ffmpeg -i test_${crf}.mp4 -i src.mp4 -lavfi libvmaf=log_path=vmaf_${crf}.json:log_fmt=json \
    -f null - 2>/dev/null
  vmaf=$(jq -r '.pooled_metrics.vmaf' vmaf_${crf}.json)
  printf 'crf=%d  %ds  %dMB  vmaf=%.2f\n' $crf $((end-start)) $((size/1024/1024)) $vmaf
done

Exercise

Pick one source clip. Run the CRF sweep above for libx264 -preset slow. Plot CRF on the x-axis, file size on one y-axis, VMAF on another. The 'knee' of the size/quality curve (where VMAF starts dropping fast) is your sweet spot. Save the script — it's the right way to choose CRF for any new content type.

Speed and quality both matter

Three metrics

Code

External links

Exercise

Progress

Comments 0