Which Machine for Which Job

"The rivalry is not a winner. It is a table with jobs down the side, and each row has an answer that the earlier lessons already computed."

The Table

Five lessons of arithmetic reduce to one question per job: where does the model land — fits, spills, splits — on each machine, and what is the stage the job spends its time in. The code block is that question as a function, with the household's own machine shapes and a rented card, and the table below is what it returns for the jobs a household actually has. Every cell is a ceiling from a vendor bandwidth or a boolean from a vendor capacity; the ranking is this quest's judgment, and the judgment column says so.

Job	Decisive fact	Answer	Evidence
One user, a model that fits a card (≤ 27B at 4 bits)	bandwidth ratio 2.2× the Studio, 4× on an H100	the card — 124 vs 57 tok/s ceiling; the Mac is not the fast choice here	derived from vendor bandwidth
One user, a model above 96 GB (a 235B mixture at 132 GB resident, a 405B at 228)	fits the 512 GB pool; the 235B also fits a pair of 96 GB cards, the 405B does not	the Mac Studio — the only single machine on the table that runs either, at 270 W and no fan you can hear	derived; vendor (power, Apple's test)
One user, a 70B at 4.5 bits per weight	fits a 96 GB workstation card at 45 tok/s, the Studio at 21, the Spark at 7; spills on a 5090 at 1.6	the workstation card if you have the room and the power; the Studio if you want it on a desk	derived
Many users at once, any model that fits	batch turns decode into compute; tensor cores win compute	NVIDIA, and the larger the batch the more so — the batch lesson measured the Mac's aggregate at 7× single-stream, a card's goes further	measured (Mac), physics
Training or a full fine-tune at any real scale	16 bytes per parameter; fifty-fold tensor compute	NVIDIA, rented or racked; the Mac for adapters and small models	physics, vendor throughput
Image generation, one user	compute stage; a card's tensor cores against the Mac's 22 TFLOP/s	the card is faster per image; the household runs it on the Mac because the same box serves everything else and the images are not urgent	measured (Mac), our-judgment
Always-on, in a room, for years	9 W idle, 270 W max, silent; against 575–1,000 W and a case fan	the Mac — the fleet track is this row	vendor (Apple support), vendor (NVIDIA)
Speech and voice for the family	quality of the frontier cloud model, not local capability	the cloud, by choice — the fleet track's cloud-by-choice lesson	the household's ruling

How the Household Answered

The fleet in Part 4 is this table, filled in. The inference hub is a Mac Studio because the models the household wants to run sit in the second row and the seventh. The image engine runs on the same Studio because the sixth row's speed difference did not matter to the way the family uses images. Nothing in the household trains at scale, and Apple's own models are trained elsewhere, so the fifth row has no local machine and does not need one. Voice went to the cloud on the last row's reasoning, which the household stated as a ruling: better quality, not missing capability. And every Mac in the house is an always-on machine in a room where people live, which is the row the rival rarely wins — the 140 W Spark is the one rival built for it — and the quest's last track is about. None of this makes the Mac the better computer. It makes it the right one for the rows this household is in — and the card the right one for the rows it is not, which is why the table is printed in a quest about Apple silicon.

Code

which_machine.py — the decision as a function of the model, the streams, the training, and the room·python

#!/usr/bin/env python3
"""Which machine for which job -- the decision the physics track licenses, as a function.
Inputs: the bytes a token READS (GB, at your bits), the bytes that must be RESIDENT (GB;
the same number for a dense model, the whole file for a mixture), how many streams at
once, the parameter count if you train, and whether the box must sit in a room with people.
Every rule below is arithmetic from earlier lessons; the ranking is this quest's judgment."""

MACHINES = {  # name: (memory GB, GB/s, watts max, quiet in a room?)
    "MacBook Pro M5 Max 128 GB":      (128,  614,  140, True),
    "Mac Studio M3 Ultra 512 GB":     (512,  819,  270, True),
    "RTX 5090 workstation":           (32,   1792, 1000, False),
    "RTX PRO 6000 workstation":       (96,   1792, 1000, False),
    "DGX Spark":                      (128,  273,  140, True),
    "rented H100 (cloud)":            (80,   3350, 700, True),      # quiet because it is elsewhere
}


def decide(read_gb, resident_gb=None, streams=1, train_params_b=None, in_room=True):
    resident_gb = read_gb if resident_gb is None else resident_gb
    rows = []
    for name, (mem, bw, watts, quiet) in MACHINES.items():
        fits = resident_gb + 4 <= mem * 0.9
        ceiling = bw / read_gb if fits else 63 / read_gb          # spill: PCIe 5 x16 carries the spilled bytes
        note = []
        if not fits: note.append("spills over PCIe" if "RTX" in name else "does not fit")
        if train_params_b and mem < train_params_b * 16:          # 16 bytes per parameter before activations
            note.append(f"no full fine-tune ({train_params_b}B x 16 B/param = {train_params_b * 16:.0f} GB)")
        if in_room and not quiet: note.append("loud/hot in a room")
        if streams > 1: note.append(f"batch x{streams}: compute-bound, favours tensor cores")
        rows.append((name, fits, ceiling, note))
    rows.sort(key=lambda r: (r[1], r[2]), reverse=True)
    print(f"reads {read_gb:.1f} GB/token, resident {resident_gb:.0f} GB, streams {streams}, "
          f"train={train_params_b or '-'}, in a room={in_room}")
    for name, fits, ceiling, note in rows:
        print(f"  {name:30} {'fits ' if fits else 'NO   '} ceiling {ceiling:6.1f} tok/s   {'; '.join(note)}")
    print()


decide(14.4)                                   # a 27B at 4 bits, one user
decide(39.7)                                   # a 70B at 4.5 bits/weight, one user
decide(12.4, resident_gb=132)                  # a 235B mixture, 22B active, at 4.5 bits: all experts resident, 22B read
decide(14.4, streams=32)                       # serving 32 users a 27B
decide(4.5, train_params_b=9)                  # fine-tuning a 9B for real

# reads 39.7 GB/token (a 70B at 4.5 bits/weight), one stream, in a room:
#   rented H100 (cloud)            fits  ceiling   84.4 tok/s
#   RTX PRO 6000 workstation       fits  ceiling   45.1 tok/s   loud/hot in a room
#   Mac Studio M3 Ultra 512 GB     fits  ceiling   20.6 tok/s
#   MacBook Pro M5 Max 128 GB      fits  ceiling   15.5 tok/s
#   DGX Spark                      fits  ceiling    6.9 tok/s
#   RTX 5090 workstation           NO    ceiling    1.6 tok/s   spills over PCIe; loud/hot in a room
# reads 12.4 GB/token, resident 132 GB (a 235B mixture, 22B active): only the 512 GB Studio fits, ceiling 66 tok/s
#   by the bytes a token reads -- dividing by the whole 132 GB would say 6, and that is the dense mistake
# reads 4.5 GB/token, train 9B: every card, the Spark and the 128 GB laptop say 'no full fine-tune (144 GB)'; the Studio does not

Exercise

Fill the table for yourself: your three real jobs, the decisive fact for each from this track's lessons, and the answer. Run which_machine.py with your numbers and compare its first row to your answer. Where they disagree, write which weight — room, power, noise, capacity, speed — you applied that the function did not, and put that weight on the card as your own.

Hint

The function knows bytes, bandwidth, capacity and watts. It does not know that the machine will sit next to a bed, that the model you want ships next month, or that you will never train. Those are the arguments that turn a table into a purchase, and the card is where they belong.

The Table

How the Household Answered

Code

External links

Exercise

Progress

Comments 0