Storage That Outlives Services

The archive lives on hardware; hardware dies

Track one worried about services disappearing. This lesson worries about the layer under your feet: disks, controllers, enclosures, power. Every storage medium has a failure rate and a failure mode — drives wear out mechanically, SSDs lose charge when unpowered for years, enclosures fail taking their contents' availability with them. An archive that exists in exactly one place is not an archive; it is a countdown.

The classic posture is 3-2-1: at least three copies of the data, on at least two different kinds of media, with at least one copy off-site (different building, different failure domain — a cloud bucket, a relative's house, an office). The numbers are a floor and a checklist-for-thinking, not a certification: the point is that each copy should fail for different reasons. Three copies on three identical drives in the same enclosure share one power supply and one thief — that is one copy with three times the failure surface.

Choosing the layers for model-scale data

Model archives have specific shapes that constrain storage choices. They are enormous but write-once: you write tens of gigabytes once and then read rarely. That profile favors capacity-dense, sequential-friendly media (spinning disks, NAS-grade drives) over speed champions, and it makes the network-attached layer — a NAS — the natural middle tier: shared across machines, independent of any one laptop's lifecycle, and big enough that the archive does not compete with your operating system for space.

A defensible three-layer layout for a household archive:

Working layer — the laptop/desktop where acquisitions land first and verification happens.
Home layer — the NAS, on independent hardware, holding the canonical copy plus its records; two-disk redundancy (mirroring or a parity scheme) absorbs a single drive death without ceremony.
Off-site layer — whatever your scale affords: a relative's drive, an encrypted drive at the office, cold cloud storage for the truly irreplaceable minority. Updated on a slower cadence than the home layer, and verified when it updates.

Note what each layer defends against: the working layer dies with the machine; the home layer dies with the building; the off-site layer dies only with the region. Redundancy is not about more copies — it is about uncorrelated copies.

Copies that fail together count as one. Redundancy is measured in independent failure domains, not in duplicate drives. Same enclosure, same power strip, same building — same obituary.

Maintenance is part of the design

Layers imply movement, and movement implies the migration checks from the integrity track: every copy re-earns its digest. Schedule the off-site refresh at whatever cadence your tolerance for loss dictates — the answer to "how much can I afford to lose?" is a date, and everything older than it is at risk. The archive's storage design is not a purchase; it is a rhythm of verifications and refreshes that keeps the countdowns from ever aligning.

Code

Measuring your own failure domains·bash

# Enumerate where an artifact actually lives:
for d in ~/models /Volumes/archive/models ~/Library/Caches; do
  [ -d "$d" ] && du -sh "$d" 2>/dev/null
 done

# Questions to answer for YOUR setup (write them down):
# 1. How many machines hold a full copy? (working layer census)
# 2. Does any copy live on independent hardware (NAS/external)?
# 3. Does any copy live in a different BUILDING?
# 4. Are any 'copies' actually on the same physical drive
#    (partitions don't count; folders really don't)?
# 5. When was each copy last digest-verified?
#
# Copy the answers next to your pins.log -- the two records
# together describe what you own and where it survives.

Exercise

Run the failure-domain census on your own setup and write the five answers. Then draw your three layers explicitly: which device serves as working, which as home, what serves (or could serve) as off-site. If any layer is missing, write the cheapest intervention that would create it — even 'an encrypted drive at a relative's house, refreshed quarterly' counts.

Hint

The most common honest finding is: one laptop, one NAS, nothing off-site. That is two layers — the fix is usually a single external drive and a calendar reminder, not a storage redesign.

The archive lives on hardware; hardware dies

Choosing the layers for model-scale data

Maintenance is part of the design

Code

External links

Exercise

Progress

Comments 0