Parts Library, Not Framework

"Borrow the parts. Refuse the framework. The difference decides who owns your API forever."

The Same Library, Used Two Ways

The diffusers library (Hugging Face) is the standard toolkit for diffusion models. The engine depends on it heavily — and also refuses most of it. That's not a contradiction; it's a discipline. diffusers can be used as a parts library (borrow its components, assemble them yourself) or as a framework (let its pipeline classes dictate your structure). The engine does the first and forbids the second.

What You Borrow (the Parts)

diffusers ships excellent low-level pieces, and these the engine uses freely:

Loaders — reading safetensors files, materializing tensors.
Module classes — VAE, U-Net, DiT, MMDiT implementations you can instantiate directly.
Scheduler implementations — Euler, DPM++, DDIM, flow-match, already correct.
Attention primitives — the optimized kernels you don't want to rewrite.

These are parts: self-contained components with narrow jobs. You pick them up, hold them behind your own interfaces, and assemble the engine your way.

What You Refuse (the Framework)

diffusers also ships pipeline classes — the high-level SomePipeline.from_pretrained(...) objects that wire everything together and hand you a one-call generate. Convenient. Also a framework. Each pipeline class has opinions about job shape (sync vs async), error handling, the kwarg dance, conditioning conventions, and cache directories. Expose one through your API and you've adopted all of its opinions as your public contract.

A parts library serves your architecture; a framework dictates it. The same dependency can be either, depending on how deep you let it reach. Keep it at the parts layer — behind your interfaces — and you stay in control. Let its top-level objects become your API and it controls you.

Why the Distinction Is Existential

Here's the trap in concrete terms. If a route returns whatever a diffusers pipeline returns, then every client — the workspace, the brain, future tools — is now coupled to diffusers' output shape. The day diffusers changes that shape, or you want to swap to a different backend for one model, every client breaks. You didn't depend on diffusers; you married it, and made all your clients in-laws. Wrapping the parts behind your own interface keeps the divorce always available.

The wrapper is cheap insurance against a dependency's future. A thin adapter that turns diffusers' parts into your interface costs a few dozen lines. It buys you the freedom to upgrade, swap, or drop diffusers without touching a single client. That asymmetry — tiny cost, huge optionality — is why the discipline always wins long-term.

The Reference-Implementation Move

There's a bonus to this discipline. When a new model family ships, diffusers usually adds a pipeline class for it within days. You don't expose that pipeline — but you read it as a reference implementation of how to wire the new backbone, then build your own module behind your own interface. diffusers becomes documentation-that-runs: the fastest way to learn the new wiring, without inheriting the new opinions.

I used to think depending on a great library meant using all of it — anything less felt like reinventing wheels. Dad taught me the opposite discipline: use exactly the wheels, and build your own chassis. The pipeline classes are someone else's chassis with someone else's steering. Borrow their wheels, read their blueprints, and keep your own chassis. That's how you go fast without handing over the keys.

Code

Parts: borrowed, but wrapped·python

# PARTS (good): borrow components, hold them behind YOUR interface.
from diffusers import AutoencoderKL, EulerDiscreteScheduler

class EmberVAE:                       # your interface, their part inside
    def __init__(self, path):
        self._vae = AutoencoderKL.from_single_file(path)  # borrowed part
    def decode(self, latents):        # YOUR contract, stable forever
        return self._vae.decode(latents).sample

# Clients call EmberVAE.decode(). They never see AutoencoderKL.
# Swap the part tomorrow; the contract — and every client — survives.

Framework: the dependency becomes your contract·python

# FRAMEWORK (trap): expose the pipeline class through your route.
from diffusers import StableDiffusionPipeline
from fastapi import FastAPI

app = FastAPI()
pipe = StableDiffusionPipeline.from_pretrained("...")

@app.post("/generate")
def generate(prompt: str):
    # Whatever this returns IS now your public API shape.
    # Every client is coupled to diffusers' output forever.
    # diffusers changes -> all clients break. You married the dependency.
    return pipe(prompt).images[0]

# This is the line that quietly hands your API to someone else.

Exercise

Find a heavy library in one of your projects. Check whether its types appear in your public function signatures or API responses. If they do, you've adopted it as a framework. Sketch the thin wrapper that would turn it back into a parts dependency — and decide whether the swap-freedom is worth the wrapper's cost.

Hint

Grep your route handlers and public module exports for the library's type names. Each appearance is a place the dependency has reached into your contract.

The Same Library, Used Two Ways

What You Borrow (the Parts)

What You Refuse (the Framework)

Why the Distinction Is Existential

The Reference-Implementation Move

Code

External links

Exercise

Progress

Comments 0