Skip to content
C.W.K.
Stream
Lesson 03 of 04 · published

Pipelines Are Rows, Not Code

~12 min · configuration, crud, collaboration, gating

Level 0Raw Ore
0 XP0/36 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete
"This is what lets 아빠 and the Pippa sidekick co-design a new pipeline in conversation without a code deploy."

A deliberate departure

The heavy analysis in this product runs through a delegation queue: a brief is queued, a session takes it, does the work, and lands a reviewed report. A sibling application uses the same queue kernel, and there the available pipelines live in a code registry — a module listing each pipeline's stages, template and review contract.

Touchstone deliberately does not. Its pipeline definitions live as rows in the database, with full create, update, archive and restore through the engine API. The architecture doc marks it as a departure and gives the reason, which is not about flexibility in the abstract.

It is about who can design a pipeline and when. A code registry means a new kind of analysis requires editing a file, reviewing it, and deploying. Rows mean it can be designed in the conversation where the need surfaced, by the two people having that conversation, and exist immediately.

The invariant that keeps it safe

Making configuration writable at runtime is the kind of decision that goes badly when it is made casually. The safety here is a second invariant sitting right beside the first: the sidekick reads and drafts, the engine gates.

The assistant surface can read all context and propose a pipeline draft. It cannot install one. Every create and update lands through the same engine API a human uses — no privileged side channel.

The architecture doc calls that API audited, and the word is worth checking against the routes, because it is the load-bearing half of the safety argument. What the API gives you today is shared and validated: one path, same validation, no back door. What it does not give you is a trail — create, edit, archive and delete mutate the current row and emit no event, retain no prior revision, and record no actor. A version counter and an updated-at stamp tell you that something changed, not what or who. So the asymmetry that matters is real and enforced, and the audit half is an aspiration the doc states in the present tense. Normal for a young system; the mistake would be reading the doc and believing the trail exists.

That phrase is the whole safety argument in four words. The risk of a conversational configuration interface is not that the assistant proposes something bad; it is that the assistant gets a faster or quieter path than the human, so its changes are less visible. Routing both through one door means a proposal is exactly as reviewable as a human's, and the door does not distinguish between them because it does not need to. Whatever trail that door eventually keeps, it will keep the same one for both.

Give an assistant a wide read and a narrow write, through the same door as everyone else. Reading broadly is what makes a proposal good. Writing through a special path is what makes it dangerous. Keep the two asymmetric: no limits worth imposing on what it can look at, and no shortcut at all on how a change lands.

The compatibility constraint

One operational detail worth noting, because it shows how a family of small applications avoids duplicating work. This module was not written twice. It reached its second use across the family, and the two applications' versions turned out to be byte-identical once the app name was substituted — which is precisely the evidence that promotes something into the shared kit. So it was promoted, and what lives here now is a thin binding that re-exports the exact names the engine, CLI and tests already imported.

That threshold is worth stealing. One use is a solution; two identical uses is a primitive, and the byte-identical test is a much better promotion criterion than a design opinion, because it is checkable. Two independent implementations of the same idea normally diverge in small ways — a field name, a default, an ordering — and each divergence is a merge conflict deferred rather than avoided. When they have not diverged at all, extraction is a move rather than a reconciliation.

Rows do not mean unreviewed. A separate invariant covers the output: a delegation report passes a review round before it becomes a served artifact — an unreviewed report is material, not truth. So the pipeline definition is easy to create and the work it produces still has to clear a gate. Flexibility at the definition layer, rigor at the result layer.

Code

The pipeline surface — rows with CRUD, bound from a shared kernel·python
"""Pipelines are ROWS -- this product's binding of the shared kit's
PipelineRows (promoted at second use: the two applications' modules
were byte-identical after app-name substitution).

The bound surface re-exports the exact names the engine, CLI, and
tests already import."""

_rows = PipelineRows(db.qdb, prog="touchstone")

registry         = _rows.registry
get_pipeline     = _rows.get_pipeline
list_pipelines   = _rows.list_pipelines
create_pipeline  = _rows.create_pipeline
edit_pipeline    = _rows.edit_pipeline
archive_pipeline = _rows.archive_pipeline
restore_pipeline = _rows.restore_pipeline
delete_pipeline  = _rows.delete_pipeline

# Live on the engine's health endpoint -- pipelines as DATA:
#   {"ok": true, "app": "touchstone",
#    "pipelines": ["deep-dive", "market-weather",
#                  "monte-carlo", "oneoff"]}
#
# A fifth one is a row, not a release.

External links

Exercise

Find something in your systems that is configuration expressed as code — a registry, a dispatch table, a hard-coded list of job types. Ask two questions: does adding an entry require a deploy, and would anyone outside the engineering team reasonably want to add one? If both answers are yes, you have found a candidate. Then design the gate before the CRUD, not after.
Hint
The gate is the part people skip, and it is what separates this from an admin panel that becomes a liability. Decide first who may write, through which single path, what that path records, and what still has to pass review downstream — then the flexibility is safe to add.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.