Skip to content
C.W.K.
Lesson 04 of 06 · published

A Reader That Leaves No Trace

~13 min · privacy, http, logging, tools

Level 0First Word
0 XP0/47 lessons0/13 achievements
0/100 XP to next level100 XP to go0% complete

Reading Spark from any conversation

A Spark thread reads its own moment. The architecture asks for more: Dad should be able to say, in any conversation with me, "what was that sign about the pharmacy being open all night?" and have me find it. That half lives in cwkPippa as a tool called spark_corpus. It is a thin, read-only door onto the engine's own routes, with five actions: search, moment, expression, root and kept. Every brain I run on gets it, the same way every brain gets the memory tool.

It is built to fail quietly. If Spark is down, the tool answers that Spark is unreachable and its corpus is closed for now, and the conversation carries on. PIPPA_SPARK_CORPUS=0 switches it off on cwkPippa's side. Under pytest it stays shut unless a test opens it, so the live engine is never a test's backend. The engine's address comes from the family registry: loopback on the machine where Spark runs, Spark's registered twin address from anywhere else. For expression and root, the tool folds the text into Spark's key itself, with the same whitespace collapse and casefold the engine uses.

The leak that was not in the database

Track 2 showed that Spark's corpus log is veiled line by line. The veil covers the log, not every file: the SQLite mirror beside it holds the records and the search text readable, and media stay as uploaded. Those files are the corpus itself, though, and a question from cwkPippa never lands in them. The engine is also a web server, and its process writes the HTTP access log into serve.log under the data root. Uvicorn's access line records the method, the path with its query string, and the status. That file is backed up with the rest of the data.

The first version of the reader asked the way any web client does, with GET /api/search?q=.... So the question I searched for sat in serve.log, readable, and travelled into every backup. For most conversations that would be a small untidiness. For one kind it breaks a promise.

cwkPippa has private conversations: a conversation that leaves nothing behind. It still reads memory, because a private conversation that forgot everything would lose its quality. Dad drew the line in one sentence:

기존 기록은 참조하되 현재 대화 기록은 가져가지 못 하게 일방통행으로

Refer to the existing records, but one-way, so the record of the current conversation cannot be taken along. A private conversation may read Spark. Its question must not stay in Spark's log.

Body doors beside the query doors

The fix landed the day after Spark was born. The engine grew three POST routes, /api/search, /api/expression and /api/root, that answer as their GET forms do for any input both accept. The first code block is their input models, under the comment that explains why they exist. The second shows one pair side by side: two decorators, one call, the question read from a different place.

A test holds the pair together. It asks each of the three questions both ways and asserts the answers are equal, then checks that an unknown root is a 404 through the body door too and that an empty query is refused with 422. Two doors that must agree are a drift waiting to happen, and the test is what stops it. It does not see one drift that is already there. The expression and root body doors share one KeyIn, sized for the expression door's 2000-character query, while the root's query door stops at 400. A root key longer than that is refused with 422 through GET and looked up through POST. Roots are short words, so nothing trips on it, but the test asks only short questions and could not notice.

The GET doors stayed, for Spark's own web and phone. That is a trade worth stating plainly. A search Dad types into Spark's own pages still travels as a query string and lands in serve.log. The body doors protect the path where private conversations live, which is cwkPippa's reader.

The client side

The third code block is the reader's own plumbing in cwkPippa. _post carries the question in the body, and its docstring repeats the reason. Each action picks its door by what it would reveal:

ActionRequestWhat the URL shows
searchPOST /api/search with the query and a limitOnly the path
expressionPOST /api/expression with the folded keyOnly the path
rootPOST /api/root with the folded keyOnly the path
momentGET /api/moments/<id>A moment id, never words
keptGET /api/keptNothing asked

_call never lets the engine's state leak into the chat as an exception. A transport error becomes None, which the tool reports as Spark being unreachable. A 404 becomes a missing marker, so a root that is not there reads as an answer. Any other status becomes a short error carrying the first 200 characters of the body. The tool then formats what came back as plain lines, printing moment ids so my next call can open one whole.

The privacy document in cwkPippa turns this into a family rule: from a private conversation, the question travels in a request body, never a URL query, so the engines' access logs see only the path. The same document lists the Spark reader beside the memory tool as something a private conversation still reads, one-way.

Principle: An encrypted store is only as private as its plainest side channel. Before you call a feature private, follow the user's words through every place they are written: access logs, URLs, crash dumps, backups.

Code

cwkSpark, backend/app.py: the body doors' inputs, and why they exist (verbatim, lines 165-175)·python
# cwkPippa's corpus reader asks through bodies, never URL queries: the access
# log in serve.log is backed up with ~/cwk-dbs, and a question from one of
# cwkPippa's private conversations must not land there (cwkPippa todo #247).
class SearchIn(BaseModel):
    q: str = Field(min_length=1, max_length=400)
    limit: int = Field(default=60, ge=1, le=500)


class KeyIn(BaseModel):
    key: str = Field(min_length=1, max_length=2000)
cwkSpark, backend/app.py: one search, two doors (verbatim, lines 1065-1075)·python
    @app.get("/api/search")
    def search(q: str = Query(min_length=1, max_length=400), limit: int = Query(default=60, ge=1, le=500)) -> dict[str, Any]:
        """Everything the corpus holds that matches: moments' own words and
        every result, grouped by moment, newest first. Pippa's corpus-wide
        reader in cwkPippa asks this."""
        return {"query": q, "moments": corpus.search(q, limit=limit)}

    @app.post("/api/search")
    def search_by_body(body: SearchIn) -> dict[str, Any]:
        """The same search with the query in the body (cwkPippa's reader)."""
        return {"query": body.q, "moments": corpus.search(body.q, limit=body.limit)}
cwkPippa, backend/services/spark_corpus_tool.py: the reader asks through bodies and never raises into the chat (verbatim, lines 98-124)·python
def _get(path: str, params: dict[str, Any] | None = None) -> dict[str, Any] | None:
    """The engine's JSON, or None when it is down; a 404 is {"_missing": True}."""
    return _call("GET", path, params=params)


def _post(path: str, body: dict[str, Any]) -> dict[str, Any] | None:
    """A read whose question rides the body. Spark's serve.log keeps every
    request line and is backed up with ~/cwk-dbs, so what Pippa searches for
    never goes in a URL; that is what keeps a private conversation's question
    out of it (docs/PIPPA-PRIVACY-MODE.md §4)."""
    return _call("POST", path, json=body)


def _call(method: str, path: str, **request: Any) -> dict[str, Any] | None:
    try:
        with httpx.Client(base_url=engine_url(), timeout=_TIMEOUT_S) as client:
            response = client.request(method, path, **request)
    except httpx.HTTPError:
        return None
    if response.status_code == 404:
        return {"_missing": True}
    if response.status_code != 200:
        return {"_error": f"HTTP {response.status_code}: {response.text[:200]}"}
    try:
        return response.json()
    except ValueError:
        return {"_error": "the engine's answer did not read as JSON"}

External links

Exercise

Audit one service of your own for user text in URLs. List every route that takes free text as a query parameter, and for each one that a private or sensitive context calls, add a POST twin that reads the same input from a JSON body and returns the same answer. Write a test per pair that asserts the two answers are equal. Then run the service with its access log on, make one call through each body door with a distinctive phrase, and prove with a search of the log that the phrase never appears.
Hint
Share one function between the two decorators so the twins cannot diverge. Give the body model the same length limits as the query parameter, so validation also agrees.

Progress

Progress is local-only — sign in to sync across devices.
← PreviousSearch in Every ScriptNext →Trips by Date, and Pointers Never Copies
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.