9 Finding
Forty-two is a perfectly good answer. The trouble, as the mice discovered, is that nobody knew the question. Recall has the opposite trouble: you know the question, and the answer was written down by somebody who’d never heard it. The engram says the retry queue drops jobs older than twenty-four hours. The person on call at three in the morning types “why do payments go missing”. The two don’t share a single word, and the engram is the answer.
Closing that gap is what this chapter is about. Chapter 3 called the beat remember and chapter 6 watched the agent walk the corridors on its first day. What’s left is the craft: how a search ranks, how an agent gathers the neighbourhood around a hit before it starts work, how it asks what was true last June instead of what’s true now, and why a colleague who recalls costs less than one who reads everything. The retry-queue engram runs through every section.
9.1 Search as recall
A search in Crystalline blends two signals, and the blend is what makes the three-in-the-morning question land. Full-text matching finds the exact: the engram that says “retry queue” when you searched for “retry queue”. Semantic similarity finds the adjacent: the engram about dropped jobs when the question was about missing payments, because the two are about the same thing in different words. Each hit is scored on both and the ranking is the blend. Text alone would miss the on-call question. Similarity alone would rank a chatty engram that mentions payments above the terse one that explains them. Together they behave like a colleague who both remembers the phrase and understands the problem.
Meaning doesn’t stop at a language: the model behind the similarity reads more than two hundred of them, so an engram written in German answers a question asked in English. Your shelf note on translations, typed in German one evening (English when a book was written in English, German translations for everything else), is what comes back months later when you ask in English which edition of a French novel to buy.
Two small priors tilt the order, and both are bounded. Salience lifts the exceptional: an engram given a salience at capture, or raised later when it turned out to unlock a task, ranks above equally relevant neighbours, while relevance keeps the upper hand and nothing is ever excluded by it (Section 4.2). Retirement fades the past: an engram whose status is deprecated, superseded, archived or legacy is softly pushed down, never filtered out. So the cutoff that applied until September sits one scroll below the behaviour that applies now, still there and still dated (Section 4.4).
Around the ranking sit the filters. A search narrows by domain, by type, by tags, by status, by any frontmatter key or by a recorded-after date. A search with no query text at all is a legitimate way to ask for everything of one kind: every runbook in engineering, every engram tagged weyland-yutani. A tag filter also matches through a domain’s tag aliases, so an old spelling keeps finding what it always found. A filter can also match one element of a frontmatter list, so one search finds every engram whose sources list names a given page, and a key with a single value counts as a list of one. Every hit is labelled with its domain, and a hit inside an observation carries its line, so the agent can point at the fact rather than the file.
What comes back is a snippet, a short window around the match, often cut off, and it’s a targeting aid. A colleague who answers from a snippet is guessing, and the routing skill draws the line where it belongs: a single atomic fact completely visible in the snippet may be answered from it. Anything that summarizes, compares or quotes an engram goes through read_engram first. When several domains answer a broad question differently, the agent says so and names which domain each answer came from. Scope itself is chapter 6’s decision: narrow to the domain whose routing line fits, sweep for a broad question and widen when the obvious domain keeps returning near-misses.
Fluid’s search shows you what the agent finds: the search page is faceted and backed by exactly the ranking an agent sees (Figure 15.2). Cmd+K is the other door: it jumps anywhere by name. The two answer different questions. Search is for when you know what it’s about. Cmd+K is for when you know what it’s called. When you want to see an engram the agent found, ask it for the link: every engram it reads or writes comes with the address of its page in Fluid, and the link can open that page at one heading.
9.2 Gathering the neighbourhood
A hit is a point, and a task usually needs the area around it: what the fact depends on, what depends on it, the runbook that mentions it. An experienced colleague does this without noticing. Ask them about the retry cutoff and they tell you about the bank unprompted, because in their head the two were never separate.
build_context is that reflex as a tool. Hand it an address, crystalline://engineering/retry-queue-gotcha, and it follows the relations and the prose links outward, one to three hops, crossing into other domains where an edge crosses, and returns the neighbours ranked by how strongly each connects to the anchor. What comes back is the shape of the neighbourhood (titles and typed relations, with an untyped prose link surfaced as links_to) and no content. The agent reads the nodes that matter with read_engram, which is the second half of the same motion. A /* suffix on the anchor globs a permalink prefix, and a permalink carries its folder, so that’s a whole folder: crystalline://engineering/runbooks/* (Section 4.6). max_related keeps the top-ranked, ten unless you say otherwise, so a densely linked engram doesn’t drown the task in its acquaintances.
The loop feeds itself. read_engram marks each relation and link as resolved or not, summarizes what links back and, when a neighbourhood exists, names the build_context anchor to run next. An edge that doesn’t resolve is a signal rather than an error: the target isn’t written yet, or lives in a domain this instance hasn’t registered. Fluid draws the same neighbourhood as an interactive graph (Figure 15.3), the quickest way to see whether an engram is wired in or standing alone.
9.3 What applied last June
Every fact in the archive is one of two kinds, and a good question knows which it’s asking for. “What is the cutoff?” wants the present. The present is a status question: filter on status: stable and the search returns what holds now, with the older spelling current folded in. “What cutoff applied in June?” wants a moment, and a moment is a window question, answered from valid_from and valid_to, the plain dates on the engrams that carry them. It’s the time circuits on the dashboard: set the destination time to June and the search returns the engram that was true when you land. Chapter 10 shows the window being closed with the real date.
Absent dates are unbounded, and that rule sets a trap in a filter. A strict comparison (valid_to greater than the first of June) skips every engram with no valid_to, which are exactly the unbounded ones the question meant to keep. The routing skill puts it plainly: reach for a window filter only for an explicit, bounded question about a date, and put it on the first search. For an ordinary present-day question, status: stable alone is both correct and sufficient. When a window filter isn’t getting traction, the supersedes chain and each hit’s status tell the same story a different way: search by topic, read which engram covers the period asked about and narrate the status of each rather than presenting a retired one as fact.
Time runs the other way too. A search takes a recorded-after date, and recent_activity answers “what changed while I was away” across every domain for the last week, day or fortnight, which is how a person back from leave starts current instead of starting over.
9.4 Recall is cheaper than reading
It’s worth stating the case as a mechanism, with no number attached, because the number depends on your archive and the mechanism doesn’t.
An agent that keeps its knowledge as a folder of markdown files has one way to use it: read it. The folder can’t say which file matters for today’s task, so the agent reads the folder, and reads it again in the next session, every file, whether or not the task touches it. That works while the folder is small, and it stops working at the scale where knowledge becomes worth having, the ladder chapter 1 climbed.
An agent working from Crystalline receives only the knowledge that applies. In Claude Code and Codex the prompt hook runs the first search before the agent has read the question and hands over at most three engrams. The routing prompt sends the task to one domain, or to a sweep, before a single engram is read. Inside the domain the indexes narrow again: full-text and semantic search for the phrase and the meaning, the graph of relations and backlinks for what the answer leans on, tags for the facet, the folder prefixes that build_context globs for the topic, the temporal fields for what holds now against what held then. What reaches the model is the hits that matter, the engrams it chose to read in full and the neighbourhood it asked for. Nothing else is loaded. Reading a folder costs the whole folder, every session. Recall costs the engrams that matter, because the ten-thousandth engram is found the same way as the tenth.
The person on call knows this instinctively. They don’t reread the wiki at three in the morning. They ask the colleague who says “hang on, we had this” and reaches for the right drawer.
Your sister has picked up the Warhammer novel you abandoned and wants to know what you made of fantasy last autumn, before you went back to science fiction. You take the question to your agent. It searches books for fantasy with a window on the first of October and gets the superseded engram, valid until the month you put the book down, with the phase as it stood and what led you there. It builds context from that engram’s address and pulls in the two Tolkien engrams and the shop that stocked them one hop away. Then it searches again with status: stable and reports what applies today, so you can answer both halves with the dates between them. You never open the old log. You don’t remember last autumn’s taste by the letter any more, and you don’t need to.
The same week you come back from a fortnight away, where the only sessions were evening ones on a balcony, and ask what the fortnight added. recent_activity over two weeks lists the two books you finished, a corrected opinion of the last Next Generation one and three new engrams under to-read/, the next Destiny volume among them, each with its domain and date. You’re current before your coffee is cold, where a folder of notes would have had you rereading, or asking.
A hybrid search runs a lexical query and a semantic query and ranks by the blend. Until embeddings exist it falls back to plain text. The salience prior is search.salience_weight (0.15 by default) and the retirement fade is search.retired_weight (0.6 by default), both bounded so relevance keeps the upper hand, and 1.0 turns the fade off. The list filter is $contains in a search’s metadata filters, and $contains_any takes up to 100 values. Embeddings are computed in the background as engrams land, so a fresh engram is findable by text at once and by meaning shortly after. On an Apple Silicon Mac the embedding model runs on the GPU, and crystalline status shows which device it runs on (sorry if you’re on a different operating system ;) ). The index behind all of it is derived from the files: crystalline reindex --full re-reads every file and keeps the embeddings, and crystalline reindex --wipe rebuilds an index that won’t open.
The engram was written for a question nobody had asked yet. Recall is what makes that fine.
Search is recall: full text finds the exact phrase, semantic similarity finds the adjacent one in any language, salience lifts the exceptional and a retired status fades the past without hiding it. Filters narrow by domain, type, tag, status, any frontmatter key or date, and a snippet is a pointer to read, never an answer. build_context turns a hit into a neighbourhood, following relations and links from an address or a /* folder glob, ranked by connection strength, and Fluid draws the same graph. Ask the present with status: stable and the past with validity windows. Absent dates mean unbounded, so a strict date filter is only for an explicitly bounded question. Recall is cheaper than reading because only what matters is loaded, routed by domain and narrowed by the indexes, so the cost stays flat as the archive grows.