Analysis — 분석
The Data Flywheel — Making the Loop Improve, Not Just Persist
Money runs the loop and tokens compute, but what makes the next turn better than the last is the data the operation accumulates. Turn the flywheel with three assets — traces, evals, learning signals — and keep data as an asset outside the model to preserve sovereignty.
Persisting and improving are different
The environment's first two faces keep the loop alive. Business makes it survive on money; token strategy runs the compute. People operate it and validation keeps it honest. But with all of this in place, miss one thing and the loop persists without improving.
That one thing is data. The only resource that makes the next turn better than the last is the data the operation accumulates. Money, tokens, people, and validation keep the loop running, but for it to get smarter over time you must feed back the data the operation produces. A loop without data merely endures; it does not grow.
Three assets of the flywheel
The data flywheel turns as three assets mesh. Operation leaves traces, traces are refined into eval sets, and evaluation produces learning signals that lift the next operation.
Execution traces — the raw record. Leave the agent's rationale, choices, handoffs, and outcomes as structured logs. The record the Decision Traceability pattern calls for is the primary data. Kept in a non-searchable form, a trace is just buried data, not institutional memory.
Evaluation data — the refined yardstick. Build an eval set from traces against business-outcome criteria. A yardstick that measures our results, not an external benchmark. If the yardstick is external — if you only chase benchmark scores — you cannot tell whether your product actually improves, and you have no data sovereignty.
Learning signal — the fuel for improvement. Feed human feedback, outcome labels, and approval records back as signals that improve the next loop. This feedback is the force that turns the flywheel. Without feeding signals back, data leaks out and all you keep is a model that gets swapped.
Data sovereignty — keep it outside the model
Data must live outside the model, as our asset. The model is a replaceable part; swap it and the traces, eval sets, and learning signals remain — that is data sovereignty. This data is the layer that lays our organization's own context on top of a generalist model, and the moat hardest for competitors to copy.
This is where the reason the State and Memory Control pattern separates short-term state from long-term memory becomes clear. If long-term memory lives only inside the model's context, it vanishes with the model. Separated outside and accumulated as an asset, data becomes a compounding asset that survives a model swap.
The flywheel turns with the stages
Data assets do not accumulate at once. They mature with the evolution stages. At stage one you simply leave calls, outcomes, and failures; at stage two you structure rationale and handoffs so they are queryable; at stage three you run a private eval on business-outcome criteria; at stage four learning signals lift the next loop automatically in a self-improving loop.
If money keeps the loop alive and tokens run it, data makes that loop better every turn. Only when data sits on top of a sustaining environment does the loop truly improve.
See also
- This theme among the environment's three faces: Data
- Sister pieces: Token Strategy · The Business Economics of the Loop
Tags