Projects as design experiments: learning architecture with AI

Expanding the unit of design learning

Across 25 pilot projects this year, the most useful way I found to improve at AI coding was to work on projects with different development principles at the same time.

When models changed, the way they interpreted the same instructions changed too. I had to adjust prompts and harness settings tuned to the previous model. Alongside those adjustments, I wanted to build judgment I could carry into the next project.

So I declared a different development principle for each project. I kept those principles in place as models and tools changed, and observed the differences in results. One project implemented features quickly and refined the structure later. Another started with detailed specifications and rules. A third required a high quality bar before merging.

As differences accumulated, I could use one project to help solve a problem in another. The scope of design learning expanded from individual codebases to the relationships between projects. This article turns that experience into a procedure: cross-project design review using reference projects.

Write the concept as a design policy

A project's concept is hard to compare if it remains a personal preference. Write down what takes priority, how decisions are made, and which costs to watch.

These are not mutually exclusive architecture types. A still needs security standards, and C can still experiment quickly. The difference is what takes priority when choices conflict. Establish a minimum baseline for every project, then vary priorities above it.

Attach an observation question to each policy. In A, how much rework did the fast first implementation create? In B, did the specification help predict the scope of a change? In C, which defects did verification catch early relative to the time it added? These are examples of what to compare in future work, not claims that the three policies have been quantitatively evaluated.

The question needs to be more specific than which project is best built: under what conditions did the benefit of this policy exceed its cost? That leaves a basis for deciding whether to apply it elsewhere.

When the same request produces different results

I once asked three projects to improve lint performance using the same prompt. Only one improved to the level I wanted; the other two could not resolve circular dependency problems. I asked for a comparison with the successful project, then gave further instructions based on the differences. The other two were resolved as well.

That experience showed that comparing projects helped unblock the work. It does not establish that a particular development policy prevented circular dependencies. That would require examining the dependency structures and the actual changes.

To turn the result into architectural learning, take the analysis one step further:

• How did the successful project constrain the direction of module dependencies?
• How did the location and ownership of shared code differ?
• Which change broke the cycle, and how was existing behavior verified?
• What conditions would make the same change valid in another project?

These questions describe evidence to seek, not established causes of that incident. If dependency direction turns out to explain the difference, the useful thing to transfer is the rule constraining dependencies and the way to verify it, rather than the entire folder layout.

A project becomes a concrete reference

Comparing projects also helped with review requests. With a project A that had undergone security verification, I could ask:

Review the security of this project using project A as the reference.

When results use structures and terms I already know, I can understand differences and give the next instruction without a long explanation. A reference project contains the code implementing a principle, its tests, and its handling of exceptions.

But the scope of "using A as the reference" needs to be explicit. Verification of A does not establish the security of the current project. Different data, user permissions, and deployment environments require different checks. Give the agent access to the relevant code and documents rather than assuming it remembers the reference project.

A more specific request could be:

Reference: a specified commit of project A and its security review record
Review scope: authentication and authorization boundaries in the current project

1. Identify A's permission checks, rejection conditions, and related tests.
2. Compare them with the corresponding code here, citing file paths as evidence.
3. Identify differences in user roles and data access that prevent direct reuse.
4. Propose a reason and a verification method for each applicable improvement.
5. Leave unverifiable items open rather than filling them with assumptions.

Use the project as a reference while specifying the review boundary and evidence.

A review cannot stop at "this differs from A." It needs to explain why the difference is a problem here. If the difference follows from a legitimate requirement, keeping it is the right choice.

Making cross-project design review repeatable

To make this experience repeatable, connect comparison to verification.

1. Narrow the problem. Choose an observable issue such as circular dependencies, missing permission checks, or latency in a particular operation, rather than improving overall quality.
2. Choose a reference project. Select one with evidence that it solved the issue and with comparable relevant requirements. Record the reference commit or document version.
3. Turn differences into a hypothesis. Compare code, contracts, and verification procedures. Explain which differences might affect the issue, separating observations from assumptions.
4. Check applicability. Consider whether the principle still holds with different scale, data, execution environments, and responsibility boundaries.
5. Apply a small change and verify it. Change only what the hypothesis requires. Check both the original problem and the preservation of existing contracts.
6. Record the decision. Keep the reference evidence, reasons for adopting or rejecting the change, verification results, and conditions for revisiting it.

For lint performance, for example, compare before and after under the same execution environment and inspection scope. Distinguish removing checks from improving performance while retaining necessary checks. If a change breaks a dependency cycle, verify the affected behavior and module contracts too.

The record becomes a starting point for the next review. Instead of saying A is good, it can explain which rule was applied under which conditions and how the result was verified.

SHARPEN feedback moves between projects

Within the OCLS loop of reopt architecture, cross-project design review uses feedback from SHARPEN to inform design decisions in another project.

Module Contract clarifies the boundary to compare. Evaluation and Guardrails provides criteria for checking improvement. Decision Traceability connects to recording why those criteria were adopted.

An improvement in one project does not automatically become the answer in another. Learning continues by checking the rule against the receiving project's responsibilities and contracts, then evaluating the result of applying it.

Record the limits of the comparison

Developing projects during the same period can reduce the problem of comparing models and tools from different points in time. Concurrency alone does not control the variables. Check whether model and harness settings match; even when they do, context and task difficulty can change the results.

Domain, scale, maturity, and the time I invested also differed across projects. This makes the approach a practical way to find and test design hypotheses rather than a controlled experiment. Recording these differences helps avoid turning an incidental success into a universal rule.

There is no need to increase the project count just to learn. Start by comparing the same kind of change in two projects you already operate. Reduce the scope if maintenance costs exceed the value of the learning.

Carry the reasoning into the next project

What helped across 25 pilots was the ability to keep comparing projects that maintained different principles. Differences in tests, security, feature delivery speed, and UX generated the next design questions.

If AI helps you work across several projects, those differences can become material for architectural learning. Check an improvement from one project against the conditions of another, then record where it applies and what verification showed. That evidence can be revisited and used even after models and tools change.

The next article will introduce the open-source project I built to develop several projects concurrently, and the workflow around it.