How I build with AI

In my day job, the technical side of product management is working with engineering, data science and applied research teams: planning systems with them, weighing trade-offs, and knowing what will be hard before it is. Building my own products on real production stacks keeps that judgment current. I make the same calls I discuss at work, about data models, APIs, test strategy, AI model choice, running costs and failure modes, and then I live with the results. Each project page lists its stack and the technical decisions behind it.

Nothing an agent writes goes straight to the live product. Each change is a pull request, and GitHub refuses to merge it until three required checks pass: type checks, lint and unit tests; a production build; and the tests of the data-access rules. That applies to the agents as much as to me.

Every pull request also gets its own private preview, and browser tests open it the way a visitor would. The agent that writes a change never approves it; I do. I judge each change by what it does on its preview, and the checks, plus a security-review agent on sensitive changes, cover the code.

What the current setup runs on:

Most of the work runs on temporary machines in the cloud rather than on my laptop. The verifier and the security reviewer aren’t given tools for editing code at all. Agents don’t touch the production database: each database change is written and tested on a throwaway copy, and I apply it to production myself. Keys and passwords for outside services live in the hosting provider’s settings, never in the code.

The riskiest rules are enforced by code, because a written rule can be forgotten. Small guard programs check every command an agent runs, block the ones that delete work or rewrite history, and stop old database changes from being edited.

When something goes wrong, I handle it the way an engineering team handles an incident: find the cause, then change the process so it can’t happen the same way again. Each rule in the agents’ written rulebook records the date and the reason. A few from the client portal:

1. 1August 14: a fix’s change-log entry went out with unrelated work, but the fix itself was never sent to GitHub. For two days the record said it had shipped when it hadn’t. “Shipped” now means confirmed on GitHub, and a script checks it.
2. 2August 18: a broken version went out, because two agents had edited the same file and one saved the other’s unfinished work along with its own. Now each agent works in its own copy of the code.
3. 3September 25: the rule to check each change exactly as it would ship kept getting skipped. It became one command, and now the same check runs automatically on every pull request, where it can’t be skipped.
4. 4September 30: notices and side threads had made the chat hard to follow. Messages are now quiet by default, each one ends with the single thing I need to do, and a page lists everything waiting on me.
5. 5October 4: a research write-up arrived full of jargon. Now an agent that wasn’t part of the work reads anything meant for me first and flags what I wouldn’t follow.

Text from outside, like written answers from people testing the product, can hide instructions aimed at an AI. This is known as prompt injection. So outside text is treated as untrusted: it’s summarized on its own, nothing in it is acted on directly, and any change it suggests comes to me as a question.

When I want a second opinion from another AI tool, it gets copies of the docs with private details removed automatically, and a check blocks the copy if it spots anything sensitive.

A long AI conversation eventually condenses itself to make room, and details get lost. So the lead agent keeps a status file of what’s in progress, what’s waiting on me and what’s been decided, and rereads it after its memory has been condensed. Every plan is saved with my original request word for word, every change adds a change-log entry, and design decisions are kept as decision records with their status.

I started the way most people do: an AI assistant in the code editor on my laptop, one prompt at a time. Monolith Ultra was built that way with OpenAI’s Codex, and so were the first six months of Safe As Houses, with Claude Code. Several chats worked in the same copy of the code, every change went straight in without a pull request, and work stopped when I closed the laptop. The August incidents above came out of that setup.

In late September I moved Safe As Houses to cloud agents, with every change going through a pull request, required checks and a preview. Aether and this site were built with cloud agents and pull requests from the start.