A Tale of Three Months with AI: Guardrails, and How We Are More Alike Than We

Close menu Writing

Projects

Tags

About

A Tale of Three Months with AI: Guardrails, and How We Are More Alike Than We Wish to Admit

20 October 2026·23 mins

Ai Programming

Table of ContentsThe greenfield honeymoonAsserting control

From the green pasture into the mud

The Engineering harnessInstitutional Knowledge

What is next: leveraging the speedup gains from agentic development and corporate fit

ConclusionPostscript

Table of ContentsThe greenfield honeymoonAsserting control

From the green pasture into the mud

The Engineering harnessInstitutional Knowledge

What is next: leveraging the speedup gains from agentic development and corporate fit

ConclusionPostscript

As I prom­ised in A fresh leaf, I want to share my ex­per­i­ences with us­ing LLMs for agen­tic de­vel­op­ment in the last three months. Sum­mar­ised: I star­ted as an ini­tial scep­tic, be­came quite en­thu­si­astic, and am now mod­er­ately en­thu­si­astic and guarded for the fu­ture. Al­low me to take you through my jour­ney.

Com­ing from a back­ground of rig­or­ous en­gin­eer­ing with up-front think­ing about pro­gram struc­tures, al­gorithms, and types, it seemed in­com­pre­hens­ible to me how a jumped-up neural net could gen­er­ate work­ing code, let alone sens­ible sys­tems. How to write un­der­stand­able soft­ware has been hotly de­bated for dec­ades, con­tinu­ously shift­ing as new lan­guages, type sys­tems, and eco­sys­tems in­tro­duce dif­fer­ent trade-offs.

Early in my ca­reer, com­ing from aca­demia, I was fix­ated on find­ing the “just-right” solu­tion: el­eg­ant ab­strac­tions, strict cor­rect­ness, and formal sound­ness. As prob­lems grew lar­ger and messier, and time be­came scarce, hold­ing out for per­fec­tion be­came an obstacle. I found far more sat­is­fac­tion in achiev­ing some­thing func­tional, with el­eg­ance as a wel­come second. Hav­ing both over­lap is fant­astic; achiev­ing the ac­tual goal re­mains both a pri­or­ity and a ne­ces­sity.

This shift in per­spect­ive ex­plains why LLMs, and agen­tic de­vel­op­ment, even­tu­ally proved so at­tract­ive to me. Ex­pect­ing a model to gen­er­ate a “just-right” ar­chi­tec­ture, code, or text only causes frus­tra­tion. Con­versely, it acts as an en­gine to reach con­crete mile­stones in areas I was al­ways curi­ous about without hav­ing the time to do things “prop­erly”, or simply elim­in­at­ing the toil of writ­ing boil­er­plate when I can de­scribe my in­tent with less mech­an­ical pre­ci­sion.

If your goal is simply to achieve some­thing con­crete, they are re­mark­ably ef­fect­ive. That spans from some­thing as simple as in­teg­rat­ing an un­fa­mil­iar lib­rary to break­ing ground in an un­fa­mil­iar prob­lem do­main where you lack the tools and vocab­u­lary.

My cata­lyst was a device sit­ting in my liv­ing room. After re­ports emerged that its suc­cessor was ag­gress­ively hos­tile to user pri­vacy, I wanted to verify whether my own hard­ware be­haved the same way. Since I had no back­ground in firm­ware in­spec­tion or packet ana­lysis on em­bed­ded devices, I was ini­tially dis­cour­aged. Luck­ily, I no­ticed right then that I had a free trial month of Co­dex ly­ing around, so I de­cided to see how far these so-called “amaz­ing fron­tier” mod­els could carry me.

I star­ted out by talk­ing in ChatGPT and ask­ing ques­tions on how to solve my prob­lem.

This quickly be­came us­ing Co­dex to work on my laptop, let­ting it down­load im­ages, and run­ning local scripts. The way of work was very in­ter­act­ive, prompt-based, and re­l­at­ively un­struc­tured and un­guided.

I was amazed by how well the model worked when told to achieve a very spe­cific goal and even more so when giv­ing the con­text and con­straints in a single prompt: as long as the prompt wasn’t too big.

When I poin­ted out er­rors, the mod­els com­pli­men­ted me lav­ishly on the dis­cov­ery and apo­lo­gised pro­fusely for the de­fect.1 Yet that con­tri­tion yiel­ded no dur­able change in their sub­sequent be­ha­viour. It was just text, learned from reams of doc­u­ments.

As soon as the code­base and pro­gram to de­tect any ne­far­i­ous be­ha­viour was grow­ing, I no­ticed the per­form­ance of the mod­els de­grade.2 Re­sponses lost their ini­tial speed, pre­ci­sion, and cor­rect­ness. The model began over­look­ing in­ter­de­pend­ent up­dates that re­quired lock-step co­ordin­a­tion, shed­ding con­text es­tab­lished in earlier turns. In other words, it re­quired more hand-holding, for which I had neither time, nor pa­tience, nor ap­pet­ite.

At that point, I also star­ted work­ing on some other ideas that had been ly­ing dormant in my head. I used an agent to cre­ate a Ter­ra­form pro­ject to spin up a small timeseries-db backup in Google Cloud con­strained such that it would stay within the free tier. Cre­at­ing the pro­vi­sion­ing scripts was the work of mere mo­ments. It checked the ex­act siz­ing re­quire­ments for stay­ing within the free tier in­clud­ing some caveats, whipped the Ter­ra­form code to­gether, and ad­ded the Tails­cale in­teg­ra­tion. All from a simple prompt of what I wanted to achieve and the fact that I wanted to be able to re­con­struct the VM from a single com­mand. That was awe­some.

It was here that I first star­ted sus­pect­ing a deeper link between mod­els and hu­mans: it all came down to how bounded the task was. Over the years, I no­ticed that when you ask a per­son an open-ended ques­tion, their an­swer is mostly in­flu­enced by their re­cent his­tory, their emo­tional state, and whatever prob­lem they were just wrest­ling with. Mod­els work al­most identic­ally. Ask them a broad, open-ended ques­tion and you get a gen­eric re­sponse shaped by their pre-trained weights and im­me­di­ate con­text win­dow. Get­ting either a hu­man or a model to think out­side that de­fault box re­quires tight, ex­pli­cit con­straints, and care­ful prod­ding (or prompt­ing).

One of the of­ten touted best-practices I en­countered in the be­gin­ning was to have a three-phase ap­proach. Use one agent to plan the work in­ter­act­ively, pro­du­cing a han­dover doc­u­ment. Then task a second agent with im­ple­ment­ing that han­dover doc­u­ment un­at­ten­ded. Fi­nally pass the con­trol to a re­view agent, which gets the diff and the plan, and re­views it. The ob­vi­ous im­prove­ment is to make the last two com­mu­nic­ate and work in a loop un­til no more find­ings are es­tab­lished.

I first tried auto­mat­ing this di­vi­sion of work (plan­ning, im­ple­ment­a­tion, and re­view) by writ­ing con­ven­tions as prose in AGENTS.md. As code grew and threads grew longer, how­ever, prose in­struc­tions be­came less ef­fect­ive and eas­ily ig­nored by the model. To take this work­flow fur­ther, I wanted to com­bine the spe­cial­ised strengths of dif­fer­ent mod­els: Co­dex as the fast, local work­horse for writ­ing code, and Gem­ini as the thought­ful re­viewer for ar­chi­tec­ture. To let them talk dir­ectly to each other, I built gemini-mcp.

Here I hit an­other quirk of model psy­cho­logy: autonom­ous mod­els pos­sess such su­preme self-certainty that re­view trig­gers re­quire ex­pli­cit pro­gram­matic con­di­tions. A loose prose dir­ect­ive like “Ask Gem­ini for help when un­cer­tain” gen­er­ated pre­cisely zero in­voc­a­tions; the model was never un­cer­tain. Con­versely, a de­term­in­istic cir­cuit breaker (“At­tempt to solve each prob­lem a max­imum of three times; if still fail­ing, con­sult Gem­ini via MCP, im­ple­ment its re­com­mend­a­tions, and re­peat”) re­li­ably promp­ted Co­dex to es­cal­ate for second opin­ions.

This put me with an im­me­di­ate se­cur­ity chal­lenge: be­cause I was us­ing this setup to as­sess the pri­vacy of the TV, I needed a clas­si­fier to pre­vent prompts between mod­els from look­ing like of­fens­ive se­cur­ity probes, which could get my Google ac­count sus­pen­ded.

Scaf­fold­ing the clas­si­fier was quick work. Mak­ing it func­tion re­li­ably, how­ever, in­volved dozens of subtle ar­chi­tec­tural choices that flew past too fast for me to prop­erly eval­u­ate. Steer­ing the model through prose in AGENTS.md simply wasn’t giv­ing me the con­trol I needed. That was the mo­ment I turned to sys­tem­atic, de­term­in­istic tool­ing, start­ing with a simple dir­ect­ive: run make check after every modi­fic­a­tion to verify that the pro­ject was still co­her­ent.

This worked so well that ever since I have been ex­tend­ing what is be­hind this check con­tinu­ously. Re­ly­ing on what I already knew worked well in lar­ger pro­jects with mul­tiple con­trib­ut­ors, I de­cided to use strict tools to check as much as pos­sible. This meant auto­matic lint­ing, format­ting, and up­grades of the dif­fer­ent types of files in the pro­ject: Mark­down, code, YAML files, ADRs, format­ting, mod­ern­isa­tion of Go, and de­pend­ency up­grades. Moreover, I de­cided to take more out of the hands from the agents and not even bother them with format­ting and easy to fix lint­ing er­rors by auto­mat­ic­ally run­ning format­ting, auto­matic im­ports, and adding --fix to golangci-lint already as part of the check step to en­sure all easy fixes were im­ple­men­ted be­fore the agent even saw the res­ult.3

I in­tro­duced this rig­or­ous track­ing in my first big vibe-coded pro­ject, semantic-editor. The whole point of the pro­ject was to let the agent worry about in­tent while del­eg­at­ing the nitty-gritty edit­ing work to de­term­in­istic AST tools. I ap­plied that same mind­set to the re­pos­it­ory it­self, us­ing strict checks for the mark­down, the ADRs, and the schemas. This ap­proach worked pretty well, and the pro­ject was pro­gress­ing smoothly.

Around that time, an­other per­sonal pro­ject came into my head. I had al­ways felt sad that in this age of search en­gines and lately om­ni­scient (and of­ten wrong) LLMs, my kids didn’t have the ex­per­i­ence to just browse an en­cyc­lo­pe­dia, guided by in­terest. For me the con­crete ex­ample of this was Mi­crosoft En­carta ‘97. It gave me end­less brows­ing, learn­ing of all kinds of things, and im­port­ant to my par­ents (and now me), re­l­at­ively safe be­cause it was cur­ated con­tent.

As you may have guessed, I de­cided to vibe code the an­swer to this prob­lem as well and make a viewer of En­carta data. Be­cause in-repository de­cision track­ing had op­er­ated sat­is­fact­or­ily, I re­lied on that es­tab­lished har­ness to guide de­vel­op­ment.

How­ever, be­cause this was my first GUI app, I sensed it needed an­other layer of con­straints and de­cision track­ing which I dubbed “GUI-invariants” as a spe­cial­ised form of ADR for GUI de­cisions such as lay­out on a more gen­eric level in­stead of spe­cific de­cisions. Get­ting the agent to con­sider the in­vari­ants proved re­l­at­ively simple by just adding it to the agent in­struc­tions. Let­ting it in­ter­pret them in a way was harder, be­cause in­vari­ants are in­her­ently gen­eric. So they needed more con­crete pos­it­ive and neg­at­ive ex­amples and be more pre­script­ive.

For some reason I had a quirk that I wanted all text to be “prop­erly” type-set such as in TeX. And why not? We are cus­tom cod­ing this pro­ject and I have tokens to burn, so let’s have at it. The agent was very help­ful in this bounded con­text. A simple as­sign­ment, a start­ing ex­ample al­gorithm, the spe­cific con­straint of the SwiftUI in­fra­struc­ture, and an or­acle to de­term­ine whether it did it right. This al­lowed me to set a simple goal and I could walk away from it.

These three main pro­jects and sev­eral side pro­jects, all shared one im­port­ant prop­erty: they were green­field pro­jects where I in­di­vidu­ally was the sole owner and de­cision maker.

In urban plan­ning terms, green­field land is ef­fort­less to tra­verse and de­velop, pre­cisely be­cause it is vir­gin ground: un­touched, open, and un­en­cumbered by his­tory. Brown­field sites, by con­trast, are con­tam­in­ated swaths of land burdened by bur­ied found­a­tions, un­doc­u­mented pipe