How to engineer an AI harness for game development

Exploring practical ways to make your game more legible to AI agents and how to make agentic iteration faster.

I gave a 45 minute presentation entitled “Your Game Is A Harness” as part of Indie Rodeo in Calgary on 9/12/26. After speaking with a lot of people after my talk, I thought it would be good to more broadly share the ideas I presented there in a form that could be enjoyed by people beyond just those who attended the conference.

Also, because I talk about AI a lot on this blog, people assume I use AI to write. This is not true! Everything below is 100% human authored, with me writing the post with basically only the slides I presented as a reference (this is not a transcription). As much as I enjoy using AI to write code and tools, I get a similar enjoyment out of writing “the old way”, especially as I have no reason to try and make any part of this process more efficient.

So with that all out of the way, I present to you my first ever blogification of a talk I gave. Enjoy!

[I give a small introduction to myself and my own work, that I’m omitting here, as you’re on my website]

I want to talk about using LLMs in game development. Not one-shot type demos that look cool on Twitter, but like real development. Where you’re working eight hours a day, 5 days a week, etc.

And before I get into it, I think it’s useful to level set about LLMs. Or at least get into some basics so the rest of what I talk about here makes sense. I recognize that for a lot of this people this will be review, so if you just want to get into the dev-loop stuff, scroll down to the next section.

My journey into using LLMs for dev work started with a moodboarding site I made called mood.site. This was pre “agentic” anything, but I wanted to test and see if ChatGPT at the time could actually code anything. The process was a lot of me just blindly copy/pasting code into ChatGPT’s web interface. The process sucked, but I was surprised at the results. I built the site! It worked! I still needed to verify a fair amount of stuff myself, but it was enough to convert me. If I didn’t have access to an LLM I wouldn’t have been able to make that site happen in near the amount of time it ended up taking me (~2-3 weeks of nights, probably like 20 hours total). I’d probably still be working on building the first version today.

Since then, mood has become something of a testbed for me moving through LLM stuff (and a mildly successful project on its own!), and has inspired me to kick up lots of other projects that use similar stacks. Additionally, LLMs have gotten better, especially with things like Claude Code and later Claude models. From my own experience, I feel like now there is very little, if anything, I think an LLM can’t do when it comes to a web-based stack.

It could be easy to then say LLMs can do everything, as so much of software is web software. But whenever I tried to use an LLM on building a non-web game thing, it was this resounding takeaway:

Any time I tried to use an LLM for a game-y thing, it would fall over. Maybe this is my own bias. I’ve been a game programmer for nearly 15 years, I think I know how things work. I can sniff out “bad” game code a mile away. But when it comes to Svelte web-apps, I’m decidedly less well versed and am inclined to think that whatever code the LLM produces is “good”. Maybe it’s actually garbage. But there is reason to believe the Svelte and web-stack stuff is quality, in part due to the fact that there are LOTS of high quality web projects with their source available online. Comparatively for games, I imagine there are orders of magnitude less data to train on.

Not only this, but games have a lot of stuff in them that isn’t code. Games aren’t just about putting static images and text in a layout. There are tons of things in games that aren’t easily representable by code. AND then on top of that there is all the same text layout stuff of web, but only as a subset of the 100 other systems a game needs to handle.

This is part of the reason why a lot of people have been making games in Three.js — they can easily represent assets as plain boxy primitives and skip a lot of the asset generation/conforming work and processing normally required for games. As a result, a lot of that stuff is slop because it lacks any real authorial voice outside of whatever the LLM median is.

A key idea of the one-shot Three.JS workflow (as well as LLMs proficiency at building web apps) is that everything is code. By having stuff as code, it’s very easy for an LLM to be able to reason about it, make changes, etc.

However for traditional games, this is not the case. Or at least the full story. Games obviously have code, but there is a lot of other stuff that is not code. This means that there are key components of games that are effectively opaque to an LLM. If your game then relies on the effective puppeting and transforming of that opaque stuff, then an LLM is going to fall over when trying to work on your game.

I think a lot of the gamedev community maybe gave LLMs a go a bit ago and ran directly into this. The “LLMs are bad/not useful/etc.” sentiment (massive caveat here, just talking about capability specifically and not potential society-ending implications) came from running headfirst into these limitations.

This is to say that, structurally, LLMs are bad at reading games. At least in their default state.

Let’s talk more about why, and what we can do about it.

Any call/prompt to any LLM service breaks down into three basic components. A system prompt, an assistant prompt, and a user prompt:

System prompt: Generic text shared across all users of a service. This usually describes the basics of the LLM itself, with something like “You are Opus 5, an intelligent Large Language Model created by Anthropic…”. This isn’t to say these aren’t important (they are), but more that they aren’t super-relevant to our use case. You can see some examples of posted (and exfiltrated) system prompts for models here.

Assistant prompt: This is kind of a sidecar prompt section meant to help guide the AI in terms of what it is responding to. You can think of it basically as the model’s basic “interaction” vector with your prompt, and represents something like the model’s “response” to the user based on some input. You usually do not have to think about this.

User Prompt: This is really The Thing. Most any input you put into any AI tool goes here. Any submitted chat, code, etc. gets dumped into this section. When people talk about “context”, they are largely talking about “how many tokens are stored in the user (and assistant) portion of a prompt”.

It may be easy to think that each chat is a “fresh” version of the above, but what actually happens is that, as you work with a model in a given session (the bounds of “session” vary based on what service you are using — this can be a single Claude Code session before closing it, a “chat” accessible in the side panel of a web hosted model, etc.), every prior message you sent to the model is re-submitted every time you send a message. It’s kind of like playing the old playground game where, on your turn, you have to say everyone else’s thing again before then adding your own thing for the next person to have to add on.

As you do this over and over, you build up the “context” of the session you’re working in.

This is all well and good, but a key thing to notice here is that you only really are able to address the “latest” thing in your context (i.e., you submit a new message). Things from previous messages get submitted over and over again to the model. Hence, you want to make sure what’s in your context is “good”, otherwise you can find yourself in degenerate scenarios where your context becomes polluted due to “bad” things earlier continually informing new work.

Managing context then is a key thing in working with LLMs, and a new engineering practice called “context engineering” has emerged in light of these constraints. Context engineering asks “how can we get the best information into the context, with as little pollution as possible”.

There is a great blog post here on “ways long contexts fail” that goes a bit more into this, but I’ll highlight the key part here:

Context Engineering itself is a really big topic that is outside the scope of this talk/blog, but we will be using some of the tenets of it here to explore how to help LLMs do a better job working with games.

Said differently, getting LLMs to work well with games is a context problem, not something that is intrinsically linked to how games or LLMs “are” (or “aren’t”!).

The complicating factor here though is, games are weird. Or at least: we have no real template to build a game. We have game engines, but there is effectively 0 standardized tooling for anything, in part because every game manifests its own needs differently from one another.

This is acutely unique to games (imo), but it manifests enough in other technological domains that a new concept has emerged in order to get agents to effectively “work” with whatever your work product is: Harnesses.

The general idea of a Harness is that it’s some set patterns of actions an agent is meant to take in response to some well (or loosely) defined expected input. You’re probably already using a harness like Claude Code, but these can be augmented, extended, distilled, abstracted, etc. to better support the specific work you are trying to do. There’s also been a LOT of action in this space recently, with many harnesses being built for developing all sorts of things:

The key things these are all pointing to is: “how can I better understand what I’m building, and guide that process as it is developed”. Harnesses can tie in things like iterating with a browser, telemetry, profiling, and even full SDLC work like making tickets, documents, etc.

However, the thing none of these do well is actually talk to your game. Your game is a bit of a black box, and most of what’s visible to an LLM is only the code-as-written, which as we talked about before, isn’t really enough. We know LLMs are already good at working with code, so it’s not like some harness itself solves that “games aren’t just code” issue we ran into in the first place.

What really needs to happen here is that we can take all the stuff in games, and try to turn it into something the LLM is able to work with. We need to crack open the black box of your game and make more of it addressable directly by an LLM.

During the talk version of this I had this Very Funny Slide that said that the arrow here was the actual content of this talk. The joke hits nowhere near as hard in blog form but I’m leaving it here for posterity:

Here’s some very tangible, actionable things you can do for your game to make it more legible and workable by an LLM.

As a brief aside before getting started, it’s also worth noting here that a lot of these tips are good for game development in general. They get supercharged a bit with AI, but doing any of this will likely make working on your own games generally just better.

A running theme through this whole talk/post is the concept of determinism. Basically, how accurately can you reproduce a very specific state in your game. The general goal is that it should be very easy, and very exact. Doing this at the very end of your production will be VERY hard to do, and would likely require fundamental changes to your game in the 11th hour.

Instead, if you assume this constraint before you write your first line of code, and stick to it as a goal, you’ll be paving the path to a much more productive end game and not find yourself fighting off entropy and spaghetti code.

Starting off here is pretty easy. At the start of your game, create a single source of seeded randomness. Make sure any call to anything non-deterministic uses this source of randomness. This means that your game will be able to deterministically recreate almost any state in the game. If your