Pragmatic guide to agentic coding – Ilya Razenshteyn

Translate

Pragmatic guide to agentic coding

Ilya Razenshteyn

Oct 27, 2026

12

2

4

Share

(AI use disclosure: AI was used to do some planning for this post, but the text itself was written purely by me, I own all the mistakes and broken English!)

In this post, I attempt try to study the following question:

If producing code is getting commoditized, what prevents software engineering from getting automated or at least getting much cheaper?

If you are a software engineer or a systems / ML researcher, you know that around the end of 2025 / beginning of 2026 your life drastically changed. You pretty much stopped writing code. Of course, people already used LLMs for a while to write a function or refactor a file, but at that time, we crossed the threshold of “no need to manually write code. at all.”

This created a mixed bag of reactions. Many people felt relief that they didn’t need to deal with minutiae of Python weirdness or Rust borrow checker, yet many people (including me) felt sadness that a skill we learned and honed for many years if not decades is not needed anymore and anxiety that now we are merely replaceable cogs in the machine. At the same time, there has been a weird mix of “now these coding agents will help us to accelerate to the bright future and anyone can build a $10M ARR business over a weekend” and “ugh, what is this slop with idiosyncratic bugs that I need to make sense of now that the machine can produce code way faster than I possibly can consume and internalize”.

But the shift happened. The ship has sailed. We just don’t write code anymore. And we are never going back.

When such drastic changes happen, it’s always good to slow down and spend some time understanding the new reality a little bit better. The Internet is full of what’s at best can be labeled as “very biased information” and at worst, “intentional misleading, snake oil peddling and ragebait” on the topic of coding agents. Moderate, balanced “it depends” takes are very rare, since they are not attention-grabbing. That’s why in the past few weeks, I interviewed a couple of dozen of engineers / researchers from all kinds of companies -- from frontier ML labs to finance to “big tech” to small start-ups -- to understand how they use coding agents at work (as opposed to their personal projects), how they feel about it and how company culture, communication and team work got transformed.

Initially, my plan was to have a quantitative survey, but when I started talking to people, I quickly discarded this idea, because the stories I was told were quite diverse and it was hard to fit them in a single mold. So what follows will be qualitative and somewhat vibes-based (and definitely affected by my own views), but I hope it still paints a certain picture that is useful to understand.

In Section 1, I will talk about which models and harnesses are most popular and how exactly they are used. In Section 2, I will cover the issue of correctness and how we achieve in in practice, whether we need to read code, how we think about tests, and are “spec driven development” and “pure vibecoding” possible? In Section 3, I will describe how rapid introduction of coding agents affected the company culture. In the final Section 4, there is a call for action, how, in my opinion, we need to adjust to the new reality going forward. Finally, in the appendix, I will list some best practices in using coding agents for work.

The most interesting insight when it comes to models and harnesses is that by and large everything is as vanilla and boring as it gets.

On the models side, almost everyone uses Opus/Sonnet or GPT models, often a combination of them. Claude-family models are more popular, but GPT models are up there as well. One person reported to use Grok as an auxiliary model if the frontier ones run out of quota. A couple of people were looking into open weights models, the reasons being: needing to keep data local, being in countries where access to Claude/GPT especially in the work settings is harder (Russia, China), being worried about inevitable future price hikes on the frontier models and researching how good the current open options are.

For harnesses, it’s almost exclusively stock Claude Code or Codex. A few people use GitHub Copilot or Cursor with Claude/GPT models in it, the former usually due to the company having some enterprise deal with Microsoft, the latter is by people who want to be more involved in the specifics of what the model is doing. For open weights models, it was OpenCode or similar open-source harnesses. Remarkably, noone has reported to use Devin (still a big mystery how Cognition achieved reported $1B ARR).

One other interesting insight: noone seem to be using fancy workflows, swarms of agents, software factories or gas town-esque set-ups. Partially, because it’s too expensive, but mostly, because this does not really seem to be worth it. In general, the standard workflow is straightforward: create a plan, review the plan, implement the plan, review the PR (with an agent and then possibly by hand), merge. Wash, rinse, repeat. A few people use a standard subagents functionality provided by Claude Code / Codex (more on this below). Number of parallel threads one uses varies a lot and mostly seem to depend on the type of work that is done, I will touch upon this in the next section.

If people use Claude and GPT together, usually one of them checks the result of another. You can create a plan using model X, then implement it using Y, and then do a review using X again. This cross-model collaboration seems to be helpful. Interestingly, the breakdown between what to use for X and Y was pretty mixed, no obvious clear winner.

Even skills are used to a limited extent, usually for operational tasks not for implementations, and sharing of skills between the colleagues and especially between different kinds of models is rare to non-existent.

The issue of correctness is perhaps one of the most debated topic of the whole agentic coding discourse. After all, LLMs are inherently approximate, and even if each of their outputs is 99.999% correct and correspond exactly what the user meant and intended (tall order!), over time, the errors can and will accumulate and the codebase will pretty much never be doing what’s intended. A related but slightly different issue is that over time and with enough changes an agent can make the codebase ugly and unmaintainable, not only for humans, but for itself too.

Before we dive deeper into correctness issue, let’s mention some generic simple practices that are fairly popular and uncontroversial to help with that: use strict linters and static analysis to find code that “smells bad”, use languages that have stricter semantics that helps to catch many of the possible runtime issues (e.g., Rust, Go or Typescript are good for that and hence popular in the agentic coding setting), make changes in smaller steps that makes it easier to test / verify them and makes it harder for agent to slip some suspicious logic, use CI (continuous integration) consistently, use computer/browser use for semi-automatic UI testing.

Now let’s actually talk about correctness proper. How should we think about reading / reviewing / testing code?

One very common practice is to use agents for code review. Not much to say here other than it’s a useful heuristic to use a model for review that is different from the one you used for implementation. E.g., if you use Claude Code for implementation, use Codex for review and vice versa. It seems that a standard loop here where you send messages between implementing and reviewing agent until they settle works quite well. But that’s just the first step!

It of course would be appealing if we could give up looking at the code altogether and treat it as assembly, maybe we’d need to look into it extremely rarely, but mostly it’d be a level of abstraction we don’t need to worry about anymore.

However, and this is the main insight of this section, we at least so far can do it only in one of the following cases:

Demos and one-off boilerplate code and scripts, where it’s OK that the code works precisely on one input and it’s more or less obvious whether it worked or not. This is pretty self-explanatory and there is not much to discuss. It might be worth saying that non-technical folks often assume that the entirety of software engineering fits under this umbrella, which leads to lots of misunderstanding (more on this below).

Strictly testable/verifiable code, where it’s straightforward to formalize the property of a software component that we want (functionality and/or performance). Many influencers on X and LinkedIn want you to believe that most of the software is verifiable, but it’s definitely not the case. However, some of it is, e.g., a CUDA kernel, where we know which function we want it to compute and we can write an extensive test suite codifying this behavior. If this is the case, we can keep running the agent until it passes the tests (perhaps with some minimal manual steering). Now, the reality is of course more nuanced, and even if you have a very thorough test suite, it’s often the case that the agent can discover some gaps in it and exploit them producing an incorrect implementation, but at least you can likely get at least something useful out of such a blind approach.

The code where it’s enough to be as kids say these days directionally correct. This is what much of (exploratory / R&D) machine learning, data science, and research code more generally is. Roughly speaking, it’s enough if our results are mostly correct, so as soon as the agent does not go completely off rails (e.g., running evals on a training set, true story!), we get some results that are at least somewhat useful, and this loss of precision is worth the increased velocity. This is the uncanny valley and arguably the most painful place to be right now, since it’s really hard to find a balance to move fast (in a “directionally correct” way) and produce results that are long-term reliable. In this setting it becomes possible to juggle dozens of parallel threads running many experiments and asking agents to try different approaches, which puts incredible cognitive pressure and can lead to subtle errors that pile up and the sense of the overall frustration. It seems that how to operate in this “ok to be directionally correct” setting near-optimally is an open problem that currently no one really knows how to solve.

In all the other cases, especially if you require any degree of maintainability, reliability from your code or your product is customer-facing, currently, you have to look at the code / tests and review them, at least to some degree. Now, assuming this, practices vary along the following axes:

How much of the code is read and to what extent;

How much the tests are read / changed (quite a few interviewees mentioned that they spend more time reading / understanding / steering tests than code, but it was far from being overwhelming majority)

Whether one’s PRs are reviewed by colleagues, and if they are, it can be code or design doc review (almost universally, for the codebases that required some maintainability, some form of human peer review is there)

What seems to be a common practice is to explicitly mark the parts of repository according to how much it’s reviewed or yolo’d to align on this separation with teammates.

As the models get better, it’s more and more rare to have outright logic bugs in the PRs, usually the problems are of the two sorts:

Agent wrote ugly mess of code, and if continued like this for a prolonged period of time, long term tech debt piles up.

Agent misunderstood the requirement and interpreted prompt / design doc incorrectly (e.g., too literally), often in some almost comical way, and the newly created tests do not contradict this misunderstanding, rather, they reinforce it.

Both of the above failure modes justify underst