You'll need nerves of steel to work with AI agents

This post will focus mainly on Composer 2.5 in Cursor. These problems may not be present in opus-5.5 or gpt-6.1-sol-pro-max-ultra-thinking-fast. But gpt-6.1-luna, for example, has all the problems listed here. I’ve also encountered many of these problems before in gpt-5.5 and opus-4.6. I don’t know whether they’re present in the newer AI models released recently.

What I like about Composer 2.5: its usage limits and speed.

For comparison, let’s take two $20 subscriptions, one for GPT and one for Cursor. With gpt-6.1-sol, my 5-hour limit runs out after just a few requests. If I ask it to change a couple of lines in a readme file, that uses up 10% of the limit. If I ask it to change something in a large codebase, the entire limit gets used up just reading dozens of files, without writing a single line of code.

But GPT also offers a $100 plan with five times the usage limits. And how long would that last me? I work with complex monorepo projects. To make changes to one service, the agent has to read dozens of files in other services and several skills. If the $20 plan runs out after reading 20 files, will the $100 plan run out after reading 100 files?

That’s why I don’t use Sol, but use Luna instead. But it’s very slow. Compared to Composer 2.5, it’s dozens of times slower.

That’s why I don’t use GPT Luna, but use Composer 2.5 instead. It’s very fast and very cheap. And in exchange for that speed and savings, I end up wasting my time and getting stressed out. I’ve encountered some of these problems in Deepseek and other open models. Some models have certain problems but not others. The number and frequency of problems vary even when using the same model, depending on the phase of the moon.

Below is a list of problems I’ve encountered over 100+ hours of working with Composer in Cursor.

The agent writes code and changes dozens of files. I start reviewing the code, and the following conversation ensues:

And I still never got an answer to the question of why that function was needed.

Now, whenever I want to ask it something, I have to turn off agent mode or explicitly tell it not to delete anything. But when I forget to do that, it starts deleting parts of the code in response to a simple question.

Sometimes I spend a lot of time reviewing the code and ask it to change just one function. It changes all the files that have already been reviewed. And these changes don’t even relate to that function.

It might even delete all the work done in this chat. And not just in this one. I ask it to make a commit, and it finds changes to other files mixed in. And it just deletes them. Even though that work was done in other chats.

Also, if I ask it to refactor the code — for example, to rename a method in 50 files — it might delete all the comments it finds in those files. Which has absolutely nothing to do with refactoring the method.

And yet, it doesn’t delete what I ask it to delete. When changing a method’s name, it leaves the old method in place and adds a new one with the new name. That’s its entire refactoring process — so it doesn’t have to change all the files where the old method was called. It’ll simply mark the old method as deprecated and leave it accessible. If I point this out, it will finally do the refactoring, but leave the old method name in the tests and documentation. The tests will fail, and it will say the task is complete and the method has been refactored, but the tests are failing. As far as the agent is concerned, failing tests don’t count as evidence that the task is incomplete.

Also, while working on a task, it might remove an important parameter from the config or from a method that has nothing to do with the task at all. When asked why it removed the parameter, the agent claims it didn’t remove anything. You have to explicitly ask it to check the git diff, because it won’t think to do that on its own.

I already have a working service that listens for events, then updates values in the database and writes them to Redis. I ask the agent to do exactly the same thing in another service.

It does it, but forgets to write to Redis. Or it doesn’t update the values in the database. Or it names files and classes completely differently. Or it puts interfaces and types in the wrong places. After that, it says everything is done. I ask it to check everything it’s done. It checks and once again insists that everything is done. And, of course, all the tests are green, which confirms its conclusion that everything is done. I have to go through many rounds, pointing out each mistake so it can fix it. While fixing one mistake, it introduces new ones.

I ask it again to go through the code and make sure everything matches the other service. It changes a lot of files and says everything is done. I open the first file and see that it’s all wrong. I write to it again, and it replies that everything is done. I open the next file, and it’s the same story. What’s more, I can even point out specific functions in a single file and explain how they differ from the reference. It fixes one, we move on to the next function, and it fixes that one too. It can’t even make a single file match the reference.

Let’s go back to the service that listens for events. It has shared, reusable methods for working with the message broker. The agent even does everything right and uses those methods instead of duplicating them. But it forgets to add types, and I point this out. And instead of adding interfaces, for some reason it decides to duplicate hundreds of lines of code from the shared methods.

It has trouble with tests in general. I give it clear instructions: first, add a test; run the test; make sure it fails; tell me why it failed; then fix the code and run the tests again so that all of them pass.

What the AI agent does: it immediately changes the code. Then it creates a test that does not fail and tells me the task is complete. As a result, I can’t verify whether the test would have failed before the code was fixed. Or whether the code actually fixed anything.

This has happened so many times that I created a separate command for it. And I had to modify it many times before the agent started following the instructions. But it still doesn’t work perfectly. At some random point, depending on the alignment of the planets, it starts editing the code again without creating a test first.

And sometimes, to create a failing test, it uses a clever trick. There’s already a discussion in the chat about a parser for JSON data. I send it a new, small 20-line JSON input and ask it to write a failing test first. It adds the input data to the test along with the expected output from the parser. And to make the test fail, it simply changes a random field in the expected output and reports that the test has failed and the code can now be changed.

The project has a well-established test structure. The test fixtures for each parser method are stored in separate directories. When we need to add a new method to the parser, we simply add a new directory with fixtures and modify the parser code. The tests run through all the fixtures and check that nothing has broken.

All of this is described in the agent’s skills and rules. And when the agent starts working on tests, I can see it reading those skills first. And in most cases, it follows this structure. But at some random point, it stops following it and adds fixtures directly into the test code. I have to point this out, and it moves them to separate files. A few messages later in the same chat, it makes the same mistake again.

If it adds a parameter to API requests, schema validation fails when it runs the tests. It uses another clever trick: it simply disables schema validation or allows any unknown parameters in the schema. And the tests pass again.

There was another case. I asked the agent to write a test for adding data through an API request. The agent spent a long time trying to get the tests to pass. So it decided to simply skip calling the API route in the tests and write the data directly to the database instead. The tests did start passing, and the data was added to the database, but no API request was actually made.

In recent months, I’ve been noticing a new pattern in sentence structure more and more often. Here are some examples:

Each sentence contains two phrases in quotation marks that follow the same stylistic pattern. I don’t know where this pattern came from, but I hadn’t seen it before. Now AI models communicate this way, and I’m noticing this style in articles and videos. Maybe it’s an improved version of "It’s not X, it’s Y"? Although these aren’t exactly antitheses.

No matter how much I try to change the rules and skills for the agents, they keep using this technique. What’s more, Composer took the worst of these two techniques and now writes things like this to me:

It uses quotation marks in almost every reply. Quotation marks are used where they shouldn’t be at all. For example, here:

I’m tired of reading this kind of text from the agent. And if I point this out, the following conversation ensues:

Okay, let’s move the rule to the top. In the next reply, right in the same chat, I get text with quotation marks again.

Now you know what to expect if you decide to use Composer 2.5 in Cursor.

And to avoid these problems, you simply need to use the new frontier models that were released a few days ago, regardless of when you’re reading this article.

Sent to . Click the link in the email to confirm.