Show, don't prompt: how QA Wolf turns a recording into a real test

Point QA Wolf's agent at an app and it'll explore, map the pages, propose coverage, and write the tests on its own. That's what you want when you're after breadth. But a lot of testing knowledge lives in someone's head, like the promo code that only works on Tuesdays, the approval step that only appears for expenses over $500, or the export button only admins can see. Describing that in a prompt is slow and loses detail, while showing it takes about 30 seconds.

So we built a recorder into the agent's chat. You open a live browser next to the conversation, press record, and click through the flow. The agent watches each action as it happens, keeps a clean list of steps, and turns that list into a test when you're done.

A test built from every click and keystroke you make while recording would be a mess, though. So the agent cleans up. It tidies the steps, fills gaps, and adds the checks you implied before writing the code. What it can't do is change what you meant to test. Every edit lands in the step list you're watching, and every step it adds comes from something that really happened in the browser.

Most flows begin the same way: sign in as someone, maybe create some data, then get to the part that matters. Recording that every time is slow, and it gives you a second copy of a login your test suite already has.

Try the AI testing platform that makes QA 12x faster.

So before recording starts, the agent looks for the setup your flows already use. Press record and the panel says "Looking for setup steps…" while the agent searches your workspace for helpers that sign in, create test data, or switch accounts. It reads each helper to learn what it takes, then offers what it found as a short list in the chat, like "Sign in as an admin" for a helper such as loginAs('admin'). Login helpers carry a note: "Uses your workspace's login, so you don't have to record it."

Keep the ones you want and press Start recording. The setup runs in the live browser, and recording starts on whatever page it leaves you on. To record the same flow as a regular user instead of an admin, ask, and the agent offers the same helper with a different persona. If your workspace has no helpers yet, the agent offers to open the app's start page, and Skip setup is there if you'd rather do it all by hand.

Howl Outfitters

howl-outfitters.example/

howl-outfitters.example/

Howl OutfittersTentsPacksCart (0)Gear for the long way roundTents, packs and sleep systems, tested on the trail.

Record Playwright Flow

A person asks the agent to record the checkout as an admin. The recording panel says it is looking for setup steps, while the agent searches the workspace. The agent then offers two helpers the workspace already has, in a card in the chat: Sign in as an admin, marked as using the workspace's login so it doesn't have to be recorded, and Add a tent to the cart. The person unticks the cart helper and presses Start recording. The sign-in helper runs, the shop in the live browser now shows Ada signed in as an admin, and the panel starts recording. Their first clicks, Tents and Add to cart, become the first steps.

Setup runs your workspace's own helpers, so the recording starts signed in. Sped up; the store is a stand-in.

The setup doesn't end when recording starts. The replay runs it again in a clean browser, and the finished flow calls the same helpers on its first lines. Nothing about signing in is recorded twice, and when your login helper changes, the recorded test changes with it.

The core problem is keeping you and the agent looking at the same list of steps when three machines are involved. Your clicks land in a browser that runs in the cloud. You watch it, and the step list, from the QA Wolf app, while the agent works on a machine of its own. Every click shows up in the step list for both of you as it happens, and any change either of you makes shows up for the other right away. If a connection drops, the list picks up where it left off.

For each click, the cloud browser also works out a locator, the Playwright code that finds the same element again later, like getByRole('button', { name: 'Add to cart' }). The locator travels with the step.

Howl Outfitters

howl-outfitters.example/tents

howl-outfitters.example/tents

Howl OutfittersTentsPacksCart (0)Ridgeline 2P TentAdd to cart

Summit 3P TentAdd to cart

Record Playwright FlowRecord a flow

Worked for a bit

The recording setup is ready. Choose whether to open the start page, then start recording and click through your flow in the browser.

Agent's machineits copy of the step list

Click Tents

Of course, browsers are noisy. Typing an email address fires a key press for every letter, and filling in one form can add a few corrections, a scroll, and a click that gets reported twice. A step list showing all of that is useless, and the agent would spend its first turn cleaning it up. The list should read the way a person would write the flow down, and it takes two passes to get there: one in the page, one on the list.

The first pass happens in the recording script. Instead of sending each key press, it waits until you stop typing and sends one fill with what the field now says. If you leave the field or the page goes away before then, it sends the fill right away. Otherwise a one-time-code field that submits on the last digit would lose its value.

But that still leaves plenty of noise. Let's look at someone signing in. The script sends 11 events. There's a click into the Email field, four fills, a Backspace, a scroll, the Sign in click reported twice, and two page changes as the app redirects. The second pass, a set of capture rules, turns those into three steps: type the email, type the password, and click Sign in.

Create flow

Click through the flow in the browser.

Each rule is small and has its own tests:

What you record is rarely a finished test. You know the cart should show one item, but you never added that check. The recorder might miss an action, you might click something you didn't need to, and the order you clicked in isn't always the order that matters. So the agent closes the gap. It can add a check you implied, put back an action the recorder missed, reword a step so the next person can read it, drop a redundant click, or move steps into the order that matters. Every edit shows up in your step list as it's made.

What it can't do is add a step for something it never saw. The agent sees every browser event, not just the ones that become steps, so it knows the locator of anything you hovered over. To add a step you didn't record, it hovers over the element in the live browser and copies the locator from that event. It never makes one up.

You and the agent also share the same controls: record, pause, replay, start over, and discard.

Steps that worked once, with your cookies, aren't a test yet. So when you press Create flow, the agent reviews the step list and then replays it from a clean start before writing any code. The hard part is that the step list records what you did, not how long the app took to respond. Replay the steps too quickly and a good step fails on a page that's still loading. Retry blindly and the replay can click the same button twice. The replay has to tell a step that's actually broken from one that only needed another moment.

The clean start comes first. The replay closes any other tabs, clears the browser's cookies, and clears the data the app's site has saved, which signs you out. Then it runs the setup you chose and opens the start page.

Then the replay runs the steps one at a time. It skips checks like “The cart shows 1 item,” because a check stays in words until the agent turns it into code for the finished test. Each other step gets up to three tries, a second apart, since a page that's still loading can make a good step fail once. A click that opens a slow page can look like a failure even though it worked, so a step whose page has already moved on counts as passed instead of being clicked again.

When a step still fails after all that, the problem is real, and the replay stops and says why in plain words. For example, if the step list is missing the click on Checkout, the replay stops before typing the name and points out that no step before it goes to /checkout.

Howl Outfitters

howl-outfitters.example/

howl-outfitters.example/

Howl OutfittersTentsPacksCart (0)Gear for the long way roundTents, packs and sleep systems, tested on the trail.

Record Playwright Flow

After Create flow, the agent replays a recorded checkout in a clean browser, driving it itself. It clicks Tents, Add to cart and Cart, then stops at the step that types a name on the checkout page: the panel turns that step yellow and says no step before it goes to /checkout, so one may be missing. The agent says the same in the chat. The person presses Resume, which records again from where the replay stopped, and clicks Checkout; the new step goes in before the one that stopped. They press Create flow again, the replay runs every step through to the confirmed order, and the flow is created.

A stopped replay, in the agent's own words. New steps go in before the step that stopped, not at the end. The store is a stand-in.

A stopped replay has three ways out, and either you or the agent can take them:

Even after the replay passes, the step list is just raw Playwright code, one line per step. That's a fine record of what happened, but the locators repeat, nothing is reused, and the next person can't tell what each block is for. We didn't want recorded tests to be a second-class kind of test, and the step list is never saved as is. It becomes a brief, and the agent writes the flow from it the same way it writes every other flow, in our page-object style (@qawolf/pom).

The brief is a flow file that opens with your setup calls, followed by a numbered list of to-dos, one per step. Each one carries the locator the recorder chose, the alternates it found, the value typed, and your checks in your own words. A subagent picks it up with the page-object skill loaded and rebuilds it: Each locator lives once, in a page object, and each action is a named method that returns the page it leads to.

import { flow } from "@qawolf/flows/web";

export default flow("Cart shows the added tent", { target: "Web - Chrome", }, async ({ test }) => { await test("Cart shows the added tent", async () => { // TODO 1: Click Tents // locator: getByRole("link", { name: "Tents" }) // on: howl-outfitters.example/ // TODO 2: Click Add to cart // locator: getByRole("button", { name: "Add to cart" }) // .first() // alternate: getByText("Add to cart").first() // TODO 3: Click Cart (1) // locator: getByRole("link", { name: /^Cart/ }) // TODO 4: Check, in your words // "the cart shows 1 item" // on: howl-outfitters.example/cart }); });

What the recorder hands over: each step with the locator it used, the alternates it found, and your check in your words.

Because the last step is ordinary flow writing, the result gets everything a flow the agent wrote on its own gets, including the page objects your suite already has, which it reuses instead of making new ones. There's no separate “recorded test” style to maintain, and no drop in quality for having recorded it.

Recording doesn't replace the agent exploring on its own, and most sessions won't be all one or the other. Sometimes the agent leads and you review, sometimes you lead and the agent tidies up, and often it's both. The agent gets stuck on a login that needs a hardware key, hands you the browser, and takes it back once you're through.

What makes that work is boring on purpose. There's one step list that both sides edit, setup that reuses what your suite already has,