How we built ChatGPT’s novel ad stack for user trust and relevance

By Ritendra Datta and Karthik Gopalratnam, Members of Technical Staff

Share

How should ads work in agentic chat?How should ads work in agentic chat?

Conversation memory, without influencing ChatGPT answers

An LLM interprets conversations and calls tools

How the orchestrator calls ad tools in an agentic loopPreparing retrieval queries that capture intent

Retrieving possible ads

Filtering ineligible candidates

Evaluating relevance and quality

Running the ads auction

Final eligibility checks

The final step: showing the ads to the user

What happens after the ad is shownHigh-power models evaluate and improve the entire stack

Human feedback steers tasteful direction

What’s next

How should ads work in agentic chat?

Conversation memory, without influencing ChatGPT answers

An LLM interprets conversations and calls tools

How the orchestrator calls ad tools in an agentic loopPreparing retrieval queries that capture intent

Retrieving possible ads

Filtering ineligible candidates

Evaluating relevance and quality

Running the ads auction

Final eligibility checks

The final step: showing the ads to the user

What happens after the ad is shownHigh-power models evaluate and improve the entire stack

Human feedback steers tasteful direction

What’s next

Traditional advertising systems are built to respond to discrete units of user intent. A user searches for or views something online, the ads system receives those queries and page views as events, and the system looks for matching ads based on those events. Building an ads stack for ChatGPT was different: user intent develops across conversations and shifts over multiple turns of a conversation, sometimes without a fitting advertising moment. To support this new paradigm, we built a bespoke ads delivery system. This new type of ad stack is based on a custom, post-trained frontier LLM that makes tool calls for retrieval (identifying the top options), ranking (predicting user engagement), relevance (evaluating the usefulness of the ad), and running auctions for final ad selection and pricing.

Our ads system supports the nuance of evolving user goals by making three key design decisions: 1) implementing stateful ads sessions, meaning information is remembered across turns of a conversation and across multiple conversations, 2) relying on a custom, post-trained LLM that uses a tool call architecture to interpret a conversation’s context and underlying intent, and then effectively select, rank, and deliver ads, and 3) running the ads selection process separately from ChatGPT’s answer generation, in its own service, to keep ads from influencing answers and maintain ChatGPT’s neutral voice. We use our post-trained LLM to orchestrate the entire ads flow, determining the right ad to surface at the right time based on both the user’s current turn and conversation history. While bootstrapping a self-improving ads system like this used to take years, ChatGPT Ads reached $1 billion in annualized revenue run rate less than 200 days after launch; now, more than 100,000 advertisers have run ads on ChatGPT since launch.

A conventional advertising request usually starts with a well-defined event. A user submits a search query or opens a webpage and the ads system receives a request. This type of system is fairly simple: it retrieves potential ad candidates, ranks them, runs an auction to appropriately price the opportunity, and shows the resulting ad to the user. ChatGPT does not naturally produce such a concrete moment.

In a ChatGPT conversation, people arrive with goals rather than a one-answer search query. They might be planning a trip, choosing software for their business, furnishing a home, or learning a new hobby. This doesn’t all happen in one turn of a conversation. ChatGPT often helps the user clarify what matters along the way.

As an example, our engineering team asked ChatGPT how to train for a race: A conversation begins with a general question about running. Several turns later, they ask about shoes to help train. The commercial meaning of the conversation changes as each constraint arrives—and this personalized context evolves across multiple sessions. A system that acted on the first message could retrieve the wrong intent entirely, potentially degrading the user’s experience and getting in their way.

Building our ads system required us to move beyond the request-response advertising architecture and build an ads stack from scratch to find potential economic opportunities while prioritizing user trust or value.

Turning every message into a standalone request, like a traditional ads system, would have carried a few risks in our chat use case: 1) misreading the user goal by oversimplifying it, 2) beginning the advertising flow too soon, before the user’s real need has become clear, 3) showing ads that get in the way of the user journey instead of supporting it, and 4) inefficiently rerunning the ads flow every time the request changes, creating noise and wasting cost. Instead of building around individual events, the ads system (consisting of retrieval, ranking, and delivery) maintains a stateful session, meaning the ads session runs for the duration of an eligible turn, remembering and updating information as it evolves. ChatGPT sends the initial request context to the ads server. The ads system follows and adapts to the developing exchange rather than relying on a single flattened query.

The ads selection process runs alongside ChatGPT’s normal response generation to ensure the ads flow doesn’t influence ChatGPT’s responses. Ads are filtered and selected through a separate system consuming the conversation under its own eligibility, quality, and delivery constraints. This important architectural boundary—different services that use separate models and separate code— lets core ChatGPT servers handle generating the organic response; meanwhile, the ads system independently evaluates whether there’s an appropriate advertising opportunity and, if so, prepares a sponsored placement through a separate serving process.

Advertising runs alongside the ChatGPT answer path.

Beyond protecting the user experience, this separation has some practical consequences. ChatGPT has no dependency on the ads system—the services are in no way intertwined and don’t share a content generation model. So if the ads serving system is unavailable or slow, ChatGPT’s organic response continues unaffected. This means the ads system can’t take down or delay the ChatGPT product. The other benefit of this separation is that it keeps the bar very high for valuable ads. Returning no ad is an ordinary, successful outcome. The system only delivers an ad if it’s clearly beneficial to users and advertisers.

The other key architectural component in our ads stack is the post-trained LLM that interprets conversations and runs the entire ads flow in an agentic loop—making tool calls to the core functions of the ads system, from the retrieval stage to ranking to auction and, ultimately, delivery. We call this LLM the orchestrator.

The first part of an ads flow is understanding user intent to start the retrieval process, which is a fast and deliberately broad sourcing of potential ads to later filter through. Receiving the conversation incrementally creates a richer but more ambiguous retrieval problem because the ads system needs to distinguish current intent from stale context. The orchestrator must interpret references to earlier messages, recognize when a user is speaking hypothetically, and avoid treating every mention of a product or service as a request to buy something.

The ads orchestrator uses the available conversation and user context, iterating to find good ads and repeatedly choosing when to call which tool with the underlying goal of finding the most appropriate ad (the ad that balances high user value and high advertiser value) for that conversation and context. If the user chooses to personalize ads, the orchestrator also considers available nonsensitive information about the user’s past conversations to find the best ad matches.

The ads orchestrator iteratively calls ads delivery tools to find valuable ads.

As the conversation progresses and the ads orchestrator gets more updated information, the orchestrator can run a fresh set of tool calls with the updated context to find better ads and return them to the client, which can then use these new ads if they arrive in time.

User value

Advertiser value

Printed Throw Pillow

Ineligible candidate

The orchestrator is designed to balance user value and advertiser value, always putting users first.

The ads system consists of individual tools, and the orchestrator calls them iteratively to find the best ads for a conversation. At a high level, the orchestrator turns conversations into brief text summaries, which are then turned into semantic embeddings (stored meanings that are easier for an ads system to sift through), retrieves ad candidates using those semantic embeddings, filters those candidates for eligibility (based on budget, user settings, and other preliminary checks), assesses their quality and relevance, and does a final ranking to identify high value opportunities for placement at the end of a chat response. Each ad must therefore meet a high bar on eligibility and relevance in order to be eligible for placement.

The orchestrator LLM first prepares one or more queries representing the relevant commercial intent. This can include rewriting conversation context to broaden the commercial possibilities beyond 1:1 search queries and then creating semantic representations, where the orchestrator distills a multiturn conversation into a summarized version for the retrieval system—retrieving ads from our database in a focused way.

Ad placements will be better for both users and advertisers if they only appear when there’s deep understanding of the user’s intent. Multiple queries can preserve different aspects of intent. A conversation about planning a trip, for example, might contain distinct needs related to transportation, lodging, or activities. The retrieval system can preserve these distinctions rather than compressing the entire conversation into one literal string.

As an example, let’s look at a potential conversation sequence about hiking shoes:

What should I cover on a two-day trip to Yosemite?

Although the user hasn’t connected the shoe brand question to the Yosemite trip, this conversation might be represented in our ads system by two queries, such as [shoes appropriate for hiking trip] and [new [brand] product lines good for travel]. This might only be the first attempt; the orchestrator is designed to iterate on these representations until it finds ads in our inventory that are highly valuable to the user.

Retrieval is the process of creating a deliberately broad pool of ad options. Candidates may come from different retrieval strategies and can later compete under the same serving constraints. We build multiple indices that optimize for different system and product constraints (for example, the index for product feed ads is designed to be aware of positive or negative product reviews as it fetches the best possible ads. This is not relevant to other ads that may not have reviews associated with them). Each index is optimized for some version of approximate-nearest-neighbor (ANN) search, using embeddings representing the user’s intent. This ANN search is an approximation algorithm for retrieval over large semantic indices, meaning ads are fetched very quickly, getting close to a good solution rather than looking for “the right” solution, while retaining most of the relevant candidates.

The ads system uses rewritten queries to create semantic embeddings during the ad retrieval process and produce a broad pool of ad candidates.

Candidates that survive eligibility checks move on to a scoring phase, where the orchestrator makes tool calls to various scoring systems—f