Reading a novel chapter by chapter: how Authrot's scanner evolved

Authrot is a writing studio for serial fiction. When a writer asks for it, an AI "reading" scans a chapter and updates the story bible: who appears, where they are, how they are related, and what changed.

We started with one big AI call per chapter. Now each chapter gets 60–85 small calls that mostly share one cached prompt. For most steps, code does the reasoning and the model only picks from short lists that code prepares.

Here is The Count of Monte Cristo, read through chapter 40. Apart from the generated pictures, everything below came out of the reading, with identities confirmed by the writer. Writers see it in the studio; readers of a published story get it as a spoiler-safe wiki.

Edmond Dantès at chapter 8, just imprisoned: 14 ties.

At chapter 40, in Paris as the Count: 61. Green is friends and family, red enemies, blue places and ships. Every tie is backed by a quote and starts at the chapter that shows it.

It tracks how people change, too. Here is his appearance, chapter by chapter:

The portrait shows him now, disguised as Sinbad the Sailor. Sinbad keeps his own entry, and Dantès's page lists him under Identities, revealed in chapter 40. The reading suggests the reveal and the writer confirms it.

This is what the reader knows, not what each character knows. The bible tracks secrets and who is in on them, but ordinary facts aren't yet scoped to who knows them. That's what we're working on now.

Read Monte Cristo on Authrot and watch its wiki fill in as you go →

A serialized novel grows one chapter at a time, and the writer edits old chapters too. Each scan has to update a structured bible:

Three constraints shape everything that follows:

The first version sent the whole chapter, the bible, and one large schema with eight sections, and asked for everything at once. It was cheap (3.8–3.9¢ a chapter on flash) and it failed in a revealing way.

One call, eight sectionsTies asked alone

One test chapter, flash, thinking off

Sections starve each other. On one test chapter, flash wrote 23 facts, 38 mentions and 0 ties. Given ties as a call of its own, the same model found 27.

Thinking eats the budget. With thinking on, one reading produced 41k output tokens. On a pro-tier model at low thinking, 35–70% of output was reasoning, and split into four parts it cost 24–27¢ a chapter.

Mentions crowd out judgment. Even in later readings, mentions were 55% of output, and half or more of them were plain name matches that code can find for free.

The one-call path is still behind a flag (READ_IN_ONE_CALL=1) so we can compare against it.

We split the reading into parts and gave every part the same opening: the same system prompt, the same schema, and the same first message (chapter plus bible). That message ends with a cache breakpoint. Only the second message, the part's task, changes.

[system: role + rules] ┐ [schema: one shared JSON schema] ├ cached, written once per chapter [user 1: chapter text + bible handles] ┘ ← breakpoint [user 2: "List the ties of kind FAMILY"] fresh, a few hundred tokens

What made it work:

A narrow call costs 0.02–0.1¢ on flash, so ten of them cost less than one big call that tries to do everything.

Narrow calls fixed cost, but not precision: ties sat near 50% against gold, and the ledger wrote many changes that weren't real. Two things moved them. Ties got one narrow call per kind of relationship, plus a check (see Results). The ledger, and the steps around it, stopped asking open questions. In such a step, code builds a numbered list of candidates and a closed list of choices. The model answers one question about them. Code then checks the answer and decides what runs next.

Code listsnumbered items C1 to Cna closed choice listcaps on words and moreModel picksone narrow call overthe cached startanswers: items, moreCode checkspick is in the choiceshandle and block existwords cut to the capSaved on the runa retry asks onlywhat is missingnext step: code decidesevery item answered none: asked again

One pipeline step

Every narrow step returns the same shape, which is part of the reading's single schema, so steps share the cached prefix:

const stepItemSchema = z.object({ n: z.string(), // the item answered: "C3", "P2" pick: z.string(), // one of the question's choices, or "none" to: z.string(), // an entry handle ("@12") when asked whose/which block: z.string(), // the block id that shows it (the evidence) words: z.string(), // capped free text }); // answer = { items: StepItem[], more: StepItem[] } // more = what code didn't list

The ledger's steps run on a small DAG scheduler. Each step declares needs. Every step whose needs are done starts at once, a wave at a time. On a cold cache the first step that asks runs alone, so it writes the prefix and the rest of its wave reads it. The runner is pure, because the asking is passed in, so every rule is unit-tested without a model.

Models copy block ids badly, so code repairs what it safely can. Blocks are shown tagged ⟦id⟧. A model's answer matches if it is the exact id, or a prefix of at least 6 characters that matches exactly one block. Anything else names no block, and the item is dropped.

The production reading today, roughly in dependency order:

The writer's rule for this phase was explicit: no thinking and no stronger model. The pipeline does the reasoning.

On the ledger, code flags doubtful facts and a narrow call confirms them. That took precision on "what this chapter changed" from 47% to 89%. The checks cost recall, which dipped to 65%; asking the story question a second way brought it back to 75%.

No checks : recall 77%77One broad doubt call: recall 77%77Doubt call per kind: recall 65%65+ definitions : recall 66%66+ reframe (asked twice): recall 75%75No checks : precision 47%47One broad doubt call: precision 61%61Doubt call per kind: precision 71%71+ definitions : precision 91%91+ reframe (asked twice): precision 89%89PrecisionRecallNo checksOne broaddoubt callDoubt callper kind+ definitions+ reframe(asked twice)

Recall, then precision judged by reading. The first three steps on Monte Cristo chapters; the last two on the pooled gold chapters of both books. “+ definitions” plots the midpoint of 89–93%

How we measured

Every change was scored before it shipped. Most of what we tried did not ship.

Each change below was measured on two runs before the next one started. Giving every kind of relationship its own narrow call raised recall from 38% to 56%. Adding a check call pushed judged precision to 97% (64% against gold) but cut recall to 40%. The check then had to learn to keep ties that the chapter, not just the quote, supports. The biggest recall gain came last: twelve parts run per person, with the check asked twice, took recall from 49% to 69%.

Baseline 2 runs: recall 25.5%26Bible fixed: recall 28%28Fixed vocabulary: recall 38%38Nine kind parts: recall 56%56+ checker : recall 40%40Checker loosened: recall 49%4912 parts, per person: recall 69%69Final 2 runs: recall 72%72Baseline 2 runs: precision 30.5%31Bible fixed: precision 48%48Fixed vocabulary: precision 56%56Nine kind parts: precision 71%71+ checker : precision 97%97Checker loosened: precision 88%8812 parts, per person: precision 85%85Final 2 runs: precision 90%90PrecisionRecallBaseline2 runsBiblefixedFixedvocabularyNine kindparts+ checkerCheckerloosened12 parts,per personFinal2 runs

Precision against gold through “Fixed vocabulary”, judged by reading from “Nine kind parts” to “12 parts”, against the widened gold set at “Final”

On a model of its own, the check runs over its own compact prompt (the chapter, plus who the ends are) and is asked twice. A list tie is dropped only when both asks reject it. A focused second look then re-asks only the ties the two asks split on, plus the kinds most often wrong. On replayed answers, that took precision from 85% to 92% for about 1.1¢ a chapter.

Costs are what writers pay, in US cents (1 credit = 1¢). The full pipeline cost 5.2–8.4¢ a chapter on the first six chapters of Monte Cristo. If that rate held, the whole book's 119 chapters would come to roughly $6–10, though later chapters, with a bigger bible, cost more.

1.9–2.7¢v2flash10–12 partsLuna, no thinking, 12 parts: 2.7–4.8¢ a chapter2.7–4.8¢Lunano thinking12 partsv1, flash, one call: 3.8–3.9¢ a chapter3.8–3.9¢v1flashone callv3 pipeline, in use, 60–85 calls: 5.2–8.42¢ a chapter5.2–8.42¢v3 pipelinein use60–85 callsPro, no thinking, 11 parts: 9–15¢ a chapter9–15¢Prono thinking11 partsPro, low thinking, 4 parts: 24–27¢ a chapter24–27¢Prolow thinking4 parts

Every row but v3 measured on a test novel (~2,200 words a chapter), v3 on six Monte Cristo chapters: different books, so compare the order of magnitude

On the same five chapters, the v3 pipeline cost about 55% more than the reading before it, making about six times the calls and checking each answer. That buys the precision gains above, and it still costs less than a pro model with thinking off (9–15¢ across its 11 parts).

Replacing each part's bespoke schema with the generic step answer cut output about sevenfold, from about 220 tokens per fact to about 35 per item. The old schema paid for every field the task didn't use.

The catch was empty answers: a call that returns no items at all for a chapter that has things to report. The model skips the task instead of answering it. With a free list ("list the standings that changed"), that happened in 6 of 11 calls. Giving the model a checklist to answer item by item fixed it: either the eight kinds of standing, or the people the chapter names most. Each item has to be answered, even if only with "none". In production, an empty or all-"none" answer is asked again.

Rewording the story part's task did the same for the ledger: from 4,700–6,400 output tokens to 1,700–2,500, and from 0.49–0.67¢ a chapter to 0.18–0.27¢.

On one Monte Cristo chapter of 3,891 words, read by the full pipeline, input was 1,386,060 tokens read from cache, 62,236 fresh and 15,672 written to cache, with 51,629 tokens of output, for 7.71¢. That is about 95% of input served from cache.

The pipeline only matters if writers can trust it and afford it. Four properties came out of building it this way:

For us, the payoff was measurement. Every call is metered on its own, so each change in this post was priced before it shipped, not guessed.

Most of what we tried was measured, then switched off. A selection:

Next. The ties goal is recall of at least 75% at 90% precision. On the ledger, we are adding per-thread trigger calls. And the bible's who-knows-what needs to reach past secrets: what each character knows, and when they learned it.

A studio for long stories. Scan a chapter and the reading in this post updates your story bible, so every character, place and promise stays straight while you write.

The Count of Monte Cristo, The Time Machine and Dante’s Inferno, each with a spoiler-safe wiki that opens up as you read.