When AI Writes the Code, Show Your Judgment

EmailAttachment

RolesDetailsCustomer, staff, installer, supplier or referrerSourceEmailDerived

Contact detailsDetailsEmail addresses and phone numbersSourceEmail

Lead sourceDetailsReferral person or platform, only when supported by evidence; otherwise UnknownSourceEmailDerived

Job stageDetailsLatest supported state of the job across the conversation, including planned versus completed workSourceDerived

JobDetailsDescription and service typeSourceEmailAttachment

Service addressDetailsFull work address and address componentsSourceEmailAttachment

Quote numberDetailsQuote IDSourceEmailAttachment

Invoice numberDetailsInvoice IDSourceEmailAttachment

EquipmentDetailsBrand, model, quantity, capacity, rooms and statusSourceEmailAttachment

EventsDetailsInquiry, site visit, quote acceptance, installation scheduled or completed, rejection and cancellationSourceEmailAttachment

Sending and paymentDetailsQuote sent, invoice sent, deposit paid and remaining balance paidSourceEmail

Event detailsDetailsDate, status, description, amount and currencySourceEmailAttachment

Email typeDetailsCustomer-related or unrelated; relevant messages may contain an enquiry, quote or invoiceSourceDerived

Review notesDetailsMissing information, uncertainty and conflictsSourceDerived

EmailRead from the body, subject or headers.

AttachmentExtracted from files in the sandbox.

DerivedInterpreted from evidence across emails and attachments.

Show all 15 rows

Table 2 Fields to extract. Derived fields need interpretation. Missing evidence stays unknown.

Based on our findings, the required fields are buried across different parts of the thread, some reside in the email body, others are located within attachments, and a few must be inferred from the conversation.

To figure out how to approach this set of extractions, we first needed to determine what pattern to use, adapting to the reality that certain groups of extractions could be handled by deterministic logic while others required an agent. As Antonio Gulli’s Agentic Design Patterns highlights, we shouldn't treat these patterns as an either-or choice; instead, we approach them as modular blocks you build upon along the way. In our case, we ended up using all the patterns, starting from basic prompt chaining to parallelization, routing, and finally agent-directed execution.

Fundamentally, this aligns with Anthropic’s perspective on building effective agents, where a workflow is defined by paths set in code. We know the desired end result because the output and the broad sequence are known beforehand, while the model is left to handle the interpretation within each step.

Prompt chaining · sequence (chosen)

Parallelization · concurrency (chosen)

Routing · selection (chosen)

Agent-directed execution · bounded tasks (chosen)

Prompt chaining. Each model call works on the output of the one before it.Attachment-reader findings feed the final extraction prompt. Email-only conversations can use a single extraction prompt. The ordinary sequence of code steps doesn’t itself count as prompt chaining.

Parallelization. Independent pieces of work run at the same time.Multiple conversations are extracted concurrently, two by default and configurable up to 40. Customer and job reconciliation runs sequentially.

Routing. A step sorts each input and sends it down its own path.Email rules decide what reaches extraction, and attachment rules and a classifier decide which documents to read. There’s no separate AI router classifying every email before extraction.

Agent-directed execution. The model chooses its own next steps and tools.Within bounded tasks, the reader can request more text and submit findings, and the extraction agent can retrieve attachment pages. Code controls the overall workflow, available tools and limits.

At a high level, the proposed workflow is the following.

PipelineParallelBranch 1AttachmentsEmailAnalyzeBranch 2AttachmentsEmailAnalyzeBranch 3AttachmentsEmailAnalyzeBranch 4AttachmentsEmailAnalyzeStructured analysis0 of 16 threads

PlayAttachments · SandboxEmail · AI agentAnalyze · AI agent

Figure 10 Extraction pipeline. Read attachments in a sandbox, read the email, then interpret the two together.

This orchestration can be done bare bones or with a framework, or even with a no-code tool.

Plain code

n8n

LangChain

Mastra (chosen)

Plain code. Direct model calls and ordinary functions. Few abstractions, but I would build the tracing, retries and experiment tooling myself.

n8n. A visual workflow with integrations and custom-code steps. Useful when much of the work is connecting services.

LangChain. An ecosystem in Python and JavaScript for composing model calls, tools and workflows.

Mastra. TypeScript agents, workflows, traces and evals in one development environment.It fit the language I knew and the short inspect-change-rerun loop I wanted.

I preferred using TypeScript since we already had the mailbox scripts built with it. LangChain (which also has a TypeScript package now) and no-code solutions could still work, but it gets a bit subjective at this point. I just really like Mastra for its trace viewers, evals UI, and their open-source philosophy. Plus, they put out great content for the community. Two of the key resources I used to understand agents more came from Sam Bhagwat's books, Principles of Building AI Agents and Patterns for Building AI Agents, and they also put out podcasts and workshops that are well worth checking out.

So, Mastra it is. Once we have a filtered golden set of grouped emails and threads, the first step Mastra handles is dealing with the attachments based on our proposed pattern in Figure 10.