Connect to MCP servers, ask typed judgments, evaluate your agents, and trace them with OpenTelemetry in RubyLLM 2.1.

RubyLLM 2.1 does less work on every call, and brings an MCP client, typed judgments, evaluations, OpenTelemetry tracing, and more control over where agents and records get their configuration. For everything that arrived in 2.0, see What’s New in 2.0.

Most of 2.1’s speed comes without changing your code. Streams parse events with RubyLLM’s own parser, append long text and reasoning in place, deduplicate the citations a provider repeats on every chunk in one pass, and compute the cost once, when the stream ends, instead of on every chunk. Requests pick their protocol once instead of once per message, derive tool names once, and parse a successful response once.

A reply no longer keeps the request it answered, so a long chat holds one copy of its history instead of one per reply. If you read that request, see Read Requests Before They Are Sent. The model registry loads once when several threads reach it together, and Vertex AI calls share one OAuth token instead of fetching their own. Persisted chats rebuild from the rows they already loaded, with fewer queries in applications that predate automatic_scope_inversing, and download a stored file only when a request sends it.

These measurements compare 2.0.0 with 2.1 on the same machine. Providers answer from canned responses, so the times show RubyLLM’s own work:

Two more speedups need you to opt in. Calls now share HTTP connections, so an adapter that keeps connections open lets every call after the first skip the TCP and TLS handshake. Use :net_http_persistent from the faraday-net_http_persistent gem with threads, as in Puma and Sidekiq, or :async_http from async-http-faraday inside an Async reactor, as in Falcon:

RubyLLM.configure do |config| config.faraday_adapter = :net_http_persistent end

Twenty calls to a local HTTPS server through :net_http_persistent took one handshake instead of twenty, and 0.4 ms each instead of 2.1 ms. Across a real network, each handshake you skip also saves its round trips. See Connection Reuse.

In Rails, run the 2.1 upgrade so chats remember the provider uploads of their stored files. A chat loaded in another process then sends the provider’s copy of a large file instead of downloading the file and uploading it again. Five turns of a chat with a 30 MB PDF, each in a new process, uploaded 150 MB with 2.0 and nothing with 2.1.

To measure on your own machine, clone RubyLLM and compare any version with your checkout. The benchmarks need no API keys or network access:

bundle exec rake "benchmark:compare[v2.0.0]"

The numbers above are medians from an AMD Ryzen 9 9900X running Ruby 4.0.7. Times vary from machine to machine; counts such as queries, downloads, and uploads do not.

Services such as Linear, GitHub, Notion, Slack, and Dropbox expose their APIs as Model Context Protocol servers. RubyLLM now includes an MCP client. Describe the server you want to connect to in a class, the way you describe a tool or an agent:

class Linear < RubyLLM::MCP url "https://mcp.linear.app/mcp" inputs :user oauth owner: :user

only :list_issues, :get_issue, :create_issue requires_approval :create_issue end

chat = RubyLLM.chat.with_mcp(Linear.new(user: current_user)) chat.ask "What's blocking the release?"

Every server tool is also a Ruby method, so you can explore a server from the console:

docs = RubyLLM.mcp(url: "https://learn.microsoft.com/api/mcp") docs.tools docs.microsoft_docs_search(query: "Azure Blob Storage").text

You decide what the model sees. Rename and redescribe tools, fix arguments the model should not choose, pass results through your own method, or build higher-level tools from the server’s primitives with a regular RubyLLM::Tool. Resources work as attachments, prompts work with ask, and a server’s requests for input pause the chat the way tool approvals do, surviving restarts in Rails.

Servers change while you use them. When a server says its tools changed, the next turn of a chat sees the new list. Call listen to hear about changes as they happen, including updates to the resources and tasks you care about, and react with after_change:

class Handbook < RubyLLM::MCP url "https://handbook.example.com/mcp" after_change { |change| ReindexPolicyJob.perform_later(change.uri) if change.is_a?(RubyLLM::MCP::Resource) } end

Handbook.new.listen(resources: ["handbook://policies"])

See Listening for Changes.

The client speaks the 2026-07-28 revision of the protocol and falls back for servers that predate it. OAuth follows the MCP authorization spec, and Rails keeps the credentials encrypted. Background jobs can connect as your app itself, with client credentials, a private key, or a workload identity token, and companies can authorize their people through their identity provider. Servers that require DPoP get tokens bound to a key.

Declare the protocol extensions your app supports with extension. RubyLLM implements two. With MCP Apps, tools come with a UI your app renders next to their results, and each tool call keeps the result its UI needs, so the UI renders again after a reload. With Tasks, a server runs a long tool call in the background while the chat pauses, the way it pauses for an approval, so no job waits for it:

class Reports < RubyLLM::MCP url "https://reports.example.com/mcp" extension :apps extension :tasks end

Tools, results, and resources also expose the _meta that extensions write, and log_level brings a server’s log messages into your logs. See MCP Client.

A tool that downloads a large file or reads a scanned document can take a while. It can now say what it is doing, and your app can show it before the result arrives:

class ReadReport < RubyLLM::Tool def execute(url:) progress "Downloading #{File.basename(url)}" pages = Scanner.pages(url) pages.each_with_index.map do |page, index| progress "Reading page #{index + 1} of #{pages.size}", value: index + 1, total: pages.size page.text end.join("\n") end end

chat.with_tools(ReadReport).after_tool_progress do |tool_call, progress| puts "#{tool_call.name}: #{progress.message}" end

MCP server tools report their progress through the same callback. It works with concurrent tool execution, on agents, and on Rails chat records. See Reporting Progress.

A few MCP servers can put hundreds of tool definitions in front of the model on every turn. Defer them, and the model searches for the tools it needs instead:

chat.with_mcp(GitHub, Linear, defer: true) chat.ask "Which open issues mention the flaky login spec?"

with_tools, the agent tools and mcp macros, and Rails chat records take the same defer: option, and tool and MCP classes declare it with defer. Anthropic and OpenAI’s Responses API search natively; other providers receive deferred tools as ordinary tools, so the same code runs everywhere. See Tool Search.

A conversation can move to another model or provider at any point, even in the middle of a tool round:

chat = RubyLLM.chat(model: "claude-sonnet-5").with_provider_tools(:web_search) chat.ask "What changed in the latest Ruby release?"

chat.with_model("gpt-5.6-luna") chat.ask "Which of those changes affect Rails apps?"

Thinking signatures, provider tool steps, compaction, and other provider-shaped content now go back only to the model that produced them. Another model receives the same conversation as text and tool calls, so it never sees blocks it cannot read. This matters inside one provider too: Bedrock, Vertex AI, Azure, and OpenRouter serve model families that refuse each other’s reasoning. Gemini continues a tool round another provider began, such as a call that waited for an approval while the user switched models.

A worker that dies mid-turn no longer leaves a conversation providers refuse. Requests leave out the blank reply it saved, and calls from a round the conversation has moved past reach the model as unfinished. See Understanding the Persistence Flow.

Some protocols cannot send every file type, such as a Word document. Convert those files yourself, and RubyLLM sends your replacement while the conversation keeps the original:

chat.convert_unsupported_attachments do |attachment| text = TextExtraction.call(attachment) RubyLLM::Attachment.new(StringIO.new(text), filename: "extracted.txt") if text end

chat.ask "Summarize this report.", with: "report.docx"

The block runs only for files the current protocol cannot send, once per file. Switch to a provider that reads the original, and it gets the original. See Converting Unsupported Attachments.

Small print needs more detail than a photo. Set resolution: on an attachment to trade tokens for detail, from :low to :ultra_high, or :original for the image as it is:

page = RubyLLM::Attachment.new("page-3.png", resolution: :original) chat.ask "What is the revenue figure?", with: page

Gemini, OpenAI, and Azure translate it to their own settings, other providers ignore it, and Rails keeps it with the stored file. See Choosing the Media Resolution.

A stable prefix and a growing conversation often deserve different cache lifetimes. Give a boundary its own ttl:, and the rest of the chat keeps the lifetime from with_caching:

chat = RubyLLM.chat.with_caching chat.with_instructions(policy).cache_until_here(ttl: "1h") chat.ask "Summarize today's tickets."

Anthropic, OpenRouter, and Bedrock Converse set the lifetime on each boundary, and Rails keeps it with the message. One-hour writes cost twice the input price on Anthropic, Claude on Vertex AI, and Bedrock. tokens.cache_write_by_ttl splits the writes by lifetime, and cost.cache_write prices each one at its own rate. See Explicit Cache Boundaries.

Prompt templates render partials the way Action View does, so instructions can share pieces:

<%# app/prompts/work_assistant/instructions.txt.erb %> <%= render "tone", display_name: display_name %> <%= render "shared/safety" %>

A bare name looks next to the current prompt, a path looks in the prompt roots, and local_assigns reads optional locals. See Partials.

Define questions about your application data and read probabilities, choices, and scores:

class TicketTriage < RubyLLM::Judge probability :urgent, "Does this need attention today?"

choice :department, "Which team should handle this?" do billing "Payments and refunds" technical "Bugs and integrations" other "Everything else" end end

judgment = TicketTriage.judge("Please refund the duplicate charge today.") judgment.urgent.probability judgment.department.choice

Judges use config.default_judgment_model unless you override the model. They accept structured input, reusable definitions, and runtime procs. Choice and score answers include full distributions and confidence so your application can choose how to act. See Judgments.

TypeSafe joins the built-in providers, bringing the total to eighteen. Use its hosted Jev models or a Jev-compatible local server through the same judgment API. OpenAI’s gpt-6-luna answers the same questions through OpenAI Decisions, including questions about images passed with with:. To judge locally, run Cloudflare’s Clef through Ollama with provider: :ollama.

Find out how often your agent gets the answer right. Write a class that runs your agent and a dataset of cases with reference answers:

app/evals/support_evaluation.rb

class SupportEvaluation < RubyLLM::Evaluation def perform(input) SupportAgent.new.ask(input) end end

app/evals/support_evaluation.yml

cases:

• name: unopened_return inputs: Can I return an unopened item after 14 days? expected_output: Yes, unopened items can be returned within 30 days.

bin/rails "ruby_llm:eval[SupportEvaluation]"

RubyLLM checks each answer against the reference with a model, and accepts different wording with the same meaning. Declare your own criteria, grade with an Agent or a Judge, add Ruby assertions on tool calls, and run the same cases as RSpec or Minitest tests. Reports include every verdi