RubyLLM 2.1: You Can Have It All

RubyLLM 2.1 is out. I released it on stage at Deccan Queen on Rails in Pune.

With 2.1, RubyLLM does everything LangChain, LangGraph, and LangSmith do together, apart from two things it leaves out on purpose. Several things those frameworks leave to you or to their paid platform, such as per-user MCP OAuth, cancelling a run from another process, and provider batch APIs, are built in. The comparison has the details.

There’s also a three-minute tour of 2.1.

It builds on 2.0 and adds an MCP client, tool search, typed judgments, evaluations, tool progress, OpenTelemetry tracing, and a nineteenth provider. It’s also faster, with no code changes. In this post:

• Compared with LangChain, LangGraph, and LangSmith
• MCP client
• Tool search
• Judgments
• Evaluations
• Faster by default
• Tool progress
• OpenTelemetry
• Smaller changes
• Upgrading from 2.0

Compared with LangChain, LangGraph, and LangSmith

RubyLLM 2.1 next to the three Python frameworks combined:

The LangChain column comes from their documentation. They have more integrations. RubyLLM has one API across nineteen providers, and it fits into Rails: approvals are a controller action and a job, evaluations live in app/evals, and costs are rows in your database.

The two rows left out belong elsewhere. An MCP server is part of your application, so use the mcp gem or fast-mcp. For vector search, neighbor and pgvector already do it well.

An MCP Client Where You Decide What the Model Sees

In RubyLLM 2.1 an MCP server is a Ruby class you own: it lives in your repo, and it says exactly which tools the model gets.

class Linear < RubyLLM::MCP
  url "https://mcp.linear.app/mcp"
  inputs :user
  oauth owner: :user

  only :list_issues, :get_issue, :create_issue # the model sees these three
  requires_approval :create_issue
end

chat = RubyLLM.chat.with_mcp(Linear.new(user: current_user)) # one connection per user
chat.ask "What's blocking the release?"

Each user signs in to the server with their own account. RubyLLM handles OAuth discovery, registration, PKCE, and refresh, and stores each user’s credentials encrypted with Active Record encryption. MCP tools that requires_approval use the same approval flow as your own tools, so in Rails the job finishes and another one continues after the user approves. The client follows the MCP 2026-07-28 specification, and the official conformance suite runs in CI.

To try a server before shaping it, connect from the console. Every tool becomes a Ruby method:

>> docs = RubyLLM.mcp(url: "https://learn.microsoft.com/api/mcp")
>> docs.microsoft_docs_search(query: "Azure Blob Storage").text
=> "..."

The same class can rename tools, rewrite their descriptions, fix arguments, and wrap results, which is how 2.1 follows the advice from April that your agent’s context window is not a junk drawer: prototype with MCP, then craft the tools you control. Before this, ruby_llm-mcp by @patvice carried MCP for RubyLLM users for a long time. Thank you. If you use it, remove it before updating, since it defines the same RubyLLM::MCP constant. With MCP Apps, a server ships its own interface, such as a chart or a form, and RubyLLM hands it to your app to show next to the tool’s results. The MCP Client guide covers OAuth, input requests, resources, prompts, MCP Apps, and Tasks.

Tool Search

A few MCP servers can put hundreds of tool definitions in front of the model on every turn. Defer them, and the model searches for the tools it needs instead:

chat.with_mcp(GitHub.new(user: current_user), Linear.new(user: current_user), defer: true) # tools stay out of the context
chat.ask "Which open issues mention the flaky login spec?"

with_tools and agents take the same defer: option. Anthropic and OpenAI’s Responses API search natively, and other providers receive deferred tools as ordinary tools, so the same code runs everywhere. See Tool Search.

Judgments

Judgments answer questions about your own data (is this urgent, which team owns it) with probabilities a model measured, instead of a confidence number it wrote as text:

class TicketTriage < RubyLLM::Judge
  probability :urgent, "Does this need attention today?"

  choice :department, "Which team should handle this?" do
    billing   "Payments and refunds"
    technical "Bugs and integrations"
    other     "Everything else"
  end

  score :frustration, "How frustrated is the customer?",
    ["Calm", "Frustrated", "Angry"]
end

judgment = TicketTriage.judge("Please refund the duplicate charge today.")
judgment.urgent.probability # => 0.96
judgment.department.choice  # => :billing

probability, choice, and score questions are all asked over the same input in one request, and your code acts on them with thresholds you pick. Judges default to Jev models from TypeSafe, a new built-in provider. Kieran Klaassen added OpenAI’s gpt-6-luna through the Decisions API (#1008). To judge on your own machine, run Cloudflare’s Clef through Ollama. See the Judgments guide.

Evaluations

An evaluation runs your agent on cases with known answers and has a model grade each response, so you can tell whether a prompt or model change made things better. Cases live in a YAML file next to the class, and bin/rails "ruby_llm:eval[DocsEvaluation]" runs them and exits non-zero on failure, so it works in CI:

class DocsEvaluation < RubyLLM::Evaluation
  evaluation :correctness # keeps the default check
  evaluation :grounded, "Every claim is supported by the documents in metadata"
  evaluation :cites_sources, "The answer links to at least one document"

  def perform(question)
    DocsAgent.new.ask(question)
  end
end

Evaluations can also grade tool calls with plain Ruby assertions, use an Agent or a Judge as the grader, and run as RSpec or Minitest tests. See the Evaluations guide.

Faster by Default

2.1 does less work on every call, with no code changes. RubyLLM’s own work, 2.0.0 against 2.1 on the same machine:

Connections are now shared across calls, threads, and fibers. To keep them open, pick a persistent adapter (from the faraday-net_http_persistent gem):

config.faraday_adapter = :net_http_persistent # keeps connections open

The benchmarks need no API keys: clone RubyLLM and run bundle exec rake "benchmark:compare[v2.0.0]" to compare any version with your checkout. The connection guide covers adapters.

Tools That Report Progress

A slow tool can now tell your user what it’s doing instead of leaving them with a spinner:

class ReadReport < RubyLLM::Tool
  def execute(url:)
    pages = Scanner.pages(url)
    pages.each_with_index.map do |page, index|
      progress "Reading page #{index + 1} of #{pages.size}", value: index + 1, total: pages.size # report progress
      page.text
    end.join("\n")
  end
end

chat.with_tools(ReadReport).after_tool_progress do |tool_call, progress| # receives every update
  puts "#{tool_call.name}: #{progress.message}"
end

MCP tools report through the same callback. See Reporting Progress.

OpenTelemetry Tracing

One line sends every model call, tool run, and workflow as a span to the tracing backend you already use:

RubyLLM::OpenTelemetry.enable

Spans join the current trace, so the HTTP call your tool makes nests under that tool. They carry metadata such as models, tokens, and tool names, and never prompts, responses, or tool arguments. The OpenTelemetry guide has the full span reference.

Smaller Changes

• Hetzner is provider number nineteen: RubyLLM.chat(model: "Qwen3.8-27B", provider: :hetzner).
• Switching providers mid-conversation works even in the middle of a tool round: each model receives only the content it can read.
• Files a provider can’t read, such as Word documents, go through convert_unsupported_attachments, which sends your converted copy while the conversation keeps the original.
• Cache lifetimes per boundary: cache_until_here(ttl: "1h") keeps a stable prefix cached longer than the rest of the chat.
• Prompt partials: prompt templates render shared pieces the way Action View does.
• Per-tenant agents: a context block can use the agent’s inputs, so each workspace can bring its own API key. Thanks to mikemikimike (#903).
• A secondary database can hold RubyLLM’s supporting records next to your chats.
• Usage beyond chats: in Rails, embeddings, transcriptions, and other one-shot calls go into the usage ledger too.
• error.request_shape lists every turn’s parts and sizes when a provider rejects a request.
• Provider uploads are reused across processes in Rails, so a large PDF isn’t uploaded again on every job.
• Perplexity chat runs on the Agent API, since Sonar retires.
• Ruby 3.2 or later is required, since 3.1 reached end of life.
• JSON 3 is allowed, thanks to Filipe Kalicki (#968), who also made model lookups use an index (#981).

Upgrading from 2.0

gem "ruby_llm", "~> 2.1.0"

bundle update ruby_llm
bin/rails generate ruby_llm:upgrade
bin/rails db:encryption:init   # only for MCP OAuth, if your app has no encryption keys yet
bin/rails db:migrate

Most 2.0 apps need no code changes. The few that do are listed in the upgrade guide, and everything new is in What’s New in 2.1.

Thanks to everyone who sent code for this release: Andrii Furmanets, Andrey Samsonov, Andy Wang, Anton Kopylov, Filipe Kalicki, Guilherme Lages Santos, Islam Gagiev, Kieran Klaassen, Marc Köhlbrugge, mikemikimike, Mikhail Topolskiy, Muhammad Zain Ul Abidin, Paul Arterburn, and Viktor Schmidt.