Ruby AI LLM Rails Open Source RubyLLM

    RubyLLM 2.0: Tool Approval, Video, Speech, Citations, and Cost Tracking

    This video has music. Unmute to hear it.

    RubyLLM 2.0 is the biggest release since 1.0. It starts with tool approval:

    class RefundOrder < RubyLLM::Tool
  description "Refunds an order"
  parameter :order_id, description: "ID of the order to refund"
  requires_approval # that's all it takes

  def execute(order_id:, tool_call: nil)
    Refunds.issue(order_id:, idempotency_key: tool_call.id)
  end
end

class SupportAgent < RubyLLM::Agent
  chat_model Chat
  tools RefundOrder
end

chat = SupportAgent.create!
chat.ask "I was charged twice for order 42." # stops before refunding

# later, when the user approves
chat = SupportAgent.find(chat.id)
chat.approve(chat.pending_approvals.first)
chat.complete # issues the refund and answers

    The pending call is saved in your database, so the approval can come tomorrow, from another process, after two deploys.

    2.0 also generates video and speech, reads scanned documents, reranks search results, runs provider-hosted tools with typed citations, accounts for every provider attempt (including the ones that failed), and works across seventeen providers.

    There’s also a two-minute tour of 2.0.

    Each section below shows one feature and links to its guide.

    Tool Approval and Durable Agents

    Declare requires_approval on a tool and the loop stops before running it. chat.pending_approvals lists the waiting calls with their arguments, and approve or deny decides. With acts_as_chat, the call and its decision are database rows, so the approval can come from any process:

    class ApprovalsController < ApplicationController
  def create
    chat = Chat.find(params[:chat_id])
    chat.approve(params[:tool_call_id]) # or chat.deny(...)
    CompleteJob.perform_later(chat.id)
  end
end

class CompleteJob < ApplicationJob
  def perform(chat_id)
    SupportAgent.find(chat_id).complete # resumes where it stopped
  end
end

    There’s no workflow engine and no state machine to serialize: the transcript holds the state. The tool approval guide and durable agents guide cover the full flow.

    The Agentic Loop

    ask still runs the whole agentic loop for you. When you want control, 2.0 breaks it into verbs (ask_later, generate, run_tools, step, complete, cancel), and each one reads the persisted messages to decide what to do next, so you can run one step per job:

    chat = RubyLLM.chat.with_tools(SearchDocs).ask_later("How do I configure webhooks?")
chat.step until chat.complete? || chat.awaiting_approval?

    halt and RubyLLM::Tool::Halt are gone: your loop decides when to stop. I wrote about this in detail in The Agentic Loop, Exposed.

    New Operations

    2.0 adds video, speech, OCR, reranking, multimodal embeddings, and tokenization. None of them needs a chat:

    RubyLLM.animate("A red panda typing Ruby code").save("panda.mp4")
RubyLLM.speak("Welcome to the Ruby study group.").save("welcome.mp3")
RubyLLM.ocr("scanned-contract.pdf").markdown
RubyLLM.rerank("How do I reset my password?", documents, model: "rerank-v3.5")
RubyLLM.embed("A red panda", with: "panda.jpg", model: "gemini-embedding-2")
RubyLLM.tokenize("Hello, Ruby!", model: "grok-4.3").ids

    Each one has a guide: video, OCR, reranking, embeddings, and tokenization.

    Speech and Transcription

    2.0 adds text to speech with RubyLLM.speak, and transcription gains streaming, speaker labels, and word timestamps. Together they let a Rails action answer a voice message with audio:

    def create
  audio = params[:audio]
  return head :bad_request unless audio.is_a?(ActionDispatch::Http::UploadedFile)

  question = RubyLLM.transcribe(audio).text
  answer = Current.user.support_chat.ask(question).content

  speech = RubyLLM.speak(answer) # text to speech
  send_data speech.to_blob, type: speech.mime_type, disposition: "inline"
end

    See the text to speech guide and the transcription guide.

    Provider Tools

    with_provider_tools turns on tools the provider runs on its own infrastructure, like web search, code execution, file search, and remote MCP servers. The aliases are portable, so you can switch models without changing the code:

    chat = RubyLLM.chat(model: "claude-sonnet-5").with_provider_tools(:web_search) # runs on Anthropic's side
response = chat.ask "What changed in the latest Ruby release?"

response.citations.each { |citation| puts "#{citation.title}: #{citation.url}" }

    When OpenAI or Azure calls a tool on a remote MCP server, the call can wait in the same pending_approvals list as your Ruby tools. RubyLLM.research_later starts a hosted deep research agent and returns a job ID you can check from any process. The provider tools guide and the hosted research guide have the details.

    Citations

    Every provider formats citations differently. RubyLLM turns them all into RubyLLM::Citation, from documents and from web search, and your own RAG tools can return RubyLLM::SearchResults to make their results citable:

    chat = RubyLLM.chat(model: "claude-sonnet-5").with_citations # document citations
response = chat.ask "What are the termination conditions?", with: "contract.pdf"

response.citations.each do |citation|
  puts "p. #{citation.start_page}: #{citation.cited_text}"
end

    The citations guide has a footnote renderer you can copy.

    Files and Attachments

    RubyLLM.upload sends a file to the provider once, and with: takes the result like a path. Files too big to send inline are uploaded for you, and tools can now return files too:

    manual = RubyLLM.upload("manual.pdf", provider: :anthropic) # upload once

chat = RubyLLM.chat(model: "claude-sonnet-5")
chat.ask "Summarize the manual.", with: manual
chat.ask "What does the warranty cover?"

    See the files guide and the attachments guide.

    Cost and Usage

    2.0 records tokens and cost per provider attempt, because providers bill attempts: retries, fallbacks, and cancelled streams all count. In Rails, each attempt is a row in ruby_llm_usages, and an unknown price is nil, not zero:

    chat = RubyLLM.chat.with_fallbacks("claude-sonnet-5")
response = chat.ask "Explain Ruby fibers in one paragraph."

response.tokens.input
response.cost.total
chat.cost.total # every attempt, including the ones that failed

    The Tokens and Costs guide explains what each total includes.

    Fallbacks, Cancellation, and Errors

    with_fallbacks retries a failed request on backup models, in order, with the same conversation, tools, and settings. chat.cancel stops a run, and in Rails it works from another process through the database, so a stop button can end a streaming job. Agents declare error handlers once with rescue_from.

    chat = RubyLLM.chat(model: "claude-sonnet-5")
  .with_fallbacks("gpt-5.6", "gemini-3.7-flash") # tried in order if Claude fails

response = chat.ask "Summarize this incident report."

    The error handling guide covers fallbacks and errors, and Rails streaming covers the stop button.

    Prompt Caching

    with_caching gives prompt caching one API across providers, and cache_until_here marks an explicit cache boundary when you know where your reusable prefix ends:

    chat = RubyLLM.chat(model: "claude-sonnet-5").with_caching(ttl: "1h")

chat.with_instructions(analysis_prompt).cache_until_here # cache boundary
chat.add_message(role: :user, content: contract_text).cache_until_here

chat.ask "Does clause 14 conflict with the termination terms in clause 3?"
chat.ask "Which clauses mention liability caps?"

    The prompt caching guide has the details.

    Batches

    Batch APIs cost less than interactive requests. RubyLLM.batch submits ordinary chats to them, and in Rails the answers are saved through the same callbacks as a normal ask:

    chats = tickets.map do |ticket|
  RubyLLM.chat(model: "claude-haiku-4-5")
    .with_instructions("Classify this ticket as billing, bug, or feature request.")
    .ask_later(ticket.body)
end

batch = RubyLLM.batch(chats) # submits them all at once
batch.id # save this

    Later, RubyLLM::Batch.find picks the batch up from any process. The batch guide covers polling, results, and setup.

    Owning the Transcript

    chat.messages = messages_for_model replaces the history you send, so you can summarize old turns or redact values, and with_compaction lets providers that support it compact long conversations themselves. before_request edits the final payload, and render shows it without calling the model:

    chat = RubyLLM.chat(model: "gpt-5.6")
  .ask_later("Summarize the changes.")

chat.before_request do |payload|
  payload[:metadata] = { review_id: "review-42" }
end

chat.render[:metadata] # => { review_id: "review-42" }
chat.complete

    Details are in the request control guide and the persistence guide.

    Providers and Protocols

    Cohere, Ollama Cloud, ElevenLabs, and Deepgram join, bringing the total to seventeen, and built-in support went from 170 provider-feature pairs to 395 (coverage matrix). Providers and protocols are now separate: a provider is the service you connect to, a protocol is the API it speaks.

    RubyLLM.chat(model: 'gpt-5.4')                              # OpenAI Responses API
RubyLLM.chat(model: 'gpt-5.4', protocol: :chat_completions) # same model, old API
RubyLLM.chat(model: 'claude-opus-4-6', provider: :vertexai) # Vertex AI, Anthropic protocol

    If the provider you need isn’t built in, ruby_llm provider-gem Acme generates a gem for it, with tests, CI, and a model catalog. I covered the design in Providers, Protocols, and Provider Gems.

    Rails Table Ownership

    A fresh install now gives you two models instead of four. Chats and messages stay yours, and RubyLLM keeps models, tool calls, usage, and batches in its own ruby_llm_ tables, the way Active Storage owns its records. The record classes are internal and the readers are public, so RubyLLM can change its storage without your app noticing:

    # app/models/chat.rb
class Chat < ApplicationRecord
  acts_as_chat
  belongs_to :user
end

# app/models/message.rb
class Message < ApplicationRecord
  acts_as_message
  has_many_attached :attachments
end

    The Rails persistence guide shows every association and reader.

    Prompt Templates

    Prompts get a directory the way views do, app/prompts, and RubyLLM.render_prompt renders them. Agents pick up their instructions from it by convention, and Rails engines can ship their own prompts. Thanks to @kryzhovnik, who extracted the renderer out of Agent’s private methods, which made this possible.

    instructions = RubyLLM.render_prompt( # renders app/prompts/support/instructions.txt.erb
  "support/instructions",
  product_name: "BillingHub",
  customer_name: current_user.name
)

    The prompt rendering guide has the details.

    Workflows and Instrumentation

    RubyLLM.workflow names a piece of work and tags every event inside it, so model calls, tool calls, and costs trace back to the run that caused them. Control flow stays plain Ruby, with no graph DSL. Instrumentation grew from five events to twenty.

    RubyLLM.workflow("Write article", id: "article-42") do |workflow| # tags every event inside
  notes = workflow.step("Research") do
    ResearchAgent.new.ask(topic).content
  end

  workflow.step("Draft") do
    WriterAgent.new.ask(notes).content
  end
end

    The instrumentation guide lists every event and payload field.

    Coding Assistant Skill

    The gem ships a RubyLLM skill for coding assistants, maintained alongside the code and guides. Install the copy that matches your bundle:

    npx skills add "$(bundle show ruby_llm)" --skill rubyllm

    Your assistant stops writing 1.x method names from its training data and stops rebuilding things RubyLLM already does. Setup details.

    Upgrading

    Upgrade from 1.16; if you’re on something older, step through the minor releases first. Pin the 2.0 series, because the 1.16 upgrade generator ships with 2.0 only, and finish this upgrade, cleanup included, before you move to 2.1:

    gem "ruby_llm", "~> 2.0.0"

    Most code changes are renames to fully spelled-out names, such as desc to description, param to parameter, and with_params to with_provider_options. For Rails persistence, bin/rails generate ruby_llm:upgrade writes the migrations. Rename mode, the default, moves your existing tables into RubyLLM’s ownership while AI activity is paused. Copy mode (--mode copy) builds the 2.0 tables next to the old ones while 1.16 keeps serving traffic, and keeps a way back to 1.16.

    The complete 2.0 upgrade guide has every rename and the full procedure.

    Get It

    bundle add ruby_llm

    What’s New in 2.0 has working examples of everything above, the upgrade guide walks a Rails app from 1.16, and the documentation covers the rest.

    Thanks to everyone who tested the release candidates, filed issues, and sent code, including 22 people who made their first contribution during this cycle.

    RubyLLM 2.1 followed on October 8, at Deccan Queen on Rails in Pune.

    Newsletter

    About Me

    At Plenty, I build AI frameworks, products, native apps, and developer tools: RubyLLM, Chat with Work, Solco, Spotifast, ZapFast, ArchSpec, Kamal Backup, and more. I help organize Berlin.rb, founded Floppy Disco, and make house and disco as Crimson Lake.

    Tags

    Startups Open Source Ruby AI OpenAI Anthropic Gemini Rails LLM API Standards DeepSeek Async Concurrency Performance RubyLLM Linux Agents Developer Experience MCP Kamal Backups

    Recent Posts

    I Get 215 GitHub Notifications a Day, So I Built a Bot to Triage Them 1

    I Get 215 GitHub Notifications a Day, So I Built a Bot to Triage Them

    RubyLLM 2.1: You Can Have It All 2

    RubyLLM 2.1: You Can Have It All

    RubyLLM 2.0: Tool Approval, Video, Speech, Citations, and Cost Tracking 3

    RubyLLM 2.0: Tool Approval, Video, Speech, Citations, and Cost Tracking

    Newsletter