Researchers call it context rot: the more tokens in the window, the harder it becomes for the model to follow instructions, retrieve information, and stay on task. Chroma tested 18 frontier models and found that accuracy drops up to 30% when you go from a focused 300-token input to 113k tokens of conversation history, with the task held constant. The model essentially became dumber.

This holds true regardless of how big the window is, yet most agent setups treat the context window like a junk drawer.

“Just toss it in there, the LLM will figure it out!”

MCP: the biggest offender

Don’t get me wrong. MCP is a fine idea. You need to talk to a service? Grab an MCP server, plug it in, and you’re running in ten minutes. For prototyping, for exploration, for answering “is this even worth building?”, it’s great.

The problem is what happens next. Which is: nothing.

People leave the MCP servers plugged in. They add more. Every MCP server you connect dumps tool descriptions, schemas, and instructions into your context. You didn’t write those. You didn’t optimize them. You probably haven’t even read them. You’re handing over a chunk of your context window to whatever some third party decided to shove in there.

Say you need a tool that checks the weather. You could plug in an MCP server and get dozens of tool descriptions, parameter schemas, and whatever instructions its author decided to write. Or you could write this:

class Weather < RubyLLM::Tool
  description "Gets current weather for a location"

  param :latitude, desc: "Latitude (e.g., 52.5200)"
  param :longitude, desc: "Longitude (e.g., 13.4050)"

  def execute(latitude:, longitude:)
    url = "https://api.open-meteo.com/v1/forecast?latitude=#{latitude}&longitude=#{longitude}&current=temperature_2m,wind_speed_10m"
    Faraday.get(url).body
  rescue => e
    { error: e.message }
  end
end

Twelve lines of RubyLLM. You wrote the description, so you know exactly what tokens are going into your context. You wrote the parameters, so the model gets precisely the interface it needs, no more. You own it, you can tune it, and nobody can inject anything into your agent’s brain through it.

Use MCP to prototype. Then replace it with crafted tools you actually control.

Tool responses are context too

Your RAG retrieves ten full documents when the model needs a paragraph. Your API call returns a massive JSON blob when the model needs two fields. You’re paying for every one of those tokens with your agent’s IQ.

The fix is progressive disclosure. At Chat with Work, when the agent searches your Google Drive, we don’t dump entire files into context. The search tool returns only some metadata and a single line from the file, the line that matched the search keywords. Fifty results, fifty lines. The AI reads those, decides which files actually matter, and only then reads them. If a file is too large, it reads it in chunks. At every step, the model is only looking at what it needs.

The same principle applies to any tool. Don’t return everything. Return enough for the model to decide what to look at next.

Your instructions are context too

Then there’s the stuff you wrote yourself. Your system prompt is context. Your tool descriptions are context. Your parameter schemas are context. Every edge case, every guardrail, every overly detailed description competes for attention. You think you’re being thorough. You’re actually drowning the instructions that matter in a sea of instructions that don’t. A focused system prompt will outperform an exhaustive one every time.

Tool count is context too

You hand-crafted 40 beautiful tools. Your agent needs 5 for this task. The other 35 sit in context doing nothing except making the model slower at picking the right one.

Don’t register every tool your agent might ever need. Load the tools the current task actually requires. If you’re building a support agent that handles billing and technical issues, don’t give it all of both. Route billing questions to a billing agent and technical questions to a technical agent. Two focused agents will outperform one bloated one.

Every token should earn its place

The context window is not a junk drawer. It’s a workbench. Everything on it should be there for a reason, and you should be able to say what that reason is.

So before you plug in another MCP server, add another RAG source, or write another paragraph in your system prompt, ask yourself one question: is this worth making my agent dumber?

Then it gets worse. You unplug your laptop, go to a conference, plug into a projector, and you’re back to editing config files backstage before your talk. You come home, dock the laptop, and the layout is wrong again.

I looked at what was available. The closest to what I wanted was Monique: spatial editor, profiles, workspace management, a hotplug daemon. It does exactly what I need. But it’s a GTK4 GUI that pulls in Python and a stack of dependencies, and the daemon was broken when I tried it. The other tools each cover parts of this: kanshi does profiles and auto-switching but has no editor, you write config files; nwg-displays and HyprMon have spatial editors but no daemon; HyprDynamicMonitors has a daemon but no real layout tool, and it pulls in UPower and D-Bus.

I wanted Monique’s feature set without the dependency baggage, in something that works over SSH when your monitors are broken. So I built hyprmoncfg.

A real spatial editor, in your terminal

The TUI is the thing I’m most proud of. It’s not a config editor with a preview pane. It’s a full spatial layout tool.

The left side is a canvas where your monitors are drawn as rectangles, proportional to their resolution. You click one to select it, drag it to move it. Monitors snap to each other’s edges as you position them, just like arranging windows in a GUI display manager. Arrow keys give you fine control: 100px per step, Shift for 10px, Ctrl for 1px.

The right side is a per-monitor inspector. Pick a resolution and refresh rate from a scrollable list. Set scale, position, transform, VRR, mirroring. All inline, no dialogs within dialogs. A third tab handles workspace planning.

And because it’s a TUI: it works over SSH. When your monitor configuration is broken and you can’t see anything, you can SSH into the machine and fix it. Try that with a GTK app.

Safe apply with automatic revert

Every apply, whether from the TUI or the daemon, follows the same path: write monitors.conf atomically (temp file + rename, no corruption), reload Hyprland, re-read the actual monitor state, and verify the result matches what was requested.

Then it gives you 10 seconds to confirm. If you don’t, maybe because the layout left you staring at a black screen, it reverts automatically. No stuck monitors. No reaching for a second machine to undo the damage.

This is the same apply engine everywhere. The TUI and the daemon share identical code. If it works when you test it interactively, it works when the daemon fires at 2am because you bumped your dock cable.

Workspace planning

Monitor configuration and workspace assignment are the same problem. If you’re rearranging monitors, you probably want workspaces to follow. hyprmoncfg has a workspace planner built into its third tab, with three strategies:

• Sequential: Groups in chunks. Workspaces 1-3 on monitor A, 4-6 on monitor B.
• Interleave: Round-robins. 1→A, 2→B, 3→A, 4→B.
• Manual: Explicit per-workspace rules when you want full control.

Workspace assignments are stored inside each profile and applied together with the layout. Switch profiles, switch workspace distribution. One operation.

Source-chain verification

Here’s something no other tool does. Before writing anything, hyprmoncfg parses your hyprland.conf and verifies it actually sources the target monitors.conf. If it doesn’t, it refuses to write.

Other tools skip this check. They silently update a file that Hyprland never reads. You spend twenty minutes debugging why nothing changed, only to realize the file was never sourced. I lost an evening to this once. Never again.

Dotfiles integration

Profiles are stored as JSON files in ~/.config/hyprmoncfg/profiles/, one per profile. The generated monitors.conf is a build artifact, you don’t commit it. You commit the profiles.

chezmoi add ~/.config/hyprmoncfg

Save a “desk” profile at home with your ultrawide. Save “conference-1080p” at one venue. Save “conference-4k” at another. Sync them across machines via your dotfiles. The daemon matches profiles to connected hardware automatically. Arrive somewhere, plug in, and the right layout applies.

This is portable. The same profile library works across machines because matching is based on the monitors you have, not on the machine you’re at.

One runtime dependency: Hyprland

Two compiled Go binaries. No Python, no GTK, no GObject introspection, no D-Bus, no UPower. Install them and you’re done. The only runtime requirement is Hyprland itself.

How it compares

Try it

On Arch:

yay -S hyprmoncfg

Or build from source:

go install github.com/crmne/hyprmoncfg/cmd/hyprmoncfg@latest
go install github.com/crmne/hyprmoncfg/cmd/hyprmoncfgd@latest

Check out the documentation for the full guide, or browse the source on GitHub.

“Add more integrations, finish the security assessment, market it well.” I said.

That didn’t help. “All those LLM providers are going to eat the whole market. They’ll ship every integration you can think of. If you want a slice of the pie, you need to pick a vertical and own it.”

I told him I was going to grab a T shaped slice of the pie instead.

He looked at me like I’d lost it.

Here’s the thing about the “pick a vertical” advice: it’s not wrong. It’s just not the only way. And for a lot of small software companies, it’s a trap dressed up as strategy.

The conventional wisdom goes like this: the market is huge, the big players are coming, so you’d better find your little corner and defend it. Specialize. Go deep. Become the AI assistant for dentists in Luxembourg or the knowledge tool for corporate lawyers in Berlin-Brandenburg. Calculate your total addressable market. Build a defensible moat. Make investors happy.

But what if you don’t care about making investors happy? Most companies don’t need investors. What if you just want to build something good?

The comb

I said T shaped in the moment. One horizontal, one vertical. But the more I thought about it, the more teeth it grew. Less like a T, more like a comb.

Here’s why. When you’re OpenAI or Google, you sample from the top of the distribution. You build what most people use first, then work your way down. The result is always the same: a broad horizontal platform that serves everyone and surprises no one.

When you’re small, you sample from what’s right in front of you. You build for yourself because no amount of user research, design thinking, or theory of mind will ever match the depth of actually needing the thing you’re making. You understand your own problems in a way that connects to your emotions, your workflow, your instincts. You can’t fake that. You can’t interview your way to it. I chose fast onboarding over full sync, because I don’t want to wait to start working. Nextcloud, Todoist, IMAP, and CalDAV: that’s my stack, so that’s where I’ll go deep next.

Then you listen to your customers. “This is cool, but I use Slack.” So you build that too. A team needs to own their data, so you add on-premises installation. Someone uses Basecamp, and you build that integration because the people behind it think like you. One tooth at a time.

The shape that emerges is yours. Not because you planned it on a whiteboard, but because you started from yourself and grew outward. It works for the small teams, the freelancers, the music collectives, the people who don’t have an IT department and don’t want one. That’s the comb: not a strategy you choose, but what naturally happens when you’re small and you give a damn.

There’s a reason people still choose Linear over Jira, or Proton over Gmail, or Plausible over Google Analytics. It’s not because the small player has more features. It’s because someone built it for themselves first, and that resonated. The entire market doesn’t need to resonate with you. Just enough of it.

So yes, the big players are coming. They’re going to ship a lot of integrations. They’re going to spend a lot of money. And they’re going to build software that feels like it was built by a company that spends a lot of money.

I’ll be over here, grabbing my comb shaped slice of pie. It’s Plenty.

Today also happens to be the day I officially founded Plenty. The papers are signed. The comb is real!

The sidebar navigation. The “On this page” outline on the right. The search that pops up with /. The homepage that actually looks like a product page, not a README with a nav bar. Dark mode that just works. Code blocks with copy buttons and language labels. It all looks like someone sat down and designed the whole experience.

Because someone did. VitePress is genuinely great. And Ruby developers know it, because some of the most visible projects in our community are shipping their docs on VitePress. Not on a Jekyll theme, not on a Ruby tool. On a JavaScript static site generator built for Vue.

I don’t blame them. I looked at what we had in the Jekyll ecosystem and understood immediately. The best option is Just the Docs, and I’ve been using it for RubyLLM. It’s solid. But I had to patch in proper dark mode support that follows the browser setting. I had to add a copy-page button. The homepage layout is narrow and document-y. It works. It doesn’t wow.

So I built Jekyll VitePress Theme.

What It Is

A Jekyll theme gem that recreates the VitePress documentation experience. Everything you’d expect:

• Top nav with mobile menu
• Left sidebar, right “On this page” outline
• Homepage layout with hero section and feature cards
• Built-in local search (press / or Cmd+K)
• Dark/light/auto appearance toggle
• Code blocks with copy buttons, language labels, and file title bars
• Doc footer with edit link, previous/next pager, and “last updated”
• GitHub star widget
• Rouge syntax highlighting with separate light and dark themes

All configured through _config.yml and _data/*.yml files. No JavaScript toolchain. No Node.js. Just Jekyll.

Getting Started

gem "jekyll-vitepress-theme"

theme: jekyll-vitepress-theme
plugins:
  - jekyll-vitepress-theme

jekyll_vitepress:
  branding:
    site_title: My Project

bundle install
bundle exec jekyll serve --livereload

That’s it. Your docs site now looks like VitePress. Customize the nav, sidebar, colors, fonts, and everything else from the configuration reference.

Why This Matters

When I came back to Ruby in 2024, I kept finding things that could be better. There wasn’t a great LLM library, so I built RubyLLM. Async deserved more attention, so I blogged about it. And our documentation sites? They didn’t look the part.

In open source, looks matter. A beautiful docs site tells potential users: this project is serious, maintained, and worth your time. It lowers the barrier to adoption. It makes people want to try your library.

VitePress understood this. Now Jekyll has it too.

gem "jekyll-vitepress-theme", "~> 1.0"

Why This Matters

RubyLLM turned one last week. 1.0 shipped on March 11, 2025 with Rails integration from day one: ActiveRecord models, acts_as_chat, Turbo streaming, persistence out of the box. 1.4 added the install generator. 1.7 brought the first scaffold chat UI with Turbo Streams. 1.12 introduced agents with prompt conventions. Each release got closer to the same thing: AI that works the way Rails works.

1.14 fully realizes that goal. A beautiful Tailwind chat UI (with automatic fallback to scaffold if you’re not using Tailwind). Generators for agents and tools. Conventional directories for everything. All of it extracted from Chat with Work, where it’s been running in production for months.

What You Get

Two generators. That’s it.

bin/rails generate ruby_llm:install
bin/rails generate ruby_llm:chat_ui

Your app now has this structure:

app/
├── agents/
├── controllers/
│   ├── chats_controller.rb
│   └── messages_controller.rb
├── helpers/
│   └── messages_helper.rb
├── jobs/
│   └── chat_response_job.rb
├── models/
│   ├── chat.rb
│   ├── message.rb
│   ├── model.rb
│   └── tool_call.rb
├── prompts/
├── schemas/
├── tools/
└── views/
    ├── chats/
    │   ├── index.html.erb
    │   ├── show.html.erb
    │   └── _chat.html.erb
    └── messages/
        ├── _assistant.html.erb
        ├── _user.html.erb
        ├── _tool.html.erb
        ├── _error.html.erb
        ├── create.turbo_stream.erb
        ├── tool_calls/
        │   └── _default.html.erb
        └── tool_results/
            └── _default.html.erb

Separate partials for each message role. Turbo Stream templates for real-time updates via broadcasts_to. A background job that handles the AI response. Tool calls and tool results each get their own rendering pipeline. A complete Tailwind chat interface, not a scaffold you need to fight with.

Full Tutorial: New App from Scratch

If you want to start from zero, this is what the demo shows. The whole thing takes just a minute.

rails new chat_app --css tailwind
cd chat_app
bundle add ruby_llm
bin/rails generate ruby_llm:install
bin/rails generate ruby_llm:chat_ui
bin/rails db:migrate
bin/rails ruby_llm:load_models
bin/dev

That’s a new Rails app with Tailwind, RubyLLM installed, the chat UI generated, the database set up, models loaded, and the server running. Open localhost:3000/chats and start talking to an AI.

Generators for Agents, Tools, and Schemas

Now the fun part. You scaffold agents, tools, and schemas the same way you’d scaffold anything else in Rails:

bin/rails generate ruby_llm:agent SupportAgent

app/
├── agents/
│   └── support_agent.rb
└── prompts/
    └── support_agent/
        └── instructions.txt.erb

The agent class comes with the 1.12 DSL ready to go. The instructions file is an ERB template for your system prompt, so you can version it, review it in PRs, and template it with runtime context.

bin/rails generate ruby_llm:tool WeatherTool

app/
├── tools/
│   └── weather_tool.rb
└── views/
    └── messages/
        ├── tool_calls/
        │   └── _weather.html.erb
        └── tool_results/
            └── _weather.html.erb

Each tool gets its own partials for rendering calls and results. Show a weather widget for the weather tool, a search results list for a search tool, all through Rails partials.

bin/rails generate ruby_llm:schema Product

app/
└── schemas/
    └── product_schema.rb

This creates a schema for structured output validation.

More on all of this in the Rails integration docs, and the dedicated guides for agents and tools.

Self-Registering Provider Config

For people building provider gems: providers now register their own configuration options instead of patching a monolithic Configuration class.

class DeepSeek < RubyLLM::Provider
  class << self
    def configuration_options
      %i[deepseek_api_key deepseek_api_base]
    end
  end
end

When the provider is registered, its options become attr_accessors on RubyLLM::Configuration automatically. Third-party gems can add their config keys without touching the core.

Bug Fixes

• Faraday logging memory bloat: logging no longer serializes large payloads (like base64-encoded PDFs) when the log level is above DEBUG.
• Agent assume_model_exists propagation: setting this on the agent class now actually works.
• Renamed model associations: foreign key references with acts_as helpers are fixed.
• MySQL/MariaDB compatibility: JSON column defaults work correctly now.
• Error.new with string argument: no longer raises a NoMethodError.

Full list in the release notes.

gem 'ruby_llm', '~> 1.14'

If your goal is to ship AI applications in 2026, Ruby is the best language to do it.

The AI Training Ecosystem Is Irrelevant

Python owns model training. PyTorch, TensorFlow, the entire notebooks-and-papers gravity well. Nobody disputes that.

But you’re not training LLMs. Almost nobody is. Each training run costs millions of dollars. The dataset is the internet!

This is what AI development today looks like:

curl https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{"model": "gpt-5.2", "messages": [{"role": "user", "content": "Hello"}]}'

That’s it. An HTTP call.

The entire Python ML stack is irrelevant to achieve this. What matters is everything around it: streaming responses to users, persisting conversations, tracking costs, switching providers when pricing changes.

That’s web application engineering. That’s where Ruby and Rails shine like no other.

“You Need a Complex Agent Framework or You’re Not Doing Real AI”

Bullshit.

You need a beautiful, truly provider-independent API. Let me show you.

Python vs JavaScript vs Ruby LLM Libraries

Simple chat

Python (LangChain):

from langchain.chat_models import init_chat_model
from langchain.messages import HumanMessage

model = init_chat_model("gpt-5.2", model_provider="openai")
response = model.invoke([HumanMessage("Hello!")])

You need to specify the provider, create an array of messages that need to be instantiated, etc.

That’s ceremony.

JavaScript (AI SDK):

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const { text } = await generateText({
  model: openai('gpt-5.2'),
  prompt: 'Hello!',
});

What if you want to use a model from another provider?

Ruby (RubyLLM):

require 'ruby_llm'

RubyLLM.chat.ask "Hello!"

Reads like it should.

Token usage tracking

If you’re running AI in production, you need to track token usage. This is how you price your app.

LangChain (GPT):

response = model.invoke([HumanMessage("Hello!")])
response.response_metadata['token_usage']
# {'completion_tokens': 12, 'prompt_tokens': 8, 'total_tokens': 20}

LangChain (Claude):

response.response_metadata['usage']
# {'input_tokens': 8, 'output_tokens': 12}

Different key and different structure!

LangChain (Gemini):

response.response_metadata
# ...nothing...

It’s not even there!

RubyLLM:

response.tokens.input   # => 8
response.tokens.output  # => 12

Same interface. Every provider. Every model.

Agents

This is where it gets fun.

Python (LangChain):

from langchain_openai import ChatOpenAI
from langchain.agents import create_agent

model = ChatOpenAI(model="gpt-5-nano")

graph = create_agent(
    model=model,
    tools=[search_docs, lookup_account],
    system_prompt="You are a concise support assistant",
)

inputs = {"messages": [{"role": "user", "content": "How do I reset my API key?"}]}

for chunk in graph.stream(inputs, stream_mode="updates"):
    print(chunk)

JavaScript (AI SDK 6):

import { ToolLoopAgent } from 'ai';
import { openai } from '@ai-sdk/openai';

const supportAgent = new ToolLoopAgent({
  model: openai('gpt-5-nano'),
  system: 'You are a concise support assistant.',
  tools: { searchDocs, lookupAccount },
});

const { text } = await supportAgent.generateText({
  messages: [{ role: 'user', content: 'How do I reset my API key?' }],
});

Ruby (RubyLLM):

require 'ruby_llm'

class SupportAgent < RubyLLM::Agent
  model "gpt-5-nano"
  instructions "You are a concise support assistant."
  tools SearchDocs, LookupAccount
end

SupportAgent.new.ask "How do I reset my API key?"

Pure joy.

It’s About Cognitive Overhead

This isn’t just about aesthetics.

It’s about cognitive overhead: how many abstractions, how many provider-specific details, how many different data structures you need to hold in your head instead of focusing on what really matters: prompts and tool design.

Low cognitive overhead compounds: faster onboarding, fewer accidental bugs, easier refactors, and cleaner debugging when production explodes at 2AM.

Ruby’s advantage here is cultural: elegant APIs are treated as first-class engineering work, not icing on the cake.

Rails Gives You the Rest of the Product for Free

Model calls are only a small chunk of your code. The rest makes up the bulk of it: auth, billing, background jobs, streaming UI, persistence, admin screens, observability, even native apps.

Rails gives you a beautiful, coherent answer for all of it.

With RubyLLM + Rails, the core streaming loop is tiny:

class ChatResponseJob < ApplicationJob
  def perform(chat_id, content)
    chat = Chat.find(chat_id)

    chat.ask(content) do |chunk|
      message = chat.messages.last
      message.broadcast_append_chunk(chunk.content) if chunk.content.present?
    end
  end
end

And on the model side:

class Chat < ApplicationRecord
  acts_as_chat
end

class Message < ApplicationRecord
  acts_as_message
  has_many_attached :attachments
end

This gives you streaming chunks to your web app and persistence in your DB in absurdly few lines of code.

It Scales

“Ruby can’t handle AI scale.”

Wrong.

LLM workloads are mostly network-bound and streaming-bound. That’s exactly where Ruby’s Async ecosystem shines. Fibers let you handle high concurrency without thread explosion and resource waste. No need to plaster the code with async/await keywords. RubyLLM became concurrent with 0 code changes.

I wrote a deep dive here: Async Ruby is the Future of AI Apps (And It’s Already Here)

Don’t Take My Word for It

Someone ported RubyLLM’s API design to JavaScript as NodeLLM. Same design. Clean code, good docs.

The JavaScript community’s response: zero upvotes on Reddit. 14 GitHub stars. Top comments: “How’s this different from AI SDK?” and “It’s always fun when you AI bros post stuff. They all look and sound the same. Also, totally unnecessary.”

RubyLLM: #1 on Hacker News. ~3,600 stars. 5 million downloads. Millions of people using RubyLLM-powered apps today.

Same design. Wildly different reception. That tells you everything about which community is ready for this moment.

And teams that switched from Python are not going back:

We had a customer deployment coming up and our Langgraph agent was failing. I rebuilt it using RubyLLM. Not only was it far simpler, it performed better than the Langgraph agent.

Our first pass at the AI Agent used langchain… it was so painful that we built it from scratch in Ruby. Like a cloud had lifted. Langchain was that bad.

At Yuma, serving over 100,000 end users, our unified AI interface was awful. RubyLLM is so much nicer than all of that.

These aren’t people who haven’t tried Python. They tried it, shipped it, and replaced it.

Go Ship AI Apps with Ruby, Rails, and RubyLLM

When we freed ourselves from complexity, this community built Twitter, GitHub, Shopify, Basecamp, Airbnb. Rails changed web development forever.

Now we have the chance to change AI app development. Because AI apps are all about the product. And nobody builds products better than Ruby developers.

So let’s start from first principles.

What’s an Agent?

An agent is an LLM that can call functions.

That’s it. When you give a language model a set of tools it can invoke – a database lookup, an API call, a file operation – and the model decides when and how to use them, you have an agent. The model reasons about the problem, picks the right tool, looks at the result, and continues reasoning. Sometimes it calls several tools in sequence. Sometimes none.

There’s no special “agent mode.” No orchestration engine. No graph of nodes. It’s just a conversation where the model can do things besides talk.

RubyLLM Always Had This

Tool calling has been a core feature of RubyLLM since 1.0:

class SearchDocs < RubyLLM::Tool
  description "Searches our documentation"
  param :query, desc: "Search query"

  def execute(query:)
    Document.search(query).map(&:title)
  end
end

chat = RubyLLM.chat
chat.with_tool(SearchDocs)
chat.ask "How do I configure webhooks?"
# Model searches docs, reads results, answers the question

That’s an agent. The model decides to search, interprets the results, and responds. You didn’t need a special class or framework to make this happen.

But there was a problem.

The Reuse Problem

In a real application, you don’t configure a chat once. You configure it in controllers, background jobs, service objects, API endpoints. The same instructions, the same tools, the same temperature – scattered across your codebase:

# In the controller
chat = RubyLLM.chat(model: 'gpt-4.1')
chat.with_instructions("You are a support assistant for #{workspace.name}...")
chat.with_tools(SearchDocs, LookupAccount, CreateTicket)
chat.with_temperature(0.2)

# In the background job
chat = RubyLLM.chat(model: 'gpt-4.1')
chat.with_instructions("You are a support assistant for #{workspace.name}...")
chat.with_tools(SearchDocs, LookupAccount, CreateTicket)
chat.with_temperature(0.2)

# In the service object...
# You get the idea

Every Rubyist’s instinct kicks in: this should be a class.

RubyLLM 1.12: A DSL for Agents

That’s exactly what 1.12 adds. Define your agent once, use it everywhere:

class SupportAgent < RubyLLM::Agent
  model 'gpt-4.1'
  instructions "You are a concise support assistant."
  tools SearchDocs, LookupAccount, CreateTicket
  temperature 0.2
end

# Anywhere in your app
response = SupportAgent.new.ask "How do I reset my API key?"

Every macro maps to a with_* call you already know. model maps to RubyLLM.chat(model:). tools maps to with_tools. instructions maps to with_instructions. No new concepts. Just a cleaner way to package what you were already doing.

Runtime Context

Static configuration is only half the story. Real agents need runtime data – the current user, the workspace, the time of day. Agents support lazy evaluation for this:

class WorkAssistant < RubyLLM::Agent
  chat_model Chat
  inputs :workspace

  instructions { "You are helping #{workspace.name}" }

  tools do
    [
      TodoTool.new(chat: chat),
      GoogleDriveTool.new(user: chat.user)
    ]
  end
end

chat = WorkAssistant.create!(user: current_user, workspace: @workspace)
chat.ask "What's on my todo list?"

Blocks and lambdas are evaluated at runtime, with access to the chat object and any declared inputs. Values that depend on runtime context must be lazy – a constraint that Ruby makes trivially natural.

Prompt Conventions

If you’re using Rails, agents follow a convention for prompt management:

class WorkAssistant < RubyLLM::Agent
  chat_model Chat
  instructions display_name: -> { chat.user.display_name_or_email }
end

This renders app/prompts/work_assistant/instructions.txt.erb with display_name available as a local. Namespaced agents map naturally: Admin::SupportAgent looks in app/prompts/admin/support_agent/.

Your prompts are ERB templates. Version them in git. Review them in PRs. Treat them like the application code they are.

Rails Integration

The chat_model macro activates Rails-backed persistence:

class WorkAssistant < RubyLLM::Agent
  chat_model Chat
  model 'gpt-4.1'
  instructions "You are a helpful assistant."
  tools SearchDocs, LookupAccount
end

# Create a persisted chat with agent config applied
chat = WorkAssistant.create!(user: current_user)

# Load an existing chat, apply runtime config
chat = WorkAssistant.find(params[:id])

# User sends a message, everything persisted automatically
chat.ask(params[:message])

create! persists both the chat and its instructions. find applies configuration at runtime without touching the database. This distinction matters when your prompts evolve faster than your data.

Also in 1.12

Agents are the headline, but this release also adds:

• AWS Bedrock full coverage via the Converse API – every Bedrock chat model through one interface
• Azure Foundry API – broad model access across Azure’s ecosystem
• Clearer with_instructions semantics – explicit append options, guaranteed message ordering

Already in Production

This isn’t a spec or a proposal. The agent DSL powers Chat with Work in production right now. The WorkAssistant examples above aren’t hypothetical – they’re simplified versions of real code handling real conversations.

If you want to see what it feels like, try it out.

The Point

The industry is making agents complicated. They’re not. An agent is an LLM with tools. You define the tools in Ruby. You package them in a class. You use the class in your app.

No graphs. No chains. No orchestration frameworks. Just Ruby.

gem 'ruby_llm', '~> 1.12'

In the Omarchy world, Hyprwhspr is getting a lot of attention after a recent DHH tweet:

He’s right: local dictation is shockingly good now. The catch is Hyprwhspr uses Python virtual environments, which don’t mix well with mise. Fortunately Pete Jackson saw that and created Voxtype to solve exactly this issue!

EDIT: five minutes after I posted this, DHH confirmed that Voxtype ships will ship with Omarchy 3.3! 🎉

Why Voxtype

Voxtype is built in Rust, so you don’t need Python virtual environments which means it works well with mise. It’s fast, it just works, and when I opened an issue asking for an Omarchy theme, the author shipped it immediately. Now it looks stunning in my setup.

With Vulkan enabled, transcription is almost instant on my Ryzen AI 9 HX370. The video at the top is not sped up. Longer text also transcribes instantly.

If you want to copy my exact configuration, here it is.

Install

sudo pacman -S wtype ydotool wl-clipboard vulkan-icd-loader # last only if you want to use your GPU
sudo yay -S voxtype

voxtype setup --download
voxtype setup gpu # if you want to use your GPU
voxtype setup systemd

Restart Waybar after the changes:

pkill -SIGUSR2 waybar

Voxtype config

~/.config/voxtype/config.toml

state_file = "auto"

[hotkey]
enabled = false

[audio]
device = "default"
sample_rate = 16000
max_duration_secs = 600

[audio.feedback]
enabled = true
# Sound theme: "default", "subtle", "mechanical", or path to custom theme directory
theme = "default"
volume = 0.7

[whisper]
model = "base.en"
language = "en"
translate = false
on_demand_loading = true # saves your GPU until it's needed

[output]
mode = "type"
fallback_to_clipboard = true

# Delay between typed characters in milliseconds
# 0 = fastest possible, increase if characters are dropped
type_delay_ms = 1

[output.notification]
on_recording_start = false
on_recording_stop = false
on_transcription = true

[text]
replacements = { "hyperwhisper" = "hyprwhspr" }

[status]
icon_theme = "omarchy"

Waybar integration

~/.config/waybar/config.jsonc

"custom/voxtype": {
  "exec": "voxtype status --follow --format json",
  "return-type": "json",
  "format": "{}",
  "tooltip": true
},

And add it to modules-right:

"modules-right": [
  "group/tray-expander",
  "custom/voxtype",
  "bluetooth",
  "network",
  "pulseaudio",
  "cpu",
  "battery"
]

~/.config/waybar/style.css

@import "voxtype.css";
@import "../omarchy/current/theme/waybar.css";

~/.config/waybar/voxtype.css

#custom-voxtype {
  margin: 0 16px 0 0;
  font-size: 12px;
  font-weight: bold;
  border-top: 2px solid transparent;
  border-bottom: 2px solid transparent;
  transition: color 150ms ease-in-out, border-color 150ms ease-in-out;
}

#custom-voxtype.recording {
  color: #ff5555;
  animation: pulse 1s ease-in-out infinite;
}

#custom-voxtype.transcribing {
  color: #ff5555;
}

#custom-voxtype.stopped {
  color: #6272a4;
}

@keyframes pulse {
  0% { opacity: 1; }
  50% { opacity: 0.5; }
  100% { opacity: 1; }
}

Keybinding

In your Hyprland config:

# Voxtype
bindd = SHIFT, XF86AudioMicMute, Transcribe, exec, voxtype record toggle

That’s it. Use your voice whenever possible. It’s faster, more natural, and keeps you in flow.

Once you know that quirk, it’s straightforward. Only caveat: you need the latest trunk or v1.9+, because that’s where we taught RubyLLM to unpack inline file data from chat responses.

Wire It Up

chat = RubyLLM
         .chat(model: "gemini-2.5-flash-image")
         .with_temperature(1.0) # optional, but you like creativity, right?
         .with_params(generationConfig: { responseModalities: ["image"] }) # also optional, if you prefer the model to return only images

response = chat.ask "your prompt", with: ["all.png", "the.jpg", "attachments.png", "you.png", "want.jpg"]

image_io = response.content[:attachments].first.source

That StringIO holds the generated image. Stream it to S3, attach it to Active Storage, or keep it in memory for a downstream processor.

Want a file?

response.content[:attachments].first.save "nano-banana.png"

That’s it. Chat endpoint, one call. Ship the image feature and go enjoy the rest of your day.

1.4.0: The Structured Output Release (Wednesday)

Getting LLMs to return data in the format you need has always been painful.

We all had code like this:

# The old struggle
response = chat.ask("Return user data as JSON. ONLY JSON. NO MARKDOWN.")
begin
  data = JSON.parse(response.content.gsub(/```json\n?/, '').gsub(/```\n?/, ''))
rescue JSON::ParserError
  # Hope and pray
end

Now with structured output:

# Define your schema with the RubyLLM::Schema DSL
class PersonSchema < RubyLLM::Schema
  string :name
  integer :age
  array :skills, of: :string
end

# Get perfectly structured JSON every time
chat = RubyLLM.chat.with_schema(PersonSchema)
response = chat.ask("Generate a Ruby developer profile")

# => {"name" => "Yukihiro", "age" => 59, "skills" => ["Ruby", "C", "Language Design"]}

No more regex. No more parsing. Just data structures that work.

Oh, and Daniel Friis released RubyLLM::Schema just for the occasion, but you can use any gem you want with RubyLLM, or even write your own JSON schema from scratch.

Rails Generators: From Zero to Chat

We didn’t have Rails generators before. Now we do:

rails generate ruby_llm:install

This creates everything you need:

• Migrations
• Models with acts_as_chat, acts_as_message, and acts_as_tool_call
• A clean initializer

Your Chat model works like any Rails model:

chat = Chat.create!(model: "gpt-4.1-nano")
response = chat.ask("Explain Ruby blocks")
# Messages are automatically persisted with proper associations

From rails new to working chat in under 5 minutes.

Tool Call Transparency

New callback to see what your AI is doing:

chat.on_tool_call do |tool_call|
  puts "🔧 AI is calling: #{tool_call.name}"
  puts "   Arguments: #{tool_call.arguments}"

  Rails.logger.info "[AI Tool] #{tool_call.name}: #{tool_call.arguments}"
end

chat.ask("What's the weather in Tokyo?").with_tools([weather_tool])
# => 🔧 AI is calling: get_weather
#    Arguments: {"location": "Tokyo"}

Essential for debugging and auditing AI behavior.

Direct Parameter Provider Access

Need that one weird parameter? Use with_params:

# OpenAI's JSON mode
chat.with_params(response_format: { type: "json_object" })
     .ask("List Ruby features as JSON")

No waiting for us to wrap every provider option.

Critical Bug Fixes and Other Improvements in 1.4.0

• Anthropic multiple tool calls: Was only processing the first tool call, silently ignoring the rest
• Streaming errors: Now handled properly in both Faraday V1 and V2
• Test fixtures: Removed 60MB of unnecessary test data
• Message ordering: Fixed race conditions in streaming responses
• JRuby support: Now officially tested and supported
• Direct access to raw responses: Get the raw responses from Faraday for debugging
• GPUStack support: A production-ready alternative to Ollama

Full release notes for 1.4.0 available on GitHub.

1.5.0: Two New Providers (Friday)

Mistral AI

63 models from France, from tiny to massive:

RubyLLM.configure do |config|
  config.mistral_api_key = ENV['MISTRAL_API_KEY']
end

# Efficient small model
chat = RubyLLM.chat(model: 'ministral-3b-latest')

# Their flagship model
chat = RubyLLM.chat(model: 'mistral-large-latest')

# Vision with Pixtral
vision = RubyLLM.chat(model: 'pixtral-12b-latest')
vision.ask("What's in this image?", with: "path/to/image.jpg")

Perplexity

Real-time web search meets LLMs:

RubyLLM.configure do |config|
  config.perplexity_api_key = ENV['PERPLEXITY_API_KEY']
end

# Get current information with web search
chat = RubyLLM.chat(model: 'sonar-pro')
response = chat.ask("What are the latest Ruby 3.4 features?")
# Searches the web and returns current information

Full release notes for 1.5.0 available on GitHub.

Rails Generator Fixes

• Fixed migration order (Chats → Messages → Tool Calls)
• Fixed PostgreSQL detection that was broken by namespace collision
• PostgreSQL users now get jsonb columns instead of json

1.5.1: Quick Fixes (Also Friday)

Found issues Friday afternoon. Fixed them. Shipped them. That’s it.

Why make users wait through the weekend with broken code?

• Fixed Mistral model capabilities (was a Hash, should be Array)
• Fixed Google Imagen output modality
• Updated to JRuby 10.0.1.0
• Added JSON schema validation for model registry

Full release notes for 1.5.1 available on GitHub.

The Philosophy: Ship When Ready

Three days. Three releases. Each one made someone’s code work better.

We could have bundled everything into one release next week. But every moment we wait is a moment someone’s dealing with a bug we already fixed.

The structured output in 1.4.0? People needed that since before RubyLLM existed. The PostgreSQL fix in 1.5.0? Someone’s migrations were failing Thursday. The Mistral fix? Breaking someone’s code Friday morning.

When code is ready, you ship.

What You Can Build Now

With structured output and multiple providers, you can build real features:

# Extract structured data from any text
class InvoiceSchema < RubyLLM::Schema
  string :invoice_number
  date :date
  float :total
  array :line_items do
    object do
      string :description
      float :amount
    end
  end
end

# Use Mistral for cost-effective extraction
extractor = RubyLLM.chat(model: 'ministral-8b-latest')
                    .with_schema(InvoiceSchema)

invoice_data = extractor.ask("Extract invoice details from: #{pdf_text}")
# Reliable data extraction at a fraction of GPT-4's cost

# Use Perplexity for current information
researcher = RubyLLM.chat(model: 'sonar-deep-research')
market_data = researcher.ask("Current Ruby job market trends in 2025")
# Real-time data, not training cutoff guesses

Use It

gem 'ruby_llm', '~> 1.5'

Full backward compatibility. Your 1.0 code still runs. These releases just made everything better.