Schematist is a general purpose JSON Schema DSL that emits Draft 2020-12 schemas. Describe an API payload, a config file, a contract between two services, or the structured output you want back from a model. Trapping that inside another gem’s namespace was a disservice to anyone looking for a great JSON Schema DSL, so it got its own name.

gem 'schematist'

It Emits Actual JSON Schema

This is the breaking change.

to_json_schema used to return this:

{ name: "PersonSchema", description: nil, schema: { type: "object", ... }, strict: true }

That’s not a JSON Schema. It’s OpenAI’s response_format envelope, with the actual schema buried one level down under a symbol key. Every consumer that wasn’t OpenAI had to dig it out, and anyone who wanted to hand the result to a validator had to know which part was real.

Now you get the document:

class Invoice < Schematist::Schema
  title "Invoice"
  description "A billing document"

  string :id, pattern: "^inv_", title: "Invoice ID"
  number :total, greater_than: 0, description: "Amount due"
  string :currency, const: "EUR"
  string :status, enum: %w[draft sent paid], default: "draft"
end

Invoice.new.to_json_schema
# => {
#   "$schema" => "https://json-schema.org/draft/2020-12/schema",
#   "title" => "Invoice",
#   "description" => "A billing document",
#   "type" => "object",
#   "properties" => {
#     "id" => { "type" => "string", "pattern" => "^inv_", "title" => "Invoice ID" },
#     "total" => { "type" => "number", "description" => "Amount due", "exclusiveMinimum" => 0 },
#     ...
#   },
#   "required" => ["id", "total", "currency", "status"],
#   "additionalProperties" => false
# }

String keys, $schema declared, no provider keys. Use it with JSON.generate unchanged and any Draft 2020-12 validator will take it.

strict went with it. It’s an OpenAI request flag, not a JSON Schema keyword, and a schema library has no business knowing OpenAI exists. Set it where you build the request.

Full Draft 2020-12 Coverage

The old gem covered the basics: types, enum, required, string and numeric bounds, nested objects and arrays, $defs and $ref, if/then/else. Schematist covers the whole vocabulary.

Composition. allOf, oneOf, and not join anyOf:

one_of :method do
  object { string :card_number }
  object { string :iban }
end

all_of :account, unevaluated_properties: false do
  object { string :id }
  object { string :status }
end

none_of :state do
  string enum: ["deleted"]
end

unevaluated_properties is the one that makes allOf usable in practice. additionalProperties can’t see across composition branches; unevaluatedProperties can.

Object keys. Constrain how many properties an object has, what its keys look like, and what the values behind a key pattern must be:

object :metadata, min_properties: 1, max_properties: 10 do
  keys { string pattern: "^[a-z_]+$" }     # propertyNames
  keys_matching(/^x-/) { string }          # patternProperties
end

Arrays. uniqueItems, fixed-length tuples via prefixItems, and contains with its bounds:

array :tags, of: :string, unique: true

tuple :period do
  string format: "date"
  string format: "date"
end

array :scores do
  integer
  contains(min: 1) { integer minimum: 10 }   # at least one score of 10 or more
end

Annotations. title, description, default, examples, deprecated, read_only, write_only. Short ones read well as keyword arguments; longer ones read better in the block, where they annotate the enclosing schema:

object :account do
  title "Account"
  description "Billing account metadata used for invoices."
  examples [{ id: "acct_123", status: "active" }]

  string :id
  string :status
end

Encoded content. For strings that carry something else inside them:

string :payload, content_encoding: "base64", content_media_type: "application/json" do
  content_schema do
    object { string :name }
  end
end

Core keywords. $id, $anchor, $comment, $dynamicAnchor, $dynamicRef, $vocabulary, at the root or on any subschema. They’re passed straight through. Resolving a dynamic reference is the validator’s job, not ours.

Also new: const on every primitive, and greater_than / less_than for exclusiveMinimum / exclusiveMaximum. I picked the Ruby-sounding names over the JSON Schema ones on purpose. You’re writing Ruby.

Values That Aren’t Known Until Render Time

You define a schema class once, at boot. The allowed values often aren’t known until a request comes in.

Any value can be a proc now, resolved when the document is rendered:

class RoleSchema < Schematist::Schema
  string :role, enum: -> { @account.roles.pluck(:name) }

  def initialize(account:)
    super()
    @account = account
  end
end

RoleSchema.new(account: account).to_json_schema

A zero-argument proc is evaluated in the instance’s context, so it can read instance variables. A proc that takes one argument gets the schema instance instead. One class, a different document per instance.

Escape Hatches

Covering the spec isn’t the same as guessing everything you’ll want to put in a document, so there are two ways out.

JSON Schema allows true and false in place of a schema object. true accepts anything, false accepts nothing:

any_of :value do
  any_schema
  string
end

And raw drops a fragment in as-is, for a vendor extension or anything else the DSL has no opinion about:

raw :vendor, { "type" => "object", "x-vendor" => true }

A Schema Doesn’t Have To Be an Object

Most schemas describe an object, so that’s the default. But JSON Schema doesn’t care. A schema can be an array, a union, a string, or a pointer somewhere else, and the root of a document is just a schema like any other.

So: a type with a name declares a property. Without a name, it declares what the schema itself is.

class Tags < Schematist::Schema
  array of: :string, unique: true       # the whole schema is an array
end

class Id < Schematist::Schema
  one_of do                             # the whole schema is a choice
    string
    integer
  end
end

class Person < Schematist::Schema
  raw({ "$ref" => "https://example.com/person.json" })
end

It works inside define too, so a reusable definition can be a string with a pattern or a shared enum, not just an object:

define :status do
  string enum: %w[draft sent paid]
end

A conditional branch is a schema too, so it can ask for a nested object instead of a flat list of fields:

given kind: "business" do
  requires :vat_id

  object :tax_details do
    string :vat_number
  end
end

No Runtime Dependencies

Schematist depends on nothing.

Migrating

gem 'schematist'                         # was: gem 'ruby_llm-schema'

class Person < Schematist::Schema        # was: RubyLLM::Schema
end

Errors moved up a level: Schematist::ValidationError, not RubyLLM::Schema::ValidationError. Schematist::Helpers replaces RubyLLM::Helpers.

If you were reaching into [:schema] to get at the document, stop. to_json_schema returns it directly now, with string keys. If you need the provider wrapper, build it where you send the request:

{ name: "Invoice", schema: Invoice.new.to_json_schema, strict: true }

There’s a final ruby_llm-schema 1.0.0 that depends on Schematist and aliases the old constants, so RubyLLM::Schema keeps resolving while you move. It warns on load and it’s the last release of that name.

RubyLLM 2.0 will depend on Schematist, so structured output will get a lot more powerful.

Use It

bundle add schematist

Schematist was always a JSON Schema DSL. Now it has the name to match.

In that time, the state, two courts, a notary, a law firm, a tax firm, and software vendors have all found a way to bill me. Every single one of them, on time.

I have spent more than 9,600 euros to start a company: a little over 7,600 in fees and bills, plus 2,000 in share capital frozen in an account I am not allowed to touch. And after five months, here is what I have to show for it:

I have not been able to send a single invoice of my own.

Not one.

The work is happening. The clients are real. The one thing the state exists to let me do, bill them cleanly, is the one thing I still can’t.

The timeline

Everyone in this story could invoice me. I am the only one who can’t invoice anyone.

“But you can invoice your German clients”

The clients abroad need a VAT ID for reverse charge, and that is exactly the one I am still waiting for. My German clients I could bill today. But a domestic invoice now would have to be reissued the moment the VAT ID arrives. Bill now, bill again later, for no reason. So those wait too.

This should have been a web form

Fill it in, pay a fee, get your company and your VAT ID in a week. Estonia does it. The UK registers a company in a day, online, for the price of a dinner. There is no law of nature that says incorporation has to take five months and arrive by post.

Germany has built a process that chains one dependency to the next, puts a fee on each, and lets a founder run up legal bills, notary bills, court fees, tax retainers, and software subscriptions on zero revenue, all before granting the one permission a company exists for: the right to send an invoice.

If you ask the government, the reason is trust: the notary, the capital, the registers, the endless checks, all there to keep bad actors out. This is the same machine that did not catch Wirecard, a two-billion-euro scam. It does, somehow, generate enough friction to scare new founders out of the country.

And no, I could not just leave instead. My first company, Freshflow, is valuable enough that walking out of Germany would trigger a massive six-figure exit tax, on gains I have not even realised, purely for the privilege of leaving. But that is a story for another post.

This is a country taxing ambition through the roof before you’ve earned a cent, then wondering why the ambitious leave.

Bonus round: my company name was “too generic”

Have you heard of Apple? A piece of fruit, and one of the most valuable brands ever built. That name would never have been approved in Germany.

Naming a company is hard. It is the word everyone who touches your work will remember. After months of turning it over, I found one I could stand behind, a name that says what I believe software should be. (That belief will be its own post, soon.) Distinctive, I thought. The kind of name you do not forget.

Plenty.

“No,” said the lawyer. German company names have to be distinctive, and “Plenty” is a plain English word. Berlin would reject it.

“Plenty Group?” Two plain words. “Plenty Labs?” “Labs” is a plain word too. “Plenty.is?” A generic word with a domain on the end is still a generic word, and there was case law to prove it.

The suggestions were worse: stick my surname on the front, Paolino Plenty Labs. Or a prefix, PG Plenty Germany. Or make up a fantasy word.

Is Plenty. Its Plenty. IsPlenty. ItsPlenty. Rejected, all of it.

Fine. They wanted a meaningless word; I gave them one. Plenty Labs, minus the space: PlentyLabs.

Approved.

A name that started out of spite. Weeks of correspondence, resolved by removing a space. A rule that does not reward clarity. It rewards nonsense.

Postscript: why a UG and Co. KG, two companies?

Why does a one-person business need two companies? Because the simple version is worse, and because I am building it into something bigger.

The simplest setup is a sole proprietorship. Thirty euros, no capital, done in an afternoon. It also makes me personally liable for everything. A client sues? They are not suing a company. They are suing me. My savings, my apartment, my name.

So I wanted real limited liability, which means a company. And for one person, the cleanest company turns out not to be one company. It is a KG, a partnership that does the work, with a tiny UG standing in as the partner that carries the liability. Strange, but standard. You probably have seen “GmbH & Co. KG” on German companies a hundred times without wondering why. This is why.

It is taxed the sane way, too. The partnership’s profit is taxed once, as my income, since I am the one who ends up with it. A plain UG would tax the company first, then tax me again when I paid myself.

Why a UG and not the famous GmbH? A GmbH wants 25,000 euros sitting in a bank account before it is allowed to exist. The UG lets you start with almost nothing, on one condition: lock away a quarter of every year’s profit until the reserve reaches 25,000, then convert to a GmbH. The 25,000 does not go away. Germany just takes it in instalments.

Which leaves the only real question. Why 25,000 at all? It is my company and my risk. If I want to start with nothing, that is my call, not a toll the state collects before it will let me try. And the cheap door has a price of its own: to some clients, “UG” reads as “not serious,” and they would rather deal with a GmbH. The structure built to let me in quietly marks me for using it.

I released RubyLLM 1.16 today. It answers these production questions.

The three headline features are about speed, visibility, and control: tools that run concurrently, structured events for everything RubyLLM does, and a configurable base URL for every native provider. None of them change how you write your app. All of them matter the moment real traffic shows up.

Tools That Run Concurrently

When a model returns several tool calls in one response, it’s telling you those calls are independent. Get the weather, look up the stock price, fetch the exchange rate. The model didn’t ask for them in order. It asked for all of them.

RubyLLM has always run them one at a time. For tools that are CPU-bound that’s fine, but most tools aren’t. Most tools are an HTTP call, a database query, another LLM request. They spend their time waiting. Running three waits back to back, when you could have waited for all three at once, it’s time your users can’t get back.

1.16 runs them together. Turn it on for every chat from one place:

RubyLLM.configure do |config|
  config.tool_concurrency = true # :threads, :fibers, true, or false
end

true uses :threads and needs no dependencies. If you’d rather not pay for a thread per tool, :fibers mode uses the async gem and gets my recommendation for I/O bound operations. Check out my previous posts on why I think async is the future of Ruby and what Ruby concurrency actually does.

When one conversation needs different behaviour than the rest, override it per chat:

chat.with_tools(Weather, StockPrice, Currency, concurrency: :fibers)
chat.with_tools(Weather, StockPrice, concurrency: false)

Inside Rails, each concurrent tool call runs wrapped in the Rails executor, so connection pools, CurrentAttributes, and reloading behave the way the rest of your app does. You don’t think about it. It just works.

And concurrency doesn’t make your UI wait for the slowest tool. Each result is added back to the conversation the moment that tool finishes, in completion order, so your streaming callbacks see results land as they happen. RubyLLM still gathers every result before going back to the model, but your users watch progress instead of a spinner.

Instrumentation Without Monkey Patching

You can’t operate what you can’t see. Some libraries popped up to add instrumentation to RubyLLM, but they monkey patch us. That’s unnecessary maintenance burden.

RubyLLM 1.16 emits structured events for the work it does, the same way Rails does. In a Rails app they flow through ActiveSupport::Notifications automatically, and you subscribe the way you’d subscribe to any framework event:

# config/initializers/ruby_llm_instrumentation.rb
ActiveSupport::Notifications.subscribe('chat.ruby_llm') do |_name, _start, _finish, _id, payload|
  Rails.logger.info(
    provider: payload[:provider],
    model: payload[:model],
    input_tokens: payload[:input_tokens],
    output_tokens: payload[:output_tokens]
  )
end

Outside Rails, point config.instrumenter at anything that responds to instrument(name, payload) { ... } and wire it into OpenTelemetry, StatsD, or your own logger. The events cover the whole surface: HTTP requests, chat completions, tool calls, embeddings, and model registry refreshes, each carrying the provider, model, token usage, and the Ruby objects an observability adapter needs.

Those payloads can hold message content, tool arguments, and full provider responses, which is exactly the sensitive data you don’t want sprayed into logs by accident. So log or export those fields only when your policy allows it. The Instrumentation guide has the full payload reference.

A Base URL for Every Native Provider

In production, your AI traffic rarely goes straight to the provider. It goes through a gateway that handles auth, a proxy that enforces rate limits, a private endpoint inside your network. RubyLLM let you point most providers at a custom base URL already. 1.16 fills the last gaps, so now every native provider has one:

RubyLLM.configure do |config|
  config.bedrock_api_base     = ENV['BEDROCK_API_BASE']
  config.mistral_api_base     = ENV['MISTRAL_API_BASE']
  config.perplexity_api_base  = ENV['PERPLEXITY_API_BASE']
  config.vertexai_api_base    = ENV['VERTEXAI_API_BASE']
  config.xai_api_base         = ENV['XAI_API_BASE']
end

Together with the bases already there for OpenAI, Anthropic, Gemini, DeepSeek, OpenRouter, Azure, Ollama, and GPUStack, you can front any provider with your own infrastructure. Each override falls back to the provider’s default when unset, so nothing you already have changes.

While I was in the HTTP layer, I made the Faraday adapter configurable too:

RubyLLM.configure do |config|
  config.faraday_adapter = :async_http # or :typhoeus, :net_http, :httpx, etc.
end

It defaults to Net::HTTP, so nothing changes unless you ask. Reach for it when you want connection pooling, HTTP/2, or whatever adapter your app already standardizes on.

Transcription Words

Transcription now exposes word-level timing when the provider returns it, so you can build word-by-word highlighting on top of OpenAI’s verbose transcriptions:

transcription = RubyLLM.transcribe("interview.mp3", model: "whisper-1")
transcription.words # => [{ word:, start:, end: }, ...]

Getting Ready for 2.0

Deprecation warnings are now yours to control:

RubyLLM.configure do |config|
  config.deprecation_behavior = :warn # :warn (default), :silence, or :raise
end

Set :raise in your test environment and a deprecated path fails the build the moment something hits it. That’s the cheapest possible way to be ready before those paths disappear in 2.0, instead of finding out on upgrade day.

Fixes and the Model Registry

A release this size carries a long tail of fixes. The ones worth calling out: Anthropic’s “prompt is too long” now raises ContextLengthExceededError so you can rescue it like any other context-length error, streaming parallel tool calls accumulate correctly, Bedrock reasoning streams properly, and Gemini function calls and inline images follow the spec. Active Storage handling in Rails got more careful about pending uploads, load order, and text attachments. And when configuration or a model lookup goes wrong, the error now tells you what happened and how to fix it.

The model registry is refreshed with the latest models, capabilities, and pricing. One fix there is worth a sentence: models.dev started shipping partial release dates like 2025-09 and 2025, RubyLLM was turning those into invalid timestamps, and model loading broke. 1.16 normalizes them to real dates so the registry keeps loading.

The full release notes have the complete list.

Use It

gem 'ruby_llm', '~> 1.16'

bundle update ruby_llm

It’s backwards compatible. Concurrency is opt-in, instrumentation stays inert until you subscribe, and every new *_api_base falls back to the provider default. Nothing you’ve built changes until you decide to reach for it. The boring infrastructure is just there now, waiting for the day your app stops being a demo and starts being production.

I understand where that comes from. There is a lot of slop, maintainers are tired of reviewing code from people who do not understand it, and people are tired of predictable cadence.

We’re also heading toward a version of the future where all code will be generated. The models are good enough that for a lot of work, especially the boring repetitive kind, typing everything by hand makes very little sense. You can describe what you want, steer the model, ask for changes, review the output, and get to a working implementation much faster than before.

That caused some people to jump from “models can generate code” to “engineering is dead”.

That is wrong.

Code Generation Is Not Engineering

Engineering is not the act of producing text that happens to run or compile.

Engineering is deciding what should exist. Understanding the constraints. Knowing what can go wrong. Making trade-offs. Reviewing the result. Being responsible for what happens after you ship it.

The model can write the code. Most of it. Maybe all of it. But the model is not accountable.

You are.

If a generated library has a security issue, people will not open an issue against the model. They will open it against you. If a generated feature behaves badly in production, your reputation will suffer. If the code is impossible to maintain six months later, the model is not at fault.

You are.

This is why your engineering skills matter more than ever. Since you are not spending most of your time typing, you can focus on what really matters.

So How Do You Tell?

The discussion around AI-generated code is confused because people focus too much on the origin.

Lots of good code will be touched by LLMs. So will code from your favorite programmers. So will lots of bad code.

The involvement of AI tells you very little by itself.

The real distinction is whether the result is owned or not. It is the care, attention, review, testing, product design, and engineering the author put into it.

You signal it by producing high-quality output and being accountable for it. By showing up. By fixing bugs. By knowing your own code inside and out. By making it clear that there is a person behind the work who understands the result and accepts responsibility for it.

That takes care, taste, engineering skill, and genuine human effort.

This goes both ways. The same skills are needed by people evaluating code and products. It is not enough to ask whether AI was involved. You have to look at the result, the behavior, the tests, the edge cases, the maintenance story, and the author’s ability to own the thing.

Engineering is not dead, because accountability isn’t.

It was fabricated.

The first code sample used RubyLLM.client, which does not exist. It called client.chat(messages: ...), which does not exist. Then it invented RubyLLM::StreamInterrupted, RubyLLM::APIError, and a stream: proc API that RubyLLM has never had.

The problem was not merely wrong information. Wrong information can be corrected. This was sold as experience with RubyLLM in production, which is a much more valuable claim.

AI slop is not just filling the web with predictable cadence. It is fabricating experience. It is letting people skip the work, skip the scar tissue, and still write in the voice of someone who has been there.

In open source, that turns into a tax. Maintainers build the thing, write the docs, publish the source, keep the examples working, answer the issues, and then have to police hallucinated articles about their own projects before users start debugging ghosts.

The Four Magic Words in Tech

Production. Scale. Security. Reliability.

In the tech world, attach one of these words to a claim and it immediately becomes true. “This does not scale” can kill a project before anyone measures it. “This is not production ready” can sabotage it without a single deploy.

So when an article says “what broke in production”, it is not just offering advice. It is claiming experience, and experience cannot be hallucinated.

The first version opened by saying the author had spent three weeks on the wrong side of the problem before getting something stable in production. That is a powerful claim. It tells the reader to relax and inherit the author’s scars.

There were no scars. The author had not even run the first example.

This is why fake experience is so dangerous. Bad code fails fast. Fake experience lingers. It gets quoted. It gets summarized. It gets used in meetings by people who do not know enough yet to see the hollow center.

The recipe is familiar. Streaming failures. Token budgets. Provider fallback. Turbo Streams. Redis circuit breakers. nginx buffering. Load testing. They sit near “LLM production” in the LLM training data. Arrange them with enough confidence and the result smells real.

Production experience is not a smell. It is a thing that happened, and none of these things happened.

What Actually Happened

Here is the short version.

Most articles about RubyLLM are good. Since it became popular, I have seen a few confident guides from people who clearly had not run the code. Usually they disappear into LinkedIn or search results. This one made the pattern impossible to ignore.

I called it out:

Author of RubyLLM here.

The very first example does not work.

The article is not merely wrong in a few places. It is fabricated.

…

The author replied:

You were right.

The code in the original article was not verified against the actual gem. RubyLLM.client, RubyLLM::StreamInterrupted, RubyLLM::APIError, stream: proc – none of it exists. You caught every fabrication accurately.

I’ve replaced the article entirely. The new version has been verified against your documentation and source. The fake “production experience” framing is gone. It’s now an honest documentation-based guide with a correction notice at the top explaining what happened.

“I’ve replaced the article entirely.”

It was a long article. The completely rewritten replacement appeared in a few minutes. The fake method names were replaced with real ones, but the posture stayed the same: “RubyLLM in production”, “what tutorials skip”, “streaming failures”, “provider fallback”, “token budgets.”

The method names got real. The experience didn’t.

The new version claimed Puma restarts produce neat RubyLLM streaming errors. They do not. If the worker dies, the Ruby process running the call is gone. It suggested deleting old persisted chat messages as context management. That is destroying conversation history. It described fallback by throwing away the chat and asking another provider the last prompt as a fresh question. That is not conversation fallback. It confused HTTP/SSE buffering with Turbo Streams over ActionCable.

Not battle scars. Guesses presented as authority.

I called the second version what it was: phony. The author then hid responses while keeping the article up.

I reported the article to Medium and contacted the publication that promoted it with the fabricated APIs, the author’s admission, and the hidden corrections. To their credit, the editor replied quickly, apologized, and removed it from the publication. But only the author can take down the original Medium article, so the piece remained available without the maintainer corrections visible next to it.

Do Not Counterfeit Experience

Please do write about your favourite software. Critique it too. Tell us maintainers where the API is wrong, the docs are bad, the abstraction leaks. Preferably in an issue so we can actually see it. That feedback is gold.

But do not counterfeit experience. If you’re using The Four Magic Words in Tech, the bar is even higher.

And if you run a technical publication, please at least check the first example.

It ships image editing, cost tracking, cleaner token accounting, inferred tool parameters, additive callbacks, and Rails fixes.

The theme is simple: stop making me write glue code. If the computer can infer it, RubyLLM should infer it. If a provider reports usage, RubyLLM should turn it into cost. If Rails already has a blob, RubyLLM should not download it and upload it again.

Image Editing

RubyLLM.paint could already generate images:

image = RubyLLM.paint("A watercolor robot holding a Ruby gem")

Now with: turns it into an image edit:

image = RubyLLM.paint(
  "Turn the logo green and keep the background transparent",
  model: "gpt-image-1",
  with: "logo.png"
)

Same method, same attachment shape.

The source can be a path, a URL, an IO-like object, or an Active Storage attachment. Multiple source images work too:

image = RubyLLM.paint(
  "Combine these references into a postcard illustration",
  model: "gpt-image-1",
  with: ["person.png", "style-reference.png"]
)

And if you need to constrain the edit, pass a mask:

image = RubyLLM.paint(
  "Replace only the background with a sunset sky",
  model: "gpt-image-1",
  with: "portrait.png",
  mask: "portrait-mask.png"
)

That’s it. paint paints. Sometimes from scratch, sometimes from an existing image.

Cost Tracking

RubyLLM has tracked tokens since 1.0. But “this used 18,432 tokens” is only half the answer. The next question is always: how much did that cost?

Calculating that was never hard. Take the input tokens, output tokens, cached tokens, maybe reasoning tokens. The pricing is already in RubyLLM’s model registry. Multiply by the per-million rate.

But why should every app have to write that code?

RubyLLM already has the usage. RubyLLM already knows the model. RubyLLM already ships the model registry. So now it does the boring math for you.

Now you can ask:

response = chat.ask("Summarize Ruby's object model.")

response.cost.total
chat.cost.total
agent.cost.total

Same for images:

image = RubyLLM.paint("A small watercolor robot", model: "gpt-image-1")

image.tokens.input
image.tokens.output

image.cost.input
image.cost.output
image.cost.total

If RubyLLM does not have pricing for part of the usage, the cost is nil. Better no answer than a fake one.

A chat with ten messages can tell you the total. An agent can tell you the total. A generated image can tell you the total. No more handrolled sums.

Token Counts That Mean What They Say

Prompt caching made token counts messy.

Some providers include cache reads in prompt tokens. Some report cache creation separately. Some don’t. If you multiply the wrong number by the wrong price, your cost tracking is wrong before it starts.

So 1.15 separates the different kinds of tokens before exposing them:

response.tokens.input       # standard input tokens
response.tokens.output      # billable output tokens
response.tokens.cache_read  # prompt cache reads
response.tokens.cache_write # prompt cache writes

tokens.input now means normal input tokens. Cache reads and cache writes are separate. tokens.output always mean billable output tokens.

The old top-level helpers still work. New code should use response.tokens.*.

No new Rails migration is required if you already ran the 1.9 token migration. If you display token counts directly, read the 1.15 upgrade notes.

Less Tool Boilerplate

Tools in RubyLLM are Ruby classes. But for very simple tools, RubyLLM still made you repeat yourself:

class Weather < RubyLLM::Tool
  description "Gets current weather for a location"
  param :latitude  # why?
  param :longitude # DRY!

  def execute(latitude:, longitude:)
    # ...
  end
end

That is silly. The method signature already says there is a latitude and a longitude.

Now this works:

class Weather < RubyLLM::Tool
  desc "Gets current weather for a location"

  def execute(latitude:, longitude:, units: "metric")
    # ...
  end
end

Required keywords become required string parameters. Optional keywords become optional string parameters.

Ruby method signatures don’t tell us JSON Schema types or descriptions, so if those matter, keep using param:

param :units, type: :string, desc: "metric or imperial", required: false

And when you need nested objects, arrays, enums, or full schema control, use params. Nothing changed there.

Also:

• desc is now an alias for description
• param accepts description: as an alias for desc:
• the tool generator now emits desc
• we retain full backwards compatibility!

Callbacks That Stack

The old on_* callbacks were replace-style callbacks. Register another one and you replaced the previous one.

That caused an obvious problem: Rails persistence wants callbacks, and your app also wants callbacks. Logging wants callbacks. Analytics wants callbacks. Replacing the previous callback is the wrong default.

So 1.15 adds additive callbacks:

chat.before_message { ... }
chat.after_message { |message| ... }
chat.before_tool_call { |tool_call| ... }
chat.after_tool_result { |result| ... }

Register five callbacks, all five run.

Rails persistence uses these internally now. Your app can layer its own callbacks on top without breaking persistence.

The old on_* callbacks are deprecated. They’ll go away in RubyLLM 2.0.

Rails Fixes

Rails got a lot of boring, important fixes:

• Action Text-backed message content is converted to plain text before being sent to the model.
• ActiveRecord support no longer sits in the core gem eager-load path, fixing standalone require "ruby_llm" with Zeitwerk eager loading.
• The acts_as API follows Rails association inference more closely.
• Existing Active Storage blobs and attachments passed through with: are reused instead of downloaded and re-uploaded.

Providers and Models

Empty tool results are now handled consistently across Anthropic, Bedrock, and Gemini. When a tool returns nothing, RubyLLM sends a small placeholder instead of provider-invalid empty content.

Streaming and non-streaming token usage is normalized across OpenAI, OpenRouter, Bedrock, and Gemini before cost calculation.

The model registry has been refreshed too: cache read/write pricing, reasoning output pricing, GPT Image pricing, and new aliases including Claude Opus 4.7, DeepSeek V4, Gemini Embedding 2, Gemma 4, and GPT-5.5.

Use It

gem 'ruby_llm', '~> 1.15'

Then:

bundle update ruby_llm

Full release notes on GitHub.

I run Chat with Work on Kamal, and I needed backups. There are already Kamal accessories for database backups. None of them also back up Active Storage. None use restic, so encryption, deduplication, and repository checks are on you. None ship a CLI with restores and drills. None produce evidence you can hand a security reviewer.

So I built one.

A gem and a Docker image

kamal-backup is two pieces: a Ruby gem you add to your Rails app, and a Docker image you boot as a Kamal accessory. They point at a restic repository you bring yourself.

The gem is your CLI. Local commands run directly on your machine using restic. Production-side commands shell out through Kamal into the accessory. The same kamal-backup binary covers setup (init, validate), on-demand operations (backup, list, check), data movement (restore local, restore production), verification (drill local, drill production), and audit (evidence).

The Docker image (ghcr.io/crmne/kamal-backup) ships with restic, pg_dump, mariadb-dump/mysqldump, and sqlite3 baked in. The default container command is kamal-backup schedule, a loop that fires every backup_schedule_seconds and writes one database snapshot and one Active Storage file snapshot per run.

The restic repository is where the encrypted snapshots end up: S3-compatible object storage, a restic REST server, or a filesystem path. kamal-backup points at it. It doesn’t run it for you.

Why restic

I didn’t want to invent a backup format, and I didn’t want to bolt encryption and deduplication onto shell scripts. Restic does what I needed:

• encrypted repositories by default;
• a tag system, so the database dump and the Active Storage tree from the same run share a run:<timestamp> and pair up at restore time;
• deduplication across runs, so a year of daily backups doesn’t grow linearly;
• restic forget --prune for retention;
• restic check for repository health;
• S3-compatible storage, a restic REST server, or a local filesystem path, so you host the repository wherever fits.

It’s a single binary that drops cleanly into a Docker image, alongside the database client tools. Nothing extra to install on the Rails host. kamal-backup is the Rails- and Kamal-shaped layer on top, and restic does the cryptography, the storage, and the integrity checks.

Setting it up

Add the gem in development:

# Gemfile
group :development do
  gem "kamal-backup"
end

Run init. It creates config/kamal-backup.yml and prints an accessory block you paste into your Kamal deploy config:

bundle install
bundle exec kamal-backup init

config/kamal-backup.yml holds the backup settings:

accessory: backup
app_name: chatwithwork
database_adapter: postgres
database_url: postgres://chatwithwork@chatwithwork-db:5432/chatwithwork_production
backup_paths:
  - /data/storage
restic_repository: s3:https://s3.example.com/chatwithwork-backups
restic_init_if_missing: true
backup_schedule_seconds: 86400

Kamal mounts that file read-only into the accessory, so the accessory block in config/deploy.yml stays small. Only secrets live in env:

accessories:
  backup:
    image: ghcr.io/crmne/kamal-backup:latest
    host: chatwithwork.com
    files:
      - config/kamal-backup.yml:/app/config/kamal-backup.yml:ro
    env:
      secret:
        - PGPASSWORD
        - RESTIC_PASSWORD
        - AWS_ACCESS_KEY_ID
        - AWS_SECRET_ACCESS_KEY
    volumes:
      - "chatwithwork_storage:/data/storage:ro"
      - "chatwithwork_backup_state:/var/lib/kamal-backup"

Validate, boot, and watch the logs:

bundle exec kamal-backup validate
bin/kamal accessory boot backup
bin/kamal accessory logs backup

validate catches missing required settings before the accessory has to be running. Once it’s up, the container loops on kamal-backup schedule.

Then run the first backup and print evidence:

bundle exec kamal-backup backup
bundle exec kamal-backup list
bundle exec kamal-backup evidence

No cron glue. No separate backup host. No “remember to install restic on production.” The accessory image already has it.

Rails data, not just a database dump

A Rails app has two things worth backing up: the database, and file-backed Active Storage. kamal-backup handles both.

Postgres uses pg_dump. MySQL and MariaDB use mariadb-dump or mysqldump. SQLite uses sqlite3 .backup. File-backed Active Storage uses restic backup from mounted volumes.

Each run writes one database snapshot and one file snapshot, both tagged with app:<name>, type:database or type:files, and the same run:<timestamp>. You pair them at restore time using that timestamp.

If your app stores Active Storage blobs directly in S3, there’s no mounted path for backup_paths to capture. kamal-backup still covers the database. The S3 side is on your bucket lifecycle and replication settings.

Restores are part of the product

The backup script is the easy part. The restore path is where most setups fail.

So kamal-backup ships with restore commands:

bundle exec kamal-backup restore local
bundle exec kamal-backup restore production

restore local pulls a production backup down to your laptop. Useful when you want to inspect real data, reproduce a production bug, or prove the backup actually comes back.

restore production prompts before it overwrites anything.

Restore drills

The command I care about most is drill.

bundle exec kamal-backup drill local \
  --check "bin/rails runner 'puts User.count'"

A drill means: restore, check, record the result.

Two modes:

• drill local restores onto your machine and runs an optional check.
• drill production restores into scratch production-side targets, never the live database.

That second one matters. For Postgres and MySQL, you give it a scratch database. For SQLite, a scratch file path. For Active Storage, a scratch restore directory. The drill uses production infrastructure, without pointing at live production.

That’s the difference between “the backup ran” and “we restored the latest production snapshot into a scratch target on April 30, ran this check, and it passed.”

Evidence for reviews

I went through a security review for Chat with Work this year. The questions were fair:

• What’s being backed up?
• Where does it go?
• Is it encrypted?
• When did the last backup run?
• When did the last repository check run?
• When was the last restore drill?
• Can you prove all of that without leaking secrets?

kamal-backup evidence prints redacted JSON: current backup settings, latest snapshots, latest restic check, latest restore drill, retention settings, tool versions.

bundle exec kamal-backup evidence

Secrets are redacted. The output is meant to land in an internal ops record or a CASA packet. Not a screenshot of a green cron job. Actual evidence.

Try it

# Gemfile
gem "kamal-backup"

Docs at kamal-backup.dev, source on GitHub.

This post answers all of it. From the ground up.

The four primitives

Ruby gives you four concurrency primitives: processes, threads, fibers, and Ractors. They nest. Every process has an implicit “main Ractor” where your code runs by default, so you never have to think about Ractors unless you explicitly create one. Without Ractors, the hierarchy is simply process – threads – fibers. With Ractors, it becomes:

Think of your computer as an office building.

Processes are fully isolated: separate offices, each with its own locked door, furniture, and files. Each process has its own memory, its own Ruby VM, and its own GVL. When you run Puma with 3 workers, you get 3 processes. They can’t corrupt each other’s state because they don’t share memory. The OS schedules them independently. The cost: each one loads your entire application into memory.

Ractors sit between processes and threads: offices that share a mailroom but not their filing cabinets. Each Ractor has its own GVL, so threads in different Ractors can execute Ruby code truly in parallel, but they can only pass notes to each other – no shared mutable objects. You communicate via message passing, copying or moving data between them. Every Ruby process has a “main Ractor” where all your code runs by default. Creating additional Ractors is opt-in.

Threads live inside a process and share its memory: workers sharing the same office, accessing the same filing cabinets, coordinating to avoid collisions. In CRuby, they are native threads, with the GVL deciding which one can execute Ruby code at a time. You don’t control when Ruby switches between them. The GVL releases during I/O, so two threads can wait on two different network calls simultaneously, but they can’t crunch numbers at the same time.

Fibers live inside a thread and are cooperatively scheduled: multiple tasks juggled by one worker at their desk. When they’re waiting for something – a phone call, a fax, a response – they set it aside and pick up the next task. A fiber runs until it explicitly yields. When it hits I/O – a network call, a database query, reading a file – it yields to the reactor, and another fiber picks up. No OS thread context switch for the fiber itself, no preemption. One thread can run thousands of fibers.

Here’s what that means for cost:

Creation and switching benchmarks are from Samuel Williams’ fiber-vs-thread performance comparison. Fibers create 20x faster and switch 10x faster than threads. The memory row is about virtual address space reserved by the platform/runtime, not resident memory. The benchmark reports actual RSS, where the gap is much smaller than the virtual stack numbers suggest. But the shape is still real: each thread is a kernel object with scheduler state and a stack reservation, while each fiber is scheduled in userspace. Ractors give you parallelism too, but can’t run Rails. Everything is a tradeoff.

How scheduling works

This is where most of the confusion lives. Let me show you what actually happens.

Thread scheduling

CRuby threads are native threads, but the GVL decides which one can run Ruby code. Your code has no say. A thread can be paused mid-calculation, mid-assignment, mid-anything.

CRuby can switch runnable threads on a time slice, but a thread blocked in I/O is parked until the socket is ready. That part matters: threads do not spin uselessly while waiting for tokens. The switch happens when a thread is runnable – including in the middle of response processing, object allocation, assignment, or any other Ruby code.

For two threads doing I/O, this works fine. The overhead is noise. For 200 threads mostly waiting for LLM tokens, the problem is the one-operation-per-thread shape: 200 kernel threads, 200 stack reservations, 200 scheduler entries, and usually 200 copies of whatever per-thread application resources the worker holds.

This is also why a worker limit means different things in Solid Queue’s current thread mode and in the fiber mode from my patch. threads: 25 is both “run 25 jobs at once” and “create 25 kernel threads.” If all 25 jobs are streaming tokens, job 26 waits. fibers: 250 is mostly an admission limit for the reactor: run up to 250 jobs as fibers on the same thread, park the ones waiting on I/O, and resume them when ready. You still need limits because APIs, sockets, memory, and databases have limits. But the cap is no longer tied to one kernel thread per job.

Cooperative scheduling (fibers)

Fibers switch only when they choose to. In practice, the async gem makes this automatic: your code yields at I/O boundaries without you writing anything special.

No OS thread context switch per fiber. No timer-based preemption between fibers. When a fiber yields, the reactor checks which fibers have I/O ready and resumes them. When nothing is ready, the reactor sleeps in the OS until something is. The kernel still does the I/O readiness work; Ruby just avoids one kernel thread per wait.

The GVL: why threads and fibers are more similar than you think

This is the part that makes thread-based Ruby less different from fiber-based Ruby than it first looks.

The GVL means only one thread can execute Ruby code at a time. Threads run in parallel only during I/O, when the GVL is released. So if your workload is I/O-bound – HTTP calls, database queries, LLM streaming – threads give you I/O concurrency, not parallelism.

Fibers give you the same I/O concurrency. One fiber yields at I/O, another picks up. The difference: fibers do it without kernel thread overhead, without the memory cost of a thread stack, and without making job concurrency itself imply one worker thread or one database slot per job.

If threads only help with I/O anyway, why pay their overhead?

There is one case where threads win: CPU-bound work that releases the GVL. Some C extensions (image processing, cryptographic operations) release the GVL while doing heavy computation. Multiple threads can then run those C extensions in parallel. Fibers can’t do that. They share a thread.

For actual Ruby-level CPU parallelism, you need processes or Ractors. Processes are production-ready and Rails-compatible. Ractors are lighter than processes, but still experimental.

What happens when a fiber hits I/O

This is the happy path and the most common question.

# Inside a fiber
response = Net::HTTP.get(URI("https://api.example.com/v1/completions"))

Here’s the full chain:

1. Net::HTTP opens a socket and sends the request
2. The socket isn’t readable yet (the server hasn’t responded)
3. Ruby calls rb_io_wait on the socket
4. The async gem’s Fiber.scheduler intercepts this call
5. The scheduler suspends the current fiber and registers the socket with the event loop
6. The reactor runs other fibers while this one sleeps
7. When the socket becomes readable, the reactor resumes this fiber
8. Net::HTTP reads the response as if nothing happened

Your code doesn’t change. No await, no callbacks, no promises. The same Net::HTTP.get call that works in a thread works in a fiber. The yield is invisible.

Bob Nystrom called this the function color problem in 2015. In languages with async/await, every function is either sync or async. An async function can only be called with await, and await can only live inside another async function. The color spreads upward through your entire call stack.

Python:

# Python: the color spreads, and you need different libraries
async def get_user(id):
    async with aiohttp.ClientSession() as session:  # can't use requests
        response = await session.get(f"/users/{id}")  # must await
        return await response.json()                   # must await

async def handle_request():  # must be async because it calls get_user
    user = await get_user(1)  # must await

You can’t use requests in async Python without blocking the event loop. You need aiohttp, httpx in async mode, or a thread wrapper. You can’t use the blocking psycopg2 API as async I/O; you need asyncpg or Psycopg’s async API. The ecosystem splits: sync libraries and async libraries, doing the same thing differently.

JavaScript:

// JavaScript: same problem, less severe (Node has fewer library splits)
async function getUser(id) {
  const response = await fetch(`/users/${id}`);  // must await
  return await response.json();                   // must await
}

async function handleRequest() {  // must be async
  const user = await getUser(1);  // must await
}

Ruby:

# Ruby: no color
def get_user(id)
  response = Net::HTTP.get(URI("https://api.example.com/users/#{id}"))  # just a normal call
  JSON.parse(response)                            # just a normal call
end

def handle_request
  user = get_user(1)  # just a normal call
end

Same Net::HTTP. Same pg. Same call stack, as long as the library uses scheduler-aware Ruby I/O. The fiber scheduler intercepts I/O at the Ruby runtime level, below your code. Your methods don’t know and don’t care whether they’re running in a thread or a fiber.

What happens when a fiber does CPU-bound work

# Inside a fiber
100_000.times { Digest::SHA256.hexdigest("work") }

This blocks the reactor. No other fiber runs until it finishes. There’s no I/O boundary to yield at, so the fiber holds the thread.

This is not a bug. It’s the current tradeoff of cooperative scheduling. Fibers are designed for I/O-bound work; CPU-bound work belongs on a thread, where CRuby can preempt it.

With my fiber-mode patch for Solid Queue, this is a configuration choice:

workers:
  - queues: [ chat, turbo, notifications ]
    fibers: 50       # I/O-bound: use fibers
  - queues: [ cpu ]
    threads: 2        # CPU-bound: use threads

One backend, two modes, matching the concurrency model to the workload.

What happens when a fiber queries the database

The pg gem has supported Fiber.scheduler since v1.3.0. When a fiber executes a query, the pg gem sends it non-blockingly via PQsendQuery, then calls rb_io_wait on the PostgreSQL socket. The scheduler intercepts this, suspends the fiber, and lets others run while PostgreSQL processes the query.

# Inside a fiber
user = User.find(42)  # yields while waiting for PostgreSQL

The fiber yields. Other fibers run. When PostgreSQL responds, the reactor resumes the fiber. Your code doesn’t know the difference.

Pool size follows database work

A database connection is busy until its query finishes. While PostgreSQL works, Ruby can run something else – another thread, or another fiber on the reactor – but that connection stays checked out.

For an LLM job, most of the wall time is not database time. Read a row, call an API, stream tokens, write a status update. The database touches are short. The long waits are external HTTP. So 100 jobs in flight does not mean 100 jobs hitting PostgreSQL at the same instant.

The reactor never preempts a fiber – it only switches when a fiber yields at an I/O boundary:

Read this as a timeline. Fiber A uses the only connection for its query. While PostgreSQL works, Fiber B waits on HTTP. After Fiber A returns the connection, Fiber B can use it for its update. If both fibers tried to query at the same time, one would wait unless the pool had another connection.

Active Record follows the same checkout rules in both cases. The current Solid Queue difference is a guardrail: thread mode expects threads + 2 connections per process, so you don’t run 50 execution threads against a 5-connection pool. Fiber mode can use a smaller baseline because fibers: 100 means “allow 100 jobs to wait,” not “create 100 execution threads.” In my patch, I/O-heavy workers often start at 3 connections per process (1 execution + 2 worker overhead). If the jobs are DB-heavy, raise it.

What happens when a fiber starts a transaction

A transaction changes the timeline. The connection cannot be returned after each statement, because the transaction state lives on that connection.

When a fiber starts a transaction, it keeps its checked-out connection for the entire duration – from BEGIN to COMMIT or ROLLBACK. The connection is not released mid-transaction. Other fibers that need the database wait for the connection to be returned.

Under fiber isolation (config.active_support.isolation_level = :fiber), Active Support’s execution state is fiber-scoped, so Active Record’s lease is associated with the current fiber instead of the surrounding thread. The connection still gets a real Monitor lock. No other fiber can touch it during a transaction.

Safe. No interleaving. Fiber B just waits.

For the target workload – LLM streaming, HTTP calls – database touches are short reads and status updates. Transactions are brief. The wait is negligible. If your jobs run long transactions, those jobs belong on a thread-based worker.

What happens when you have too many fibers

Fibers aren’t free. Each one uses memory (~4KB), and each one might hold open connections to external services. If you spawn 10,000 fibers that all hit the same API, you’re opening 10,000 connections to that API. The API will not be happy.

Async doesn’t eliminate resource limits; it changes where they show up. With threads, the limit is explicit: 25 threads, 25 concurrent jobs. With fibers, the limit is implicit: you keep going until something else breaks.

The fix is a semaphore. The FiberPool in my Solid Queue patch uses one:

semaphore = Async::Semaphore.new(size)

# Only `size` fibers run concurrently
semaphore.async do
  perform_job
end

When you configure fibers: 100 with the patch, that’s not “unlimited fibers.” It’s a semaphore capping concurrency at 100. You control the ceiling.

“Why not just configure more Solid Queue threads?”

In plain Ruby, more threads can be reasonable. In Solid Queue thread mode, threads: 200 means more than “allow 200 jobs to wait on I/O.”

Kernel threads are the expensive unit. Fibers don’t make I/O complete faster; they let you wait on far more of it at once for a fraction of the cost. Samuel Williams’ benchmarks show fibers allocate 20x faster (~3μs vs ~80μs) and switch 10x faster (~0.1μs vs ~1.3μs) than threads. The OS can manage thousands of threads, but scheduler state, stack reservations, wakeups, and GVL coordination make that a poor default concurrency knob.

Solid Queue currently enforces a database-pool guard. Today it expects threads + 2 database connections per process, so 200 threads across 2 processes won’t boot unless the pool is at least 404. That guard may be conservative for I/O-heavy jobs; there’s an open issue about making it advisory or bypassable. But it is still a guard you hit today.

A blocked job still occupies its worker thread. The OS can park an LLM streaming thread until the socket is ready, but in Solid Queue thread mode it still consumes one of the configured thread workers. If all 25 are streaming tokens, job 26 waits.

Fibers make the Solid Queue limit mean “how many jobs may wait at once” instead of “how many kernel threads should exist.” They still need limits, but the limit is no longer one kernel thread per waiting job.

“Why not Ractors?”

Ractors solve a different problem. Fibers give you I/O concurrency – many things waiting at once. Ractors give you CPU parallelism – many things computing at once.

Here’s what they look like:

# Two Ractors computing fibonacci in parallel
r1 = Ractor.new { fibonacci(38) }
r2 = Ractor.new { fibonacci(38) }

r1.value  # Ruby 4.0+
r2.value  # Both ran in parallel, each with their own GVL

Each Ractor has its own GVL, so they can execute Ruby code truly in parallel across CPU cores. The tradeoff: strict isolation. You can only share immutable (frozen) objects. Everything else gets copied or moved between Ractors via message passing. Access a mutable variable from an outer scope? Ractor::IsolationError.

When Ractors win, they win big. Fibonacci(38) five times: 0.68s with Ractors vs 2.26s sequential. 3.3x speedup. Real parallelism.

But they are not a practical answer for Rails jobs yet:

• Still experimental in Ruby 4.0. Creating a Ractor still emits the experimental API warning.
• Many gems don’t work without changes. Gems that rely on mutable constants, global variables, class variables, or shared process state can hit Ractor::IsolationError.
• No Rails integration. ActiveRecord, ActionCable, the router, the logger – Rails is built on shared mutable state. None of it runs inside a Ractor.
• No Ractor-based job queue exists.
• Still active bug surface. The Ruby bug tracker still has Ractor-related issues, including recent crash reports.

For I/O concurrency, Ractors don’t help at all. Each Ractor still has threads constrained by its own GVL. Fibers within those threads still do the actual I/O multiplexing. Ractors add CPU parallelism, which is not what LLM streaming needs.

For Rails jobs that need CPU parallelism today, processes are still the boring answer. Puma already uses that model for web workers. Ractors may become useful for isolated CPU-heavy Ruby work, but they are not the answer to this Solid Queue I/O problem.

“Isn’t this just what JavaScript does?”

No. I showed the code comparison above. JavaScript’s async/await is a colored concurrency model: the async keyword spreads upward through every caller. Ruby’s fibers are colorless: your existing code works unchanged, and the scheduler handles yields below your code.

There’s a deeper difference too. JavaScript async/await runs on an event loop. Ruby fibers run on top of a multi-threaded runtime. You can have multiple Ruby threads, each running its own reactor with its own fibers, and mix fibers and threads in the same application. Node can run JavaScript in parallel with worker_threads, but that’s a worker/isolate model, not the same thing as putting multiple reactors inside ordinary application threads.

“Isn’t this just what Go does?”

Closer. Goroutines are lightweight, runtime-scheduled, and multiplexed across OS threads. Conceptually similar to Ruby fibers, but Go’s scheduler can also preempt goroutines.

Two differences:

1. Go has true parallelism. Goroutines run across multiple OS threads with no GVL equivalent. CPU-bound goroutines run in parallel. Ruby fibers don’t.

2. Ruby has existing code. If you have a Rails application with hundreds of thousands of lines of Ruby, you can add fiber-based concurrency without rewriting anything. Your models, your controllers, your views, your gems – they all work. With Go, you’re rewriting.

If you’re starting from scratch and need both I/O concurrency and CPU parallelism, Go is a strong choice. If you have a Ruby application and need I/O concurrency, fibers give you that without a rewrite.

“Fibers need Async do blocks. That’s still new syntax.”

Someone on Hacker News called this out: I said “no async/await” but the examples show Async do and .wait.

Here’s the actual change:

# Before
chat = RubyLLM.chat
response = chat.ask("Hello")

# After
Async do
  chat = RubyLLM.chat
  response = chat.ask("Hello")
end

Two lines of wrapping. Your application code inside doesn’t change. Your models don’t change. Your gems don’t change. Nothing gets a new keyword.

In Python, adopting async means rewriting every function signature in the call chain to async def, adding await to every call, and replacing or wrapping blocking libraries. requests becomes aiohttp or async httpx. Blocking database APIs become async database APIs. Your test framework changes. Your middleware changes. It’s a rewrite.

Two lines of wrapping vs. rewriting your stack. That’s not even the same conversation.

When to use what

• I/O-bound work (LLM streaming, HTTP calls, webhooks, email delivery): fibers. Low overhead, high concurrency, database connections sized to database work rather than waiting jobs.
• CPU-bound work (image processing, data crunching, PDF generation): threads. CRuby can preempt them, and C extensions can release the GVL for parallelism.
• CPU parallelism with Rails: processes. Each one gets its own GVL, its own memory, its own everything. Puma already does this.
• CPU parallelism without Rails: Ractors (when they graduate from experimental). Lighter than processes, true parallelism, but strict isolation means most gems don’t work.
• All of them at once: that’s what a well-configured Rails app does. Puma forks processes. Each process runs threads. Fibers run inside those threads for I/O-heavy jobs. They coexist.

# Solid Queue with the fiber-mode patch: all three working together
workers:
  - queues: [ chat, turbo ]
    fibers: 50        # I/O-bound: fibers
    processes: 2       # parallelism: processes
  - queues: [ pdf, images ]
    threads: 4         # CPU-bound: threads
    processes: 1

No single model is universally better. The right answer is matching the model to the workload.

This covers every “what happens when” question I’ve gotten so far. If I missed yours, find me on Twitter; I’ll either update this post or write a follow-up.

Update, July 31, 2026: Solid Queue 1.6.0 has shipped with fiber worker execution. The default Rails job queue can now run many long-running, cooperative I/O-bound jobs like LLM streaming far more efficiently, without making the queue database pool grow with the number of jobs waiting on I/O. The setup below now uses the official release.

Last year I moved the LLM streaming jobs in Chat with Work to Async::Job. It was fast. Genuinely fast. Fiber-based execution with Redis, thousands of concurrent jobs on a single thread. I was so convinced that I wrote a whole post about why async Ruby is the future for AI apps and recommended it to everyone.

Then I started hitting walls.

Async::Job doesn’t persist jobs. They go into Redis and they’re gone. Mission Control shows nothing. Background jobs in Rails are already quieter than the rest of your application – they fail without anyone noticing unless you go looking. Even with Honeybadger catching exceptions, I still want to see the full picture: which jobs are queued, which are running, which failed, what the system looks like right now. Without job persistence, you don’t get that.

Solid Queue is the default in Rails 8. Every new Rails app ships with it. When someone picks up Rails to build an LLM application and their 25-thread worker pool can only handle 25 concurrent streaming conversations, the answer shouldn’t be “swap your entire job backend.” It should be “change one line of config.”

So I opened a PR. On July 31, it shipped in Solid Queue 1.6.0.

Threads vs fibers, quickly

If you already know this, skip ahead to the config.

By default, Solid Queue runs each job on its own thread. Those threads can all query the database concurrently, so conservative pool sizing assumes one connection per execution thread. There is also stack memory and kernel thread overhead. For a job that crunches data for 30 seconds, that’s fine – the thread is busy. For a job that streams an LLM response for 30 seconds but spends 99% of that time waiting for tokens, the thread is just sitting there holding resources.

Fibers sidestep much of this. They are cooperatively scheduled in userspace on a single thread. When a fiber hits scheduler-aware I/O – an Async::HTTP request or waiting for the next token through a compatible client – it steps aside and another fiber picks up. One thread, hundreds of concurrent jobs. No kernel thread overhead per job, and database pools can be sized for simultaneous database work rather than every job waiting on network I/O.

The async gem installs the fiber scheduler. Ruby operations such as Kernel.sleep, scheduler-aware IO, and fiber-aware libraries yield without changing the job itself. This is not magic around every blocking call: a library or C extension that does not cooperate with the scheduler can still block the reactor thread.

For the full deep dive – processes, threads, fibers, the GVL, I/O multiplexing – see Async Ruby is the Future.

The switch

Fiber mode ships in Solid Queue 1.6.0. Upgrade Solid Queue and add async as an application dependency:

# Gemfile
gem "solid_queue", "~> 1.6"
gem "async" # required for fiber workers

Then switch that worker’s execution setting:

# config/queue.yml
production:
  workers:
    - queues: ["*"]
      # threads: 10
      fibers: 100  # <- that's it
      processes: 2

Your jobs don’t change. Your queue doesn’t change. The worker runs them as fibers instead of threads.

threads or fibers. Pick one per worker.

Fiber-scoped Rails isolation is required. Add this to your Rails application configuration:

# config/application.rb
config.active_support.isolation_level = :fiber  # required for fibers

Fibers share a thread, so they need fiber-scoped state instead of the default thread-scoped state. Solid Queue validates this at boot and refuses to start fiber workers if the application still uses thread-scoped isolation.

That isolation setting is global to the Rails application, not local to Solid Queue. Also, fiber worker execution is separate from Solid Queue’s supervisor async mode. The configuration above uses the default fork supervisor, so processes: 2 creates two worker processes and each gets its own fiber reactor. If you start bin/jobs --mode async, the workers share the supervisor process and the processes setting is ignored.

Under the hood

The core of the implementation is FiberPool. It starts its reactor lazily when the first execution is posted, so the pool can be constructed safely before the default supervisor forks. A single thread runs one async reactor, with a semaphore capping concurrency at whatever number you set:

def start_reactor
  create_thread do
    Async do |task|
      semaphore = Async::Semaphore.new(size, parent: task)
      boot_queue << :ready

      wait_for_executions(semaphore)
    end
  rescue Exception => error
    register_fatal_error(error)
    raise
  end
end

When the worker picks up jobs, it hands them to the pool. Each one becomes a fiber:

def wait_for_executions(semaphore)
  while execution = pending_executions.pop
    semaphore.async(execution) do |_execution_task, scheduled_execution|
      perform_execution(scheduled_execution)
    end
  end
end

The worker poller claims only as many jobs as the pool has capacity for and pushes them into a Thread::Queue. Its pop is fiber-scheduler-aware, so the reactor can run execution fibers while it waits for more work. Each compatible I/O wait yields back to the reactor instead of occupying a dedicated execution thread.

CPU-bound work gets nothing from fibers. They don’t parallelize computation. A CPU-heavy job or blocking call stalls every execution fiber in that worker until it returns. In the default fork supervisor mode, the supervisor and other worker processes keep running, but that worker’s reactor does not. Put CPU-bound or blocking jobs on a thread worker instead.

The database connection math

I wrote about this last year:

For 1000 concurrent conversations using traditional job queues like SolidQueue or Sidekiq, you’d need 1000 worker slots. That means 1000 kernel threads across your worker fleet, plus enough database pool capacity for whatever fraction of those jobs can hit the database at the same time. Even when the jobs are 99% idle waiting for streaming tokens, the thread resources are still reserved.

That framing is about worker resources, not a special Active Record rule. The released code’s pool-size check is specifically about the Solid Queue database pool (SolidQueue::Record.connection_pool), not every database your job might use. It estimates connections for polling, heartbeats, and job execution. Size any application database pools touched by the job separately.

Solid Queue 1.6 does not give thread workers the same small pool estimate. It still estimates one execution connection per configured thread, plus one for polling and one for heartbeats. That’s threads + 2.

There is a separate change that makes this easy to confuse: since Solid Queue 1.5, that estimate is advisory. A thread worker can boot with a smaller pool and wait for a connection when the pool is busy, although it can still hit a checkout timeout under sustained contention. But 1.6 did not make the thread and fiber estimates equal.

Here is the relevant version history:

So the connection-sizing distinction is Solid Queue 1.6 fiber workers on Active Record 7.2+ versus every thread worker. The warning-versus-boot-error distinction is older: it changed between Solid Queue 1.4 and 1.5.

For fiber workers, the estimate depends on the Active Record version. On Active Record 7.2+, Solid Queue assumes ordinary query paths release connections between queries, so it estimates one execution connection plus two worker connections regardless of the fiber count: 1 + 2 = 3. On Active Record 7.1, it conservatively estimates one execution connection per fiber, so the estimate is fibers + 2.

The three-connection estimate is a starting point, not a guarantee. Long transactions, ActiveRecord::Base.connection, lease_connection, direct pool checkouts, and long-lived with_connection blocks can pin connections across waits. If your jobs do that or generate simultaneous database work, increase the relevant pool.

Here is the exact warning threshold calculated by Solid Queue 1.6 for a worker process at different concurrency levels:

On Active Record 7.2+, the thread estimate scales linearly while the fiber estimate stays flat. In the default fork mode, multiply the per-process pool by the number of worker processes: 6 processes with 50 execution slots means 312 configured queue connections for thread workers versus 18 for fiber workers. PostgreSQL’s default max_connections is 100.

Again, Solid Queue only calculates this estimate and warns. It does not configure the pool automatically. In supervisor async mode, workers share a process, so their connection needs must be added together rather than applying the per-process table independently.

The benchmarks below use two pool policies. The primary Solid Queue comparison deliberately gives both modes the same pool, DB_POOL = concurrency + 5 per worker process, so it measures the executor instead of measuring pool starvation. The stress suite uses mode-specific pools to show the operational failure envelope under higher connection demand.

The benchmarks

I reran the benchmark suite on April 28, 2026. These results were produced from the PR branch before the final 1.6.0 implementation was reorganized during review, so treat them as benchmarks of that implementation, not fresh Solid Queue 1.6.0 numbers. The architecture is the same, but a new run is required before attributing the exact deltas to the release tag. I will update this post over the next few weeks with fresh benchmarks against Solid Queue 1.6.0.

The headline Solid Queue comparison covers four workloads across per-process concurrency 5, 10, 25, 50, and 100; process counts 1, 2, and 6; and both execution modes. Three runs per cell, median real run reported, with total concurrency capped at 60 so the main comparison stays about executor behavior.

The workloads:

• Sleep: 50ms Kernel.sleep. Pure cooperative wait. The I/O upper bound.
• Async HTTP: HTTP request to a local server with 50ms delay via Async::HTTP. Real fiber-friendly I/O.
• CPU: 50,000 SHA256 iterations. Pure computation. The control.
• RubyLLM Stream: Actual RubyLLM chat completion through a fake OpenAI SSE endpoint, with token-by-token Turbo Stream broadcasts. 40 tokens at 20ms each. The closest thing to a production AI workload you can benchmark repeatably.

Results

RubyLLM Stream is the workload that matters. It runs an actual RubyLLM chat completion with streaming, database writes, and Turbo broadcasts per token – the same thing Chat with Work does in production. Fiber wins every single paired experiment there: 9 out of 9.

The CPU row is the control. Fibers don’t help computation, and the average confirms it: essentially flat. That’s how you know the I/O gains are real and not measurement noise.

That table shows the best observed point and the paired-cell deltas. Here’s the full spread. Some configurations favor threads for synthetic workloads, but the paired averages are the steadier signal: fiber wins the I/O workloads, and RubyLLM Stream always favors fiber.

The newer suite also adds database-shaped workloads. With matched pools, short DB bursts still favor fiber: db_queries averages +12.6%, and a read/API/write mix averages +6.9%. The transaction case is the useful caveat: when each job pins a connection for the whole transaction, fiber still averages +3.5%, but the win is less consistent. That’s exactly the workload where you should be more careful with pool sizing.

Thread mode hit the wall

Those benchmarks cap total concurrency at 60. I wanted to see what breaks when you push past that, so I ran a stress suite: per-process concurrency 25, 50, 100, 150, and 200; process counts 2 and 6; three runs per cell. Read this as the April PR implementation’s failure-envelope test, not a Solid Queue 1.6.0 result or a universal law about threads and fibers.

The result is stark. Thread mode only completed the smallest cell for each workload. Fiber mode completed every planned cell.

PostgreSQL’s default max_connections is 100. In this stress run, thread mode at concurrency 50 with 2 processes asked for 110 worker-pool connections. With 6 processes, even concurrency 25 asked for 180. The one surviving thread cell was the smallest: concurrency 25, 2 processes.

Fiber mode in the stress suite used a smaller mode-specific pool: 6 connections per process for 2-process runs, 10 per process for 6-process runs. That is 60 worker-pool connections at concurrency 200 across 6 processes, while the benchmark’s thread policy would configure 1,230. The exact constants are benchmark policy, but the shape is the point for this worker design: Solid Queue’s thread estimate scales with thread concurrency; the Active Record 7.2+ fiber baseline scales with worker process overhead plus actual database concurrency.

One backend, two modes

Fiber mode isn’t universally better. CPU-bound jobs get nothing from it, and blocking libraries or C extensions that do not cooperate with Ruby’s fiber scheduler stall the reactor. And that’s fine – you don’t have to pick one.

As Trevor Turk pointed out in the PR discussion, that’s the whole point: separately configured worker pools. Here’s what Chat with Work actually runs in production:

workers:
  - queues: [ chat ]
    fibers: 10
    processes: 2
    polling_interval: 0.1
  - queues: [ turbo ]
    fibers: 10
    processes: 1
    polling_interval: 0.05
  - queues: [ notifications, default, maintenance ]
    fibers: 5
    processes: 1
    polling_interval: 0.2
  - queues: [ cpu ]
    threads: 1
    processes: 1

Almost everything uses fibers. LLM streaming, Turbo broadcasts, notifications, maintenance jobs – all fiber-based. Only the cpu queue uses threads, and right now it’s just one thread for the occasional heavy extraction. One backend. One deployment. Mission Control shows all of it.

Instead of running Solid Queue and Async::Job side by side – two processors, two configurations, two sets of things to monitor – you run one. I moved Chat with Work to this setup, and Brad Gessler has been running it in production too.

Async::Job is actually faster if you compare raw throughput against Redis. It is a backend comparison, not a Solid Queue executor comparison, but the ceiling is useful:

If you want raw speed and don’t need persistence, Async::Job is the right call. But if you want job visibility, failure tracking, retries, Mission Control, everything Rails gives you out of the box, fiber mode gets you there. Same concurrency. You can size database connections to database work instead of the number of jobs waiting on network I/O. You set fibers: N and keep building.

Fiber mode is now available in Solid Queue 1.6.0. The PR has the implementation history, and the benchmark suite is open source. Run your own numbers, or challenge mine.

Researchers call it context rot: the more tokens in the window, the harder it becomes for the model to follow instructions, retrieve information, and stay on task. Chroma tested 18 frontier models and found that accuracy drops up to 30% when you go from a focused 300-token input to 113k tokens of conversation history, with the task held constant. The model essentially became dumber.

This holds true regardless of how big the window is, yet most agent setups treat the context window like a junk drawer.

“Just toss it in there, the LLM will figure it out!”

MCP: the biggest offender

Don’t get me wrong. MCP is a fine idea. You need to talk to a service? Grab an MCP server, plug it in, and you’re running in ten minutes. For prototyping, for exploration, for answering “is this even worth building?”, it’s great.

The problem is what happens next. Which is: nothing.

People leave the MCP servers plugged in. They add more. Every MCP server you connect dumps tool descriptions, schemas, and instructions into your context. You didn’t write those. You didn’t optimize them. You probably haven’t even read them. You’re handing over a chunk of your context window to whatever some third party decided to shove in there.

Say you need a tool that checks the weather. You could plug in an MCP server and get dozens of tool descriptions, parameter schemas, and whatever instructions its author decided to write. Or you could write this:

class Weather < RubyLLM::Tool
  description "Gets current weather for a location"

  param :latitude, desc: "Latitude (e.g., 52.5200)"
  param :longitude, desc: "Longitude (e.g., 13.4050)"

  def execute(latitude:, longitude:)
    url = "https://api.open-meteo.com/v1/forecast?latitude=#{latitude}&longitude=#{longitude}&current=temperature_2m,wind_speed_10m"
    Faraday.get(url).body
  rescue => e
    { error: e.message }
  end
end

Twelve lines of RubyLLM. You wrote the description, so you know exactly what tokens are going into your context. You wrote the parameters, so the model gets precisely the interface it needs, no more. You own it, you can tune it, and nobody can inject anything into your agent’s brain through it.

Use MCP to prototype. Then replace it with crafted tools you actually control.

Tool responses are context too

Your RAG retrieves ten full documents when the model needs a paragraph. Your API call returns a massive JSON blob when the model needs two fields. You’re paying for every one of those tokens with your agent’s IQ.

The fix is progressive disclosure. At Chat with Work, when the agent searches your Google Drive, we don’t dump entire files into context. The search tool returns only some metadata and a single line from the file, the line that matched the search keywords. Fifty results, fifty lines. The AI reads those, decides which files actually matter, and only then reads them. If a file is too large, it reads it in chunks. At every step, the model is only looking at what it needs.

The same principle applies to any tool. Don’t return everything. Return enough for the model to decide what to look at next.

Your instructions are context too

Then there’s the stuff you wrote yourself. Your system prompt is context. Your tool descriptions are context. Your parameter schemas are context. Every edge case, every guardrail, every overly detailed description competes for attention. You think you’re being thorough. You’re actually drowning the instructions that matter in a sea of instructions that don’t. A focused system prompt will outperform an exhaustive one every time.

Tool count is context too

You hand-crafted 40 beautiful tools. Your agent needs 5 for this task. The other 35 sit in context doing nothing except making the model slower at picking the right one.

Don’t register every tool your agent might ever need. Load the tools the current task actually requires. If you’re building a support agent that handles billing and technical issues, don’t give it all of both. Route billing questions to a billing agent and technical questions to a technical agent. Two focused agents will outperform one bloated one.

Every token should earn its place

The context window is not a junk drawer. It’s a workbench. Everything on it should be there for a reason, and you should be able to say what that reason is.

So before you plug in another MCP server, add another RAG source, or write another paragraph in your system prompt, ask yourself one question: is this worth making my agent dumber?