# Carmine Paolino > I build AI tools at Chat with Work and RubyLLM, help organize Berlin.rb, and co-founded Freshflow. I also make music and run Floppy Disco. ## About Carmine Paolino is an AI engineer, open-source maintainer, and founder building AI tools at Chat with Work and RubyLLM. He helps organize Berlin.rb, previously co-founded Freshflow, and also publishes music. - Role: AI engineer, open-source maintainer, and founder ## Key Topics - Ruby - Ruby on Rails - Jekyll - AI applications - Large language models - LLM APIs - RubyLLM - Open source software - Async Ruby - Developer tooling - Product engineering - Startups - Music technology - House music ## Core Topics Ruby, Rails, Jekyll, AI applications, LLM APIs, open source, async Ruby, developer tooling, product engineering, startups, and music. ## Primary Projects RubyLLM, Chat with Work, Fastpotify, ArchSpec, Schematist, hyprmoncfg, Kamal Backup, FastsApp, Cluster Headache Tracker, Jekyll VitePress, Omarchy plugins (OmaStats, Ultimate Guitar Tabs, Lyrics Synced with Music, hyprmoncfg for Omarchy, Active Window + Icon, and Media Controls + Album Art), Berlin.rb, Crimson Lake, Floppy Disco, and Mindscape Productions. Explore the directory at https://paolino.me/projects/. ## Posts ### Berlin.rb: A New Monthly Ruby Meetup in Berlin URL: https://paolino.me/berlin-rb/ Date: 2026-09-02 [Berlin.rb](https://berlinrb.org) is a new monthly meetup for people who write Ruby, or are curious about it. A few of us from Berlin's Ruby community are organizing talks, project show-and-tells, and time to meet other people who work in the same language. It's free and open to anyone, whether you've been writing Ruby for fifteen years or installed it last week. **The first one is Tuesday, September 8, 18:45, at Potsdamer Straße 121. Use the Grover entrance next to Rossmann and head to the third floor.** The evening is hosted and sponsored by [ablefy](https://ablefy.io), with drinks and snacks provided. After that, the second Tuesday of every month. [Register here.](https://luma.com/lnmj7085) If you can't make it to Berlin, there's a [livestream](https://luma.com/psvh08jx) and it'll be on [YouTube](https://www.youtube.com/@berlinrb_org) afterwards. ## The first meetup I'm giving the first talk anywhere on [RubyLLM](https://rubyllm.com) 2.0: **RubyLLM 2.0: Beyond Agents**. **Paweł Strzałkowski**, CTO at Visuality, is giving the second: **HiFuMi: The Story of Building an Online Rails Apps Generator**. ## Why another one Berlin already has [RUG::B](https://www.rug-b.de), a long-running Ruby user group. Berlin is big enough for both, and more Ruby events are a good thing. Berlin.rb adds a predictable rhythm: second Tuesday, every month. Put it in your calendar once and stop checking. ## Bigger than Berlin We don't want this to be just a room in Berlin. 1. We're livestreaming the meetup and publishing the talks on [YouTube](https://www.youtube.com/@berlinrb_org) afterwards, so joining Berlin.rb doesn't require being in Berlin. 2. Berlin.rb is part of [Ruby Europe](https://rubyeurope.com), the umbrella bringing together Ruby communities across the continent and moving speakers between cities. ## Come talk We want speakers! A deep technical talk, a lightning talk about something you built last weekend, a story about something that went badly in production. If you've never given a talk before, this is a good place for the first one. Send a sentence or two to [hello@berlinrb.org](mailto:hello@berlinrb.org) and we'll take it from there. ## Host or sponsor us We want to move around the city. Different neighbourhoods, different offices, so it's easy to reach wherever you happen to live, and so you get to see where other Berlin Ruby teams actually work. If your company has a space free on a weekday evening, or wants to cover food and drinks, we'd love to hear from you. Sponsorship is what keeps this free and open to everyone, and hosting is the easiest way to put your team in front of Berlin's Ruby community. [hello@berlinrb.org](mailto:hello@berlinrb.org) ## The people Berlin.rb is put together by a group of us from around Berlin's Ruby community. The current organizers are listed on [berlinrb.org](https://berlinrb.org/#faq). [berlinrb.org](https://berlinrb.org) · [Luma](https://luma.com/berlinrb) · [YouTube](https://www.youtube.com/@berlinrb_org) · [X](https://x.com/berlinrb_org) · [Bluesky](https://bsky.app/profile/berlinrb.bsky.social) · [LinkedIn](https://www.linkedin.com/company/berlinrb) See you on the 8th. --- ### RubyLLM 2.0: The Agentic Loop, Exposed URL: https://paolino.me/rubyllm-2-0-agentic-loop/ Date: 2026-08-28 Strip any agent framework down and you find the same loop: call the model, run the tools it asked for, call the model again, stop when it answers without wanting a tool. RubyLLM has run that loop inside `ask` since 1.0. In 2.0, you can also control each step. ```ruby # Run the agentic loop automatically response = RubyLLM.chat(model: "claude-sonnet-4-6") .with_tools(Weather) .ask("What's the weather in Paris?") # => # "Here's the current weather in **Paris, France**:\n\n- 🌡️ **Tempera... ``` `ask` still runs the loop for you, stopping when the answer is ready or a tool needs human approval. You can also call each part yourself: * `ask_later` stages your message without sending anything. * `generate` makes one model call and appends the response. * `run_tools` executes pending tool calls and appends their results without calling the model. * `step` runs pending tools if any are unanswered, otherwise it calls the model. * `complete?` tells you when the conversation is settled: the model answered without calling a tool. * `complete` steps until done or awaiting approval. `ask` is `ask_later` followed by `complete`. This lets you set an iteration budget, batch the next generation, wait for approval, or save progress and continue in another job. Your code can check what happened between calls. ## One Step Per Job Each verb decides what to do next by reading the persisted messages. That means the loop doesn't need to live in one process, or one machine, or one deploy: ```ruby class AgentTurnJob < ApplicationJob def perform(chat_id) chat = Chat.find(chat_id).with_tools(Weather) chat.step AgentTurnJob.perform_later(chat_id) unless chat.complete? || chat.awaiting_approval? end end ``` Each step gets its own job and retry boundary. A long sequence can release the worker between steps; an individual provider request or tool still takes as long as it takes. The loop is resumable mid-tool-round too. `run_tools` skips calls whose results have been saved. If a process dies after saving one result out of three, reloading the chat and calling `step` executes only the remaining two. If it dies after an external action succeeds but before saving its result, the tool can run again. Use `tool_call.id` as an idempotency key for writes. On Rails 8.1 and later, you can use ActiveJob Continuations to build on this: checkpoint after each move and an agent run survives a redeploy, resuming from the persisted messages with no cursor to manage. Batches are the same idea at scale: a batch is `generate` deferred for many chats at once, with `run_tools` run locally between rounds. ## Cancelable generation `chat.cancel` cancels a run from another thread. At the next checkpoint, before a model call, before a tool executes, or between streamed chunks, the run raises `RubyLLM::CancelledError` and clears the flag so the chat can be reused. In Rails, `acts_as_chat` stores the cancellation request on the chat record, so the signal travels through the database. A stop button in your web process halts a background job mid-stream: ```ruby class ChatsController < ApplicationController def cancel current_user.chats.find(params[:id]).cancel head :no_content end end ``` The job checks the chat record at cancellation checkpoints. It cannot interrupt arbitrary Ruby code inside a running tool; the next checkpoint observes the request. ## Halt Is Gone RubyLLM 1.x let a tool terminate the loop from the inside: return `halt("done")` and the conversation ended. That put control flow inside a return value, and it's gone in 2.0, along with `RubyLLM::Tool::Halt`. Tools return results. Stopping belongs to the caller: ```ruby until chat.complete? || chat.awaiting_approval? chat.step break if handed_off? # application-specific stopping condition end ``` If what you want is one tool call per model response rather than a condition, `chat.with_tool_options(calls: :one)` does that. For a total round budget, count `step` or `generate` calls in the loop you control. The full guide, including the workflow patterns built on these verbs, is at https://rubyllm.com/next/agentic-workflows/. --- ### RubyLLM 2.0: Providers, Protocols, and Provider Gems URL: https://paolino.me/rubyllm-2-0-providers-and-protocols/ Date: 2026-08-27 RubyLLM 2.0 is almost ready. It isn't out yet, but it will be soon, and I have been looking forward to showing you what is in it. There is a lot in this release. Too much for one enormous announcement, and most of it deserves more than a bullet point. So this is the first in a series of posts about what's coming in RubyLLM 2.0. Let's start with providers and protocols. In 2.0, OpenAI uses the Responses API by default. Providers and protocols are separate things. Four new providers bring the total to seventeen. And if the provider you need is still missing, a new generator gives you a complete provider gem to start from. ```ruby RubyLLM.chat(model: 'gpt-5.4') # OpenAI Responses API RubyLLM.chat(model: 'gpt-5.4', protocol: :chat_completions) # same model, old API RubyLLM.chat(model: 'claude-opus-4-6', provider: :vertexai) # Vertex AI, Anthropic protocol ``` ## A provider is not a protocol A provider is the service you connect to: OpenAI, Mistral, Vertex AI, or one of the others. It knows the host, credentials, configuration, and model catalog. A protocol is the API it speaks: Chat Completions, Responses, Anthropic, Gemini, Bedrock Converse, or Cohere. It knows how to build a request, parse the response, and handle streaming. Mistral, DeepSeek, Perplexity, Ollama, and most self-hosted services all speak some version of OpenAI's Chat Completions API. In 1.x, I handled that with inheritance. `Mistral "Bearer #{@config.mistral_api_key}" } end def batch_cost_multiplier(**) = 0.5 class << self def capabilities Mistral::Capabilities end def models_dev_alias(...) Mistral::Models.models_dev_alias(...) end def configuration_options %i[mistral_api_key mistral_api_base] end def configuration_requirements %i[mistral_api_key] end end end end end ``` That's the whole file. Request formatting and parsing live in the protocol classes, so this adapter can concentrate on endpoints, authentication, and choosing the right protocol. ## Build the next provider yourself The provider and protocol split pays off outside RubyLLM too. The most common feature request is another provider. In 2.0, you do not have to wait for me to add one. ```bash ruby_llm provider-gem Acme --api-base https://api.acme.ai/v1 ``` That command creates `ruby_llm-providers-acme`, initializes Git, installs the bundle, and gives you an ignored `.env`, provider registration, a model-catalog task, live-recording specs, RuboCop, Flay, ArchSpec, and CI across every supported Ruby version. It gets the same kind of care as RubyLLM itself, already wired up. The important part is what you do not have to build. If Acme speaks Chat Completions, it reuses RubyLLM's existing protocol for requests, responses, streaming, tools, and errors. If it speaks another familiar API, pass `--dialect responses`, `anthropic`, `gemini`, `converse`, or `ollama`. The provider only needs to supply its host, authentication, model catalog, and any real quirks it has. That is what the split makes possible. A new or smaller provider no longer needs another OpenAI-compatible implementation, another Anthropic-compatible implementation, or a growing pile of API-base switches and provider exceptions inside RubyLLM. It gets to reuse the protocol and remain a small adapter. When it is ready, tell your users to add it to their Gemfile: ```ruby gem 'ruby_llm-providers-acme', require: 'ruby_llm/providers/acme' ``` With their API key configured, that is it. The gem registers itself, brings its model catalog, and works through the same `RubyLLM.chat` API as a built-in provider. Once 2.0 is out, go ahead and try it yourself. I'll save the complete tutorial for another post. ## A model registry applications can rely on The model registry that RubyLLM uses for capabilities and pricing is now published at [rubyllm.com/models.json](https://rubyllm.com/models.json). For the first time, anyone can download, inspect, or build on the same catalog RubyLLM uses itself. [models.dev](https://models.dev) is an excellent source, but RubyLLM needs more than a copy of it. Every six hours, RubyLLM rebuilds its registry from models.dev and the providers' own APIs, reconciles aliases, fills gaps, applies the few provider-specific corrections that remain, validates the result, and refuses suspicious regressions before publishing it. RubyLLM applications use the same registry to validate model names, choose protocols, check capabilities, and price provider usage. That last part demands precision: if a price, modality, or capability is wrong, the answer your application gets is wrong too. I will cover the cost ledger in another post, but the registry is what makes it possible. ```ruby RubyLLM.models.refresh ``` `refresh` fetches the latest main registry and persists it. New models, prices, context windows, and capabilities can reach your application without waiting for the next gem release. Provider gems can ship their own `models.json` too. RubyLLM loads it as a read-only fallback behind the main registry, so installing a provider gem is enough to use its models normally: The provider gem's models appear in `RubyLLM.models` and can be selected with the normal `model:` and `provider:` keywords. The global `refresh` never refreshes or rewrites provider gem catalogs. Their authors update them with `rake models` inside the provider gem. The important part is that none of this makes the public API more complicated. `RubyLLM.chat`, `embed`, `paint`, and the Rails integration still work the same way. Most applications get broader provider coverage through the APIs they already use. The new `protocol:` option is there for the times when you want to choose. That is the first piece of RubyLLM 2.0. Next up: the agentic loop, and how 2.0 lets you stop it, resume it, and run it one step at a time. The full guide to writing providers and protocols is at [rubyllm.com/next/custom-providers](https://rubyllm.com/next/custom-providers/). --- ### kamal-backup 1.0: A Backup Is Only Real After You Restore It URL: https://paolino.me/kamal-backup-1-0/ Date: 2026-08-27 I released [kamal-backup](https://kamal-backup.dev) 1.0 today. Not because the backup command works. That part worked months ago. I called it 1.0 because I used it to move [Chat with Work](https://chatwithwork.com), the application that pays my bills, to a new Hetzner instance. The old host took the backup. The new host restored it under a temporary hostname. I opened the real application, checked the real data and files, and only then moved DNS. A backup is only real after you restore it. ## The migration that made 1.0 The move found two bugs that a green backup log never could. First, a fresh backup accessory on the new host could start its schedule before the restore and write a perfectly valid, completely empty snapshot into the same repository. Ask for `latest` during the migration and you could restore that one. Second, Kamal had already run Rails' `db:prepare` on the new host. PostgreSQL's normal `pg_restore --clean` path could not reliably replace that prepared schema when target-only foreign keys got in the way. Drops failed, creates failed because objects still existed, data never loaded, and `pg_restore` could still look more successful than the database it left behind. Those problems became the restore model in 1.0. A replacement host can boot its accessory with scheduled backups disabled. PostgreSQL restores remove the target's user schemas first, recreate `public`, restore with a client matching the server, and fail if `pg_restore` reports ignored errors. The workflow is now written down in the [host migration guide](https://kamal-backup.dev/migrating-hosts/): restore under a temporary hostname, verify the application, stop writes to the old host, take and restore a final backup if necessary, then move traffic. ## Exact means exact A restore should leave the target looking like the snapshot, not like the snapshot layered over whatever happened to be there already. That promise now applies to every supported database: - PostgreSQL removes every non-system schema before restoring the custom-format dump. - MySQL and MariaDB remove existing views, tables, sequences, routines, functions, and events before importing. - SQLite uses its native `.backup` and `.restore` APIs, checks the downloaded database before touching the target, and runs `quick_check` again afterward. The file restore runs before the database restore. That matters when a Rails SQLite database and file-backed Active Storage live on the same volume: the clean SQLite backup stays separate from the raw database, WAL, and shared-memory files, and restoring the files cannot delete the database that was just restored. Production restores require the application, jobs, and other database writers to be stopped. Restore drills use scratch databases, scratch SQLite files, and scratch file paths instead of touching the live targets. The point is not to make destructive operations feel casual. It is to make the safe procedure obvious and repeatable. ## Tested beyond PostgreSQL The Chat with Work migration was PostgreSQL. I do not have a production MySQL or SQLite migration story to pretend otherwise. What 1.0 has instead is an exact backup-and-restore matrix that builds the real accessory image and exercises: - PostgreSQL 14, 15, 16, 17, and 18; - MySQL 8.0 and 8.4; - MariaDB 10.11, 11.4, and 11.8; - SQLite in WAL mode, including backup and restore through restic's rclone backend. The matrix creates data, views, routines, functions, triggers, events, custom PostgreSQL schemas, and MariaDB sequences. It adds objects that exist only in the restore target, restores the snapshot, and checks that the stored objects work and the target-only objects are gone. The accessory image ships for amd64 and arm64. It includes matching PostgreSQL clients, the MariaDB client tools used for MySQL and MariaDB, SQLite, restic, rclone, and SSH. Restic's native repositories still work, SFTP works with a dedicated SSH key, and rclone opens the rest of its storage providers without turning kamal-backup into a storage abstraction of its own. ## The same small Kamal accessory The shape of the project has not changed since [I first released it](/kamal-backup/): a Ruby gem gives your Rails repository a CLI, and a Docker image runs the scheduled backups beside your application as a Kamal accessory. ```ruby group :development do gem "kamal-backup", "~> 1.0" end ``` ```sh bundle install bundle exec kamal-backup init bundle exec kamal-backup validate bin/kamal accessory boot backup bundle exec kamal-backup backup --force bundle exec kamal-backup drill production latest ``` It backs up one or more PostgreSQL, MySQL, MariaDB, or SQLite databases, plus the file paths you explicitly configure for file-backed Active Storage. Restic handles encryption, deduplication, retention, and repository integrity. `kamal-backup evidence` turns the configuration, latest snapshots, checks, drills, retention, and tool versions into redacted JSON for the next person who asks whether the backups actually work. Version 1.0 does not mean backups are finished. It means the interface and the restore promises are ready to depend on. Read the [documentation](https://kamal-backup.dev), look through the [restore guide](https://kamal-backup.dev/restore/), and get the source on [GitHub](https://github.com/crmne/kamal-backup). Then run a restore drill. The green backup log is the beginning of the test, not the end. --- ### ArchSpec 1.0: Executable Architecture Specification for Ruby's Agentic Coding Era URL: https://paolino.me/archspec/ Date: 2026-08-20 More and more code is written by a model. Tests still tell you it works. RuboCop still tells you it's tidy. Nothing tells you it still follows your architecture. I released [ArchSpec](https://archspecrb.dev) 1.0 today. It's an architecture linter for Ruby and Rails. You declare your components and boundaries in one file, and every change gets checked, whether a person or an agent wrote it. _This is part of my push towards making Ruby one of the best languages to build with AI. [RubyLLM](https://rubyllm.com) is one piece. [Schematist](/schematist/) is another. [Making the default Rails job queue fiber-based](/solid-queue-doesnt-need-a-thread-per-job/) is another._ ## People and AIs take shortcuts An agent or a person that's in a hurry or doesn't fully understand your architecture takes shortcuts. The shortcuts work. They pass tests, implement features, and _cross your boundaries_. Months later, you realise your beautifully crafted architecture is now a patched mess. Or, perhaps, you're starting a new project and you want to ensure that the agent is using a _good_ architecture, without having to police it at every step. So you write it down in prompts and `AGENTS.md`. It helps, but it gets buried in its context window and something slips. Again, then again. RuboCop reads your code and enforces a style. [Herb](https://herb-tools.dev) does it for templates. Nothing did it for architecture. Until now. ## ArchSpec: your architecture in one file Declare your components and their rules in an `Archspec.rb` at the root of the project: ```ruby component :models, in: "app/models/**/*.rb" component :controllers, in: "app/controllers/**/*.rb" component :services, in: "app/services/**/*.rb" models.cannot_use :controllers services.cannot_call :render, :redirect_to, receiver: :none controllers.can_only_use :models, :services ``` Then `archspec check` verifies every change. Models can't reach into controllers. Domain code can't touch adapters. Query objects can't call `save!`. A pack exposes a public API and keeps everything else private. A directory has to stay empty, and says why. If you'd rather not write rules at all, start from a preset: ```ruby architecture :vanilla_rails ``` That one line is the 37signals playbook: rich models, no service objects, no form objects, no policy objects, and `app/services` fails the build with a reason if anything shows up in it. There are presets for Rails, layered, hexagonal, clean architecture, modular monoliths, CQRS, and event-driven too. ## Static Analysis, Not AI I'm the author of RubyLLM so you'd think this uses AI. Nope. _ArchSpec doesn't use AI_. Here's how it works: Prism parses your Ruby, then ArchSpec extracts facts, references, inheritance, mixins, calls, definitions, and evaluates your rules against them. It's deterministic, it's offline, and it's fast enough that you'll leave it on: the full Discourse app, 1,899 files, was checked in 2.5 seconds, without booting the app. Prism is its only runtime dependency. No Rails, no ActiveSupport, nothing else, so it works on any Ruby codebase. RubyLLM is a plain gem and it's been the main proving ground since June. It also won't guess. ArchSpec doesn't try to infer the "true" design pattern of arbitrary Ruby. You describe the architecture you want, and it tells you whether the code still matches. The AI is on the other side of the loop, writing the code that gets checked. ## Failures an Agent Can Act On When a rule breaks, you get this: ```text [error] models must not depend on controllers [dependencies.forbid] app/models/user.rb:3:5 2 │ def admin_path → 3 │ UsersController.admin_path_for(self) │ ^~~~~~~~~~~~~~~ 4 │ end note: User references UsersController 1 architecture violation found. ``` The format is a deliberate homage to clang and to Herb. Exact location, the offending span underlined, the evidence as a note, the rule id in brackets so you can suppress it narrowly. A human reads it at a glance. An agent gets everything it needs to fix its own mistake without asking you: the file, the line, the rule, and why. ## What It Caught in RubyLLM I added ArchSpec checks to [RubyLLM](https://rubyllm.com) [2 months ago](https://github.com/crmne/ruby_llm/commit/f1cf3b0e92e3e9244a94a2c1b5c7d4f2716d2aae), in CI and as a pre-commit hook, and it has been instrumental in the big Protocol/Provider separation that's coming in RubyLLM 2.0. A protocol is a wire format, like Chat Completions, Responses, Anthropic's Messages API, Gemini, or Bedrock Converse. A provider is an account you can talk to, like OpenAI, Azure, DeepSeek, or Ollama. DeepSeek speaks Chat Completions. VertexAI speaks four: Gemini, Anthropic, Mistral, and Chat Completions. Writing each Protocol once is the reason the gem supports as many providers as it does. That distinction is easy to state and easy to erode. The shortcut is to put a piece of wire format inside the provider that needs it, because right now that's the only provider that needs it. Not on my watch: ```ruby providers.cannot_reference_constants 'RubyLLM::Protocol' ``` A provider can subclass a protocol family to change an endpoint or work around a quirk. Subclassing the bare `Protocol` means it's inventing a wire format inside an adapter: ```text [error] providers must not reference RubyLLM::Protocol [constants.forbid] lib/ruby_llm/providers/elevenlabs/audio.rb:9:21 8 │ # image endpoints, so those seams are left unimplemented. → 9 │ class Audio .+)/) .requires('%s', on: agent, scope: :class, except: %i[with_temperature with_max_output_tokens]) ``` `Agent` is a declarative wrapper over `Chat`, so every `Chat#with_x` setter needs a matching class-level macro on `Agent`. Adding the setter is the interesting half. Adding the macro is not. Add `Chat#with_verbosity`, forget `Agent.verbosity`, and the build tells you before your users do. ### Make promises to your users ```ruby domain.cannot_reference_constants 'RubyLLM::ActiveRecord' ``` `require "ruby_llm"` works without Rails. That only stays true if the plain-Ruby objects never reach into the Rails integration. Without a check, you find out from a bug report. Not every codebase needs a spec that long. [Chat with Work](https://chatwithwork.com) runs its entire ruleset in one line, `architecture :vanilla_rails`, because the moat is the product and the code should stay plain and boring. ## But Isn't This Packwerk? [Packwerk](https://github.com/Shopify/packwerk) is good, and if packs are what you need, use it. ArchSpec covers that case with `architecture :modular_monolith`, and then keeps going. Packwerk checks constant references between packages. By design it ignores method calls, and it leans on Zeitwerk to resolve names, which means it's shaped like a Rails app. ArchSpec checks constant references too, plus calls, inheritance, mixins, required methods, cycles, and naming conventions, in one Ruby file, on any Ruby codebase. Different scopes for different folks. ## Getting It Into Your Codebase If your app is conventional, a preset is the whole file: `architecture :rails`, `architecture :vanilla_rails`, `architecture :hexagonal`. If it isn't, describe your architecture to a coding agent and have it draft the `Archspec.rb`. Agents are genuinely good at this. They can read the whole tree, they already know what your components are, and the DSL is small. Then read what it wrote, carefully, because the spec is the part you own, then add the parts the agents missed. If one of our `architecture` presets is incomplete, or you'd like to add another one, send me an [issue](https://github.com/crmne/archspec/issues) or a [PR](https://github.com/crmne/archspec/pulls). I want to make this the best architecture linter around. Existing codebases have existing violations. `archspec check --update-todo` records them in a todo file, so the build goes green on today's code and fails on new drift. Work the list down whenever. Then put it in a pre-commit hook and a CI step. Every provider gem scaffolded by RubyLLM 2.0's new provider generator is born with its own `Archspec.rb`, an archspec step in the default rake task, and the hook. New code starts life with an architecture spec the same way it starts with tests. Keep those pesky agents with `--dangerously-skip-permissions` accountable. ## Where It Came From This started at RubyConf Austria in May. The [AI panel](https://radan.dev/news/ruby-conf-at) and the hallway conversations around it, mostly with Chad Fowler and José Valim, kept landing in the same place from different directions: if models write the code, the human job concentrates in the decisions above the code. I've argued before that [engineering is not dead, because accountability isn't](/engineering-is-not-dead/). Vienna sharpened the follow-up. What do you actually use to hold that line? Flying home, I kept thinking about the tools we already have. So I built one, showed an early version to José, who was encouraging, and released 0.1 quietly in June. It's been running on every commit in ArchSpec itself, in RubyLLM, in [Chat with Work](https://chatwithwork.com), and in everything else I've made in Ruby ever since. Every release is torture-tested against pinned checkouts of Discourse, Mastodon, and Basecamp's Fizzy, where the per-rule diagnostic counts have to match recorded snapshots before anything ships. ## Use It ```sh bundle add archspec bundle exec archspec init bundle exec archspec check ``` Docs at [archspecrb.dev](https://archspecrb.dev), source on [GitHub](https://github.com/crmne/archspec). File issues for anything it gets wrong. Agents can write the code. The architecture is still yours to keep. --- ### RubyLLM::Schema Is Now Schematist: A JSON Schema DSL for Ruby with Full Draft 2020-12 Coverage URL: https://paolino.me/schematist/ Date: 2026-08-11 I want to make Ruby the best language to work with LLMs. Part of that is a great JSON Schema DSL. [Schematist](https://github.com/crmne/schematist) is a general purpose JSON Schema DSL that emits Draft 2020-12 schemas. Describe an API payload, a config file, a contract between two services, or the structured output you want back from a model. Trapping that inside another gem's namespace was a disservice to anyone looking for a great JSON Schema DSL, so it got its own name. ```ruby gem 'schematist' ``` ## It Emits Actual JSON Schema This is the breaking change. `to_json_schema` used to return this: ```ruby { name: "PersonSchema", description: nil, schema: { type: "object", ... }, strict: true } ``` That's not a JSON Schema. It's OpenAI's `response_format` envelope, with the actual schema buried one level down under a symbol key. Every consumer that wasn't OpenAI had to dig it out, and anyone who wanted to hand the result to a validator had to know which part was real. Now you get the document: ```ruby class Invoice { # "$schema" => "https://json-schema.org/draft/2020-12/schema", # "title" => "Invoice", # "description" => "A billing document", # "type" => "object", # "properties" => { # "id" => { "type" => "string", "pattern" => "^inv_", "title" => "Invoice ID" }, # "total" => { "type" => "number", "description" => "Amount due", "exclusiveMinimum" => 0 }, # ... # }, # "required" => ["id", "total", "currency", "status"], # "additionalProperties" => false # } ``` String keys, `$schema` declared, no provider keys. Use it with `JSON.generate` unchanged and any Draft 2020-12 validator will take it. `strict` went with it. It's an OpenAI request flag, not a JSON Schema keyword, and a schema library has no business knowing OpenAI exists. Set it where you build the request. ## Full Draft 2020-12 Coverage The old gem covered the basics: types, `enum`, `required`, string and numeric bounds, nested objects and arrays, `$defs` and `$ref`, `if`/`then`/`else`. [Schematist](https://github.com/crmne/schematist) covers the whole vocabulary. **Composition.** `allOf`, `oneOf`, and `not` join `anyOf`: ```ruby one_of :method do object { string :card_number } object { string :iban } end all_of :account, unevaluated_properties: false do object { string :id } object { string :status } end none_of :state do string enum: ["deleted"] end ``` `unevaluated_properties` is the one that makes `allOf` usable in practice. `additionalProperties` can't see across composition branches; `unevaluatedProperties` can. **Object keys.** Constrain how many properties an object has, what its keys look like, and what the values behind a key pattern must be: ```ruby object :metadata, min_properties: 1, max_properties: 10 do keys { string pattern: "^[a-z_]+$" } # propertyNames keys_matching(/^x-/) { string } # patternProperties end ``` **Arrays.** `uniqueItems`, fixed-length tuples via `prefixItems`, and `contains` with its bounds: ```ruby array :tags, of: :string, unique: true tuple :period do string format: "date" string format: "date" end array :scores do integer contains(min: 1) { integer minimum: 10 } # at least one score of 10 or more end ``` **Annotations.** `title`, `description`, `default`, `examples`, `deprecated`, `read_only`, `write_only`. Short ones read well as keyword arguments; longer ones read better in the block, where they annotate the enclosing schema: ```ruby object :account do title "Account" description "Billing account metadata used for invoices." examples [{ id: "acct_123", status: "active" }] string :id string :status end ``` **Encoded content.** For strings that carry something else inside them: ```ruby string :payload, content_encoding: "base64", content_media_type: "application/json" do content_schema do object { string :name } end end ``` **Core keywords.** `$id`, `$anchor`, `$comment`, `$dynamicAnchor`, `$dynamicRef`, `$vocabulary`, at the root or on any subschema. They're passed straight through. Resolving a dynamic reference is the validator's job, not ours. Also new: `const` on every primitive, and `greater_than` / `less_than` for `exclusiveMinimum` / `exclusiveMaximum`. I picked the Ruby-sounding names over the JSON Schema ones on purpose. You're writing Ruby. ## Values That Aren't Known Until Render Time You define a schema class once, at boot. The allowed values often aren't known until a request comes in. Any value can be a proc now, resolved when the document is rendered: ```ruby class RoleSchema { @account.roles.pluck(:name) } def initialize(account:) super() @account = account end end RoleSchema.new(account: account).to_json_schema ``` A zero-argument proc is evaluated in the instance's context, so it can read instance variables. A proc that takes one argument gets the schema instance instead. One class, a different document per instance. ## Escape Hatches Covering the spec isn't the same as guessing everything you'll want to put in a document, so there are two ways out. JSON Schema allows `true` and `false` in place of a schema object. `true` accepts anything, `false` accepts nothing: ```ruby any_of :value do any_schema string end ``` And `raw` drops a fragment in as-is, for a vendor extension or anything else the DSL has no opinion about: ```ruby raw :vendor, { "type" => "object", "x-vendor" => true } ``` ## A Schema Doesn't Have To Be an Object Most schemas describe an object, so that's the default. But JSON Schema doesn't care. A schema can be an array, a union, a string, or a pointer somewhere else, and the root of a document is just a schema like any other. So: a type with a name declares a property. Without a name, it declares what the schema itself is. ```ruby class Tags "https://example.com/person.json" }) end ``` It works inside `define` too, so a reusable definition can be a string with a pattern or a shared enum, not just an object: ```ruby define :status do string enum: %w[draft sent paid] end ``` A conditional branch is a schema too, so it can ask for a nested object instead of a flat list of fields: ```ruby given kind: "business" do requires :vat_id object :tax_details do string :vat_number end end ``` ## No Runtime Dependencies [Schematist](https://github.com/crmne/schematist) depends on nothing. ## Migrating ```ruby gem 'schematist' # was: gem 'ruby_llm-schema' class Person < Schematist::Schema # was: RubyLLM::Schema end ``` Errors moved up a level: `Schematist::ValidationError`, not `RubyLLM::Schema::ValidationError`. `Schematist::Helpers` replaces `RubyLLM::Helpers`. If you were reaching into `[:schema]` to get at the document, stop. `to_json_schema` returns it directly now, with string keys. If you need the provider wrapper, build it where you send the request: ```ruby { name: "Invoice", schema: Invoice.new.to_json_schema, strict: true } ``` There's a final `ruby_llm-schema` 1.0.0 that depends on [Schematist](https://github.com/crmne/schematist) and aliases the old constants, so `RubyLLM::Schema` keeps resolving while you move. It warns on load and it's the last release of that name. RubyLLM 2.0 will depend on Schematist, so structured output will get a lot more powerful. ## Use It ```bash bundle add schematist ``` [Schematist](https://github.com/crmne/schematist) was always a JSON Schema DSL. Now it has the name to match. --- ### Founding a Company in Germany: €9,600, 152 Days, and I Still Can't Send an Invoice URL: https://paolino.me/founding-a-company-in-germany/ Date: 2026-06-24 I started founding my second company in Germany in late January. It is now late June. In that time, the state, two courts, a notary, a law firm, a tax firm, and software vendors have all found a way to bill me. Every single one of them, on time. I have spent more than 9,600 euros to start a company: a little over 7,600 in fees and bills, plus 2,000 in share capital frozen in an account I am not allowed to touch. And after five months, here is what I have to show for it: I have not been able to send a single invoice of my own. Not one. The work is happening. The clients are real. The one thing the state exists to let me do, bill them cleanly, is the one thing I still can't. ## The timeline .ftl-tl{--ftl-accent:#ffe66a;--ftl-ink:#17181e;--ftl-gray:#bbb;--ftl-grayblue:#4a4b51;--ftl-line:#e6e6e6;--ftl-radius:3px;margin:2.5em 0;font-family:"Inter",ui-sans-serif,-apple-system,system-ui,"Segoe UI",Helvetica,Arial,sans-serif;color:var(--ftl-ink);-webkit-font-smoothing:antialiased} .ftl-tl *{box-sizing:border-box} .ftl-tl ol{list-style:none;margin:0;padding:0;position:relative} .ftl-tl ol::before{content:"";position:absolute;left:31px;top:18px;bottom:30px;width:2px;background:linear-gradient(180deg,var(--ftl-line),var(--ftl-line) 70%,#f2f2f2);border-radius:2px} .ftl-row{position:relative;display:grid;grid-template-columns:64px minmax(0,1fr) auto;column-gap:18px;align-items:start;padding:0 0 34px} .ftl-cal{position:relative;z-index:2;width:64px;border-radius:6px;overflow:hidden;background:#fff;border:1px solid var(--ftl-line);box-shadow:0 1px 0 rgba(23,24,30,.05),0 6px 16px -11px rgba(23,24,30,.4);text-align:center;line-height:1} .ftl-cal::after{content:"";position:absolute;top:0;right:0;border-width:0 11px 11px 0;border-style:solid;border-color:#f4f4f4 #fff;border-bottom-color:var(--ftl-line)} .ftl-cal-m{display:block;background:var(--ftl-accent);color:var(--ftl-ink);font-size:10px;font-weight:700;letter-spacing:.11em;text-transform:uppercase;padding:4px 2px 3px;font-family:"IBM Plex Mono",ui-monospace,Menlo,Consolas,monospace} .ftl-cal-d{display:block;font-family:"Lora",Georgia,serif;font-weight:700;font-size:22px;padding:7px 2px 8px} .ftl-cal-d small{display:block;font-family:"IBM Plex Mono",ui-monospace,Menlo,Consolas,monospace;font-weight:600;font-size:9px;line-height:1.3;letter-spacing:.07em;color:var(--ftl-grayblue);text-transform:uppercase} .ftl-row.ftl-today .ftl-cal{border-color:var(--ftl-ink);box-shadow:0 0 0 3px rgba(255,230,106,.6),0 6px 16px -11px rgba(23,24,30,.45)} .ftl-ev{padding-top:1px;min-width:0} .ftl-date{display:block;font-family:"IBM Plex Mono",ui-monospace,Menlo,Consolas,monospace;font-size:11px;letter-spacing:.04em;text-transform:uppercase;color:var(--ftl-grayblue);margin:0 0 4px} .ftl-ev p{margin:0;font-size:.92em;line-height:1.5} .ftl-ev p code{font-family:"IBM Plex Mono",ui-monospace,Menlo,Consolas,monospace;font-size:.82em;font-weight:600;background:#fbf6dc;border:1px solid #efe5b0;color:var(--ftl-ink);padding:1px 5px;border-radius:var(--ftl-radius)} .ftl-money{align-self:start;text-align:right;white-space:nowrap;padding-top:1px} .ftl-money code{font-family:"IBM Plex Mono",ui-monospace,Menlo,Consolas,monospace;font-size:.78em;font-weight:600;background:#fbf6dc;border:1px solid #efe5b0;color:var(--ftl-ink);padding:3px 8px;border-radius:var(--ftl-radius)} .ftl-money.ftl-locked code{background:repeating-linear-gradient(135deg,#f5f5f6,#f5f5f6 5px,#ececee 5px,#ececee 10px);border:1px solid #dadade;color:var(--ftl-grayblue)} .ftl-tag{display:block;margin-top:5px;font-family:"IBM Plex Mono",ui-monospace,Menlo,Consolas,monospace;font-size:9px;letter-spacing:.09em;text-transform:uppercase;color:var(--ftl-gray)} .ftl-locked .ftl-tag{color:var(--ftl-grayblue)} .ftl-gap{position:relative;display:flex;padding:0 0 30px} .ftl-gap span{margin-left:10px;display:inline-flex;align-items:center;gap:8px;font-family:"IBM Plex Mono",ui-monospace,Menlo,Consolas,monospace;font-size:10.5px;letter-spacing:.05em;text-transform:uppercase;color:var(--ftl-grayblue);background:#fff;border:1px dashed #dcdce0;border-radius:999px;padding:3px 12px 3px 9px;position:relative;z-index:2} .ftl-gap span::before{content:"";display:inline-block;width:7px;height:7px;border-radius:50%;background:var(--ftl-accent);box-shadow:0 0 0 3px #fff} .ftl-total{margin-top:6px;background:var(--ftl-ink);color:#fff;border-radius:var(--ftl-radius);padding:18px 20px;box-shadow:0 8px 24px -12px rgba(23,24,30,.4)} .ftl-total-row{display:flex;justify-content:space-between;align-items:baseline;gap:14px;font-size:.82em;line-height:1.7;color:#d6d6da} .ftl-total-row code{font-family:"IBM Plex Mono",ui-monospace,Menlo,Consolas,monospace;background:transparent;border:0;color:#fff;font-weight:600;padding:0;font-size:1em} .ftl-total-row.ftl-grand{margin-top:10px;padding-top:12px;border-top:1px solid #34353d;font-size:.96em;color:#fff;font-weight:700} .ftl-total-row.ftl-grand code{color:var(--ftl-accent);font-size:1.08em;font-weight:700} .ftl-total-row.ftl-zero{margin-top:8px} .ftl-total-row.ftl-zero code{color:var(--ftl-accent);font-weight:700} @media (max-width:560px){.ftl-row{grid-template-columns:52px minmax(0,1fr);row-gap:8px}.ftl-tl ol::before{left:25px}.ftl-cal{width:52px}.ftl-cal-d{font-size:19px}.ftl-money{grid-column:2;text-align:left;align-self:start;padding-top:0}.ftl-money .ftl-tag{display:inline;margin:0 0 0 8px}} Jan2323 JanFirst call with a law firm to set up the company. The clock and the hourly billing start.Feb55 FebI sign the mandate and send my ID. Drafting begins.Feb1818 FebThe structure is set: PlentyLabs UG & Co. KG, technically two companies. The name is a saga of its own.about 1 month of draftingMar66 MarIncorporation documents ready.Mar1717 MarDocuments approved. The hunt for a notary begins.7 days for the appointmentMar2424 MarNotary in Berlin reads the deeds aloud and certifies that I am who I say I am.€1,575.24Notary feesMar2525 MarI pay in €2,000.00 of share capital. Money I cannot touch; it has to stay there.€2,000.00Locked, not a feeMar2626 MarThe register court demands a fee advance.€300.00Court advance17 days after the notaryApr1010 AprFirst company entered in the commercial register.1 week moreApr1717 AprSecond company entered.€260.00Register, 200 + 60Apr2020 AprI ask the firm I already pay to handle the tax registration too.2.5 weeks just to startMay66 MayBefore the tax work can begin, a fresh engagement is required: proposal, power of attorney, ID checks, per company.€630.00Tax registration quoteMay2828 MayThe incorporation legal bill lands.€4,462.50Legal feesMay2929 MayTax questionnaires submitted. I request standard VAT and a VAT ID, urgently.Jun33 JunFirst bill from the accounting software.€426.97Accounting softwareJun99 JunI am told the VAT ID will arrive by post. A letter.Jun24today24 Jun, todaySeven weeks since the tax firm, almost four weeks since the questionnaires. No VAT ID. No invoice sent.Billed by everyone else€7,654.71Share capital I cannot touch€2,000.00Total gone€9,654.71Invoices I have managed to send0 Everyone in this story could invoice me. I am the only one who can't invoice anyone. ## "But you can invoice your German clients" The clients abroad need a VAT ID for reverse charge, and that is exactly the one I am still waiting for. My German clients I could bill today. But a domestic invoice now would have to be reissued the moment the VAT ID arrives. Bill now, bill again later, for no reason. So those wait too. ## This should have been a web form Fill it in, pay a fee, get your company and your VAT ID in a week. Estonia does it. The UK registers a company in a day, online, for the price of a dinner. There is no law of nature that says incorporation has to take five months and arrive by post. Germany has built a process that chains one dependency to the next, puts a fee on each, and lets a founder run up legal bills, notary bills, court fees, tax retainers, and software subscriptions on zero revenue, all before granting the one permission a company exists for: the right to send an invoice. If you ask the government, the reason is trust: the notary, the capital, the registers, the endless checks, all there to keep bad actors out. This is the same machine that did not catch Wirecard, a two-billion-euro scam. It does, somehow, generate enough friction to scare new founders out of the country. And no, I could not just leave instead. My first company, Freshflow, is valuable enough that walking out of Germany would trigger a massive six-figure exit tax, on gains I have not even realised, purely for the privilege of leaving. But that is a story for another post. This is a country taxing ambition through the roof before you've earned a cent, then wondering why the ambitious leave. ## Bonus round: my company name was "too generic" Have you heard of Apple? A piece of fruit, and one of the most valuable brands ever built. That name would never have been approved in Germany. Naming a company is hard. It is the word everyone who touches your work will remember. After months of turning it over, I found one I could stand behind, a name that says what I believe software should be. (That belief will be its own post, soon.) Distinctive, I thought. The kind of name you do not forget. Plenty. "No," said the lawyer. German company names have to be distinctive, and "Plenty" is a plain English word. Berlin would reject it. "Plenty Group?" Two plain words. "Plenty Labs?" "Labs" is a plain word too. "Plenty.is?" A generic word with a domain on the end is still a generic word, and there was case law to prove it. The suggestions were worse: stick my surname on the front, Paolino Plenty Labs. Or a prefix, PG Plenty Germany. Or make up a fantasy word. Is Plenty. Its Plenty. IsPlenty. ItsPlenty. Rejected, all of it. Fine. They wanted a meaningless word; I gave them one. Plenty Labs, minus the space: PlentyLabs. Approved. A name that started out of spite. Weeks of correspondence, resolved by removing a space. A rule that does not reward clarity. It rewards nonsense. ## Postscript: why a UG and Co. KG, two companies? Why does a one-person business need two companies? Because the simple version is worse, and because I am building it into something bigger. The simplest setup is a sole proprietorship. Thirty euros, no capital, done in an afternoon. It also makes me personally liable for everything. A client sues? They are not suing a company. They are suing me. My savings, my apartment, my name. So I wanted real limited liability, which means a company. And for one person, the cleanest company turns out not to be one company. It is a KG, a partnership that does the work, with a tiny UG standing in as the partner that carries the liability. Strange, but standard. You probably have seen "GmbH & Co. KG" on German companies a hundred times without wondering why. This is why. It is taxed the sane way, too. The partnership's profit is taxed once, as my income, since I am the one who ends up with it. A plain UG would tax the company first, then tax me again when I paid myself. Why a UG and not the famous GmbH? A GmbH wants 25,000 euros sitting in a bank account before it is allowed to exist. The UG lets you start with almost nothing, on one condition: lock away a quarter of every year's profit until the reserve reaches 25,000, then convert to a GmbH. The 25,000 does not go away. Germany just takes it in instalments. Which leaves the only real question. Why 25,000 at all? It is my company and my risk. If I want to start with nothing, that is my call, not a toll the state collects before it will let me try. And the cheap door has a price of its own: to some clients, "UG" reads as "not serious," and they would rather deal with a GmbH. The structure built to let me in quietly marks me for using it. This is also why Chat with Work, my fully private Work AI, is still free: I cannot invoice you yet! Try it before that changes. --- ### RubyLLM 1.16: Concurrent Tool Execution, Rails-Style Instrumentation, and api_base for Every Provider URL: https://paolino.me/rubyllm-1-16/ Date: 2026-06-09 When you first reach for an LLM library, the only question is whether it works. Can it call the model, parse the response, run a tool. Once your app is actually in production, the questions change. Is it fast? Can I see what it's doing when something goes wrong? Can I send its traffic through my own infrastructure instead of straight out to the provider? I released [RubyLLM](https://rubyllm.com) 1.16 today. It answers these production questions. The three headline features are about speed, visibility, and control: tools that run concurrently, structured events for everything RubyLLM does, and a configurable base URL for every native provider. None of them change how you write your app. All of them matter the moment real traffic shows up. ## Tools That Run Concurrently When a model returns several tool calls in one response, it's telling you those calls are independent. Get the weather, look up the stock price, fetch the exchange rate. The model didn't ask for them in order. It asked for all of them. RubyLLM has always run them one at a time. For tools that are CPU-bound that's fine, but most tools aren't. Most tools are an HTTP call, a database query, another LLM request. They spend their time waiting. Running three waits back to back, when you could have waited for all three at once, it's time your users can't get back. 1.16 runs them together. Turn it on for every chat from one place: ```ruby RubyLLM.configure do |config| config.tool_concurrency = true # :threads, :fibers, true, or false end ``` `true` uses `:threads` and needs no dependencies. If you'd rather not pay for a thread per tool, `:fibers` mode uses the `async` gem and gets my recommendation for I/O bound operations. Check out my previous posts on [why I think async is the future of Ruby](/async-ruby-is-the-future/) and [what Ruby concurrency actually does](/ruby-concurrency-what-actually-happens/). When one conversation needs different behaviour than the rest, override it per chat: ```ruby chat.with_tools(Weather, StockPrice, Currency, concurrency: :fibers) chat.with_tools(Weather, StockPrice, concurrency: false) ``` Inside Rails, each concurrent tool call runs wrapped in the Rails executor, so connection pools, `CurrentAttributes`, and reloading behave the way the rest of your app does. You don't think about it. It just works. And concurrency doesn't make your UI wait for the slowest tool. Each result is added back to the conversation the moment that tool finishes, in completion order, so your streaming callbacks see results land as they happen. RubyLLM still gathers every result before going back to the model, but your users watch progress instead of a spinner. ## Instrumentation Without Monkey Patching You can't operate what you can't see. Some libraries popped up to add instrumentation to RubyLLM, but they monkey patch us. That's unnecessary maintenance burden. RubyLLM 1.16 emits structured events for the work it does, the same way Rails does. In a Rails app they flow through `ActiveSupport::Notifications` automatically, and you subscribe the way you'd subscribe to any framework event: ```ruby # config/initializers/ruby_llm_instrumentation.rb ActiveSupport::Notifications.subscribe('chat.ruby_llm') do |_name, _start, _finish, _id, payload| Rails.logger.info( provider: payload[:provider], model: payload[:model], input_tokens: payload[:input_tokens], output_tokens: payload[:output_tokens] ) end ``` Outside Rails, point `config.instrumenter` at anything that responds to `instrument(name, payload) { ... }` and wire it into OpenTelemetry, StatsD, or your own logger. The events cover the whole surface: HTTP requests, chat completions, tool calls, embeddings, and model registry refreshes, each carrying the provider, model, token usage, and the Ruby objects an observability adapter needs. Those payloads can hold message content, tool arguments, and full provider responses, which is exactly the sensitive data you don't want sprayed into logs by accident. So log or export those fields only when your policy allows it. The [Instrumentation guide](https://rubyllm.com/instrumentation) has the full payload reference. ## A Base URL for Every Native Provider In production, your AI traffic rarely goes straight to the provider. It goes through a gateway that handles auth, a proxy that enforces rate limits, a private endpoint inside your network. RubyLLM let you point most providers at a custom base URL already. 1.16 fills the last gaps, so now every native provider has one: ```ruby RubyLLM.configure do |config| config.bedrock_api_base = ENV['BEDROCK_API_BASE'] config.mistral_api_base = ENV['MISTRAL_API_BASE'] config.perplexity_api_base = ENV['PERPLEXITY_API_BASE'] config.vertexai_api_base = ENV['VERTEXAI_API_BASE'] config.xai_api_base = ENV['XAI_API_BASE'] end ``` Together with the bases already there for OpenAI, Anthropic, Gemini, DeepSeek, OpenRouter, Azure, Ollama, and GPUStack, you can front any provider with your own infrastructure. Each override falls back to the provider's default when unset, so nothing you already have changes. While I was in the HTTP layer, I made the Faraday adapter configurable too: ```ruby RubyLLM.configure do |config| config.faraday_adapter = :async_http # or :typhoeus, :net_http, :httpx, etc. end ``` It defaults to `Net::HTTP`, so nothing changes unless you ask. Reach for it when you want connection pooling, HTTP/2, or whatever adapter your app already standardizes on. ## Transcription Words `Transcription` now exposes word-level timing when the provider returns it, so you can build word-by-word highlighting on top of OpenAI's verbose transcriptions: ```ruby transcription = RubyLLM.transcribe("interview.mp3", model: "whisper-1") transcription.words # => [{ word:, start:, end: }, ...] ``` ## Getting Ready for 2.0 Deprecation warnings are now yours to control: ```ruby RubyLLM.configure do |config| config.deprecation_behavior = :warn # :warn (default), :silence, or :raise end ``` Set `:raise` in your test environment and a deprecated path fails the build the moment something hits it. That's the cheapest possible way to be ready before those paths disappear in 2.0, instead of finding out on upgrade day. ## Fixes and the Model Registry A release this size carries a long tail of fixes. The ones worth calling out: Anthropic's "prompt is too long" now raises `ContextLengthExceededError` so you can rescue it like any other context-length error, streaming parallel tool calls accumulate correctly, Bedrock reasoning streams properly, and Gemini function calls and inline images follow the spec. Active Storage handling in Rails got more careful about pending uploads, load order, and text attachments. And when configuration or a model lookup goes wrong, the error now tells you what happened and how to fix it. The model registry is refreshed with the latest models, capabilities, and pricing. One fix there is worth a sentence: models.dev started shipping partial release dates like `2025-09` and `2025`, RubyLLM was turning those into invalid timestamps, and model loading broke. 1.16 normalizes them to real dates so the registry keeps loading. The [full release notes](https://github.com/crmne/ruby_llm/releases/tag/1.16.0) have the complete list. ## Use It ```ruby gem 'ruby_llm', '~> 1.16' ``` ```bash bundle update ruby_llm ``` It's backwards compatible. Concurrency is opt-in, instrumentation stays inert until you subscribe, and every new `*_api_base` falls back to the provider default. Nothing you've built changes until you decide to reach for it. The boring infrastructure is just there now, waiting for the day your app stops being a demo and starts being production. --- ### Engineering Is Not Dead, Because Accountability Isn't URL: https://paolino.me/engineering-is-not-dead/ Date: 2026-05-22 A lot of people have developed a gag reflex against anything touched by AI. I understand where that comes from. There is a lot of slop, maintainers are tired of reviewing code from people who do not understand it, and people are tired of [predictable cadence](https://x.com/jorgemanru/status/2053183727514091820). We're also heading toward a version of the future where all code will be generated. The models are good enough that for a lot of work, especially the boring repetitive kind, typing everything by hand makes very little sense. You can describe what you want, steer the model, ask for changes, review the output, and get to a working implementation much faster than before. That caused some people to jump from "models can generate code" to "engineering is dead". That is wrong. ## Code Generation Is Not Engineering Engineering is not the act of producing text that happens to run or compile. Engineering is deciding what should exist. Understanding the constraints. Knowing what can go wrong. Making trade-offs. Reviewing the result. Being responsible for what happens after you ship it. The model can write the code. Most of it. Maybe all of it. But the model is not accountable. You are. If a generated library has a security issue, people will not open an issue against the model. They will open it against you. If a generated feature behaves badly in production, your reputation will suffer. If the code is impossible to maintain six months later, the model is not at fault. You are. This is why your engineering skills matter more than ever. Since you are not spending most of your time typing, you can focus on what really matters. ## So How Do You Tell? The discussion around AI-generated code is confused because people focus too much on the origin. Lots of good code will be touched by LLMs. So will code from your favorite programmers. So will lots of bad code. The involvement of AI tells you very little by itself. The real distinction is whether the result is owned or not. It is the care, attention, review, testing, product design, and engineering the author put into it. You signal it by producing high-quality output and being accountable for it. By showing up. By fixing bugs. By knowing your own code inside and out. By making it clear that there is a person behind the work who understands the result and accepts responsibility for it. That takes care, taste, engineering skill, and genuine human effort. This goes both ways. The same skills are needed by people evaluating code and products. It is not enough to ask whether AI was involved. You have to look at the result, the behavior, the tests, the edge cases, the maintenance story, and the author’s ability to own the thing. Engineering is not dead, because accountability isn't. --- ### Production Experience Cannot Be Hallucinated URL: https://paolino.me/production-experience-cannot-be-hallucinated/ Date: 2026-05-13 I paid five dollars to read a [Medium article](https://mrrazahussain.medium.com/the-rails-llm-stack-is-finally-ready-for-production-here-is-what-i-learned-shipping-it-ff9d20298c5c) about [my own free, open source library](https://rubyllm.com). It was sold as hard-won production experience. It was fabricated. The first code sample used `RubyLLM.client`, which does not exist. It called `client.chat(messages: ...)`, which does not exist. Then it invented `RubyLLM::StreamInterrupted`, `RubyLLM::APIError`, and a `stream: proc` API that RubyLLM has never had. The problem was not merely wrong information. Wrong information can be corrected. This was sold as experience with RubyLLM in production, which is a much more valuable claim. AI slop is not just filling the web with [predictable cadence](https://x.com/jorgemanru/status/2053183727514091820). It is fabricating experience. It is letting people skip the work, skip the scar tissue, and still write in the voice of someone who has been there. In open source, that turns into a tax. Maintainers build the thing, write the docs, publish the source, keep the examples working, answer the issues, and then have to police hallucinated articles about their own projects before users start debugging ghosts. ## The Four Magic Words in Tech Production. Scale. Security. Reliability. In the tech world, attach one of these words to a claim and it immediately becomes true. "This does not scale" can kill a project before anyone measures it. "This is not production ready" can sabotage it without a single deploy. So when an article says "what broke in production", it is not just offering advice. It is claiming experience, and experience cannot be hallucinated. [The first version](/assets/receipts/2026-05-13-production-experience-medium-original-article-2026-05-12.md) opened by saying the author had spent three weeks on the wrong side of the problem before getting something stable in production. That is a powerful claim. It tells the reader to relax and inherit the author's scars. There were no scars. The author had not even run the first example. This is why fake experience is so dangerous. Bad code fails fast. Fake experience lingers. It gets quoted. It gets summarized. It gets used in meetings by people who do not know enough yet to see the hollow center. The recipe is familiar. Streaming failures. Token budgets. Provider fallback. Turbo Streams. Redis circuit breakers. nginx buffering. Load testing. They sit near "LLM production" in the LLM training data. Arrange them with enough confidence and the result smells real. Production experience is not a smell. It is a thing that happened, and none of these things happened. ## What Actually Happened Here is the short version. Most articles about RubyLLM are good. Since it became popular, I have seen a few confident guides from people who clearly had not run the code. Usually they disappear into LinkedIn or search results. This one made the pattern impossible to ignore. [I called it out](/assets/receipts/2026-05-13-production-experience-maintainer-first-correction.png): > Author of RubyLLM here. > > The very first example does not work. > > The article is not merely wrong in a few places. It is fabricated. > > ... [The author replied](/assets/receipts/2026-05-13-production-experience-author-admission.png): > You were right. > > The code in the original article was not verified against the actual gem. `RubyLLM.client`, `RubyLLM::StreamInterrupted`, `RubyLLM::APIError`, `stream: proc` -- none of it exists. You caught every fabrication accurately. > > I've replaced the article entirely. The new version has been verified against your documentation and source. The fake "production experience" framing is gone. It's now an honest documentation-based guide with a correction notice at the top explaining what happened. "I've replaced the article entirely." It was a long article. The completely rewritten replacement appeared in a few minutes. The fake method names were replaced with real ones, but the posture stayed the same: "RubyLLM in production", "what tutorials skip", "streaming failures", "provider fallback", "token budgets." The method names got real. The experience didn't. The new version claimed Puma restarts produce neat RubyLLM streaming errors. They do not. If the worker dies, the Ruby process running the call is gone. It suggested deleting old persisted chat messages as context management. That is destroying conversation history. It described fallback by throwing away the chat and asking another provider the last prompt as a fresh question. That is not conversation fallback. It confused HTTP/SSE buffering with Turbo Streams over ActionCable. Not battle scars. Guesses presented as authority. [I called the second version what it was: phony](/assets/receipts/2026-05-13-production-experience-maintainer-second-correction.png). [The author then hid responses](/assets/receipts/2026-05-13-production-experience-responses-hidden.png) while keeping the article up. I reported the article to Medium and contacted the publication that promoted it with the fabricated APIs, the author's admission, and the hidden corrections. To their credit, the editor replied quickly, apologized, and removed it from the publication. But only the author can take down the original Medium article, so the piece remained available without the maintainer corrections visible next to it. ## Do Not Counterfeit Experience Please do write about your favourite software. Critique it too. Tell us maintainers where the API is wrong, the docs are bad, the abstraction leaks. Preferably in an issue so we can actually see it. That feedback is gold. But do not counterfeit experience. If you're using The Four Magic Words in Tech, the bar is even higher. And if you run a technical publication, please at least check the first example. --- ### RubyLLM 1.15: Image Editing, Cost Tracking and Less Tool Boilerplate URL: https://paolino.me/rubyllm-1-15/ Date: 2026-05-07 I released [RubyLLM](https://rubyllm.com) 1.15 today. It ships image editing, cost tracking, cleaner token accounting, inferred tool parameters, additive callbacks, and Rails fixes. The theme is simple: stop making me write glue code. If the computer can infer it, RubyLLM should infer it. If a provider reports usage, RubyLLM should turn it into cost. If Rails already has a blob, RubyLLM should not download it and upload it again. ## Image Editing `RubyLLM.paint` could already generate images: ```ruby image = RubyLLM.paint("A watercolor robot holding a Ruby gem") ``` Now `with:` turns it into an image edit: ```ruby image = RubyLLM.paint( "Turn the logo green and keep the background transparent", model: "gpt-image-1", with: "logo.png" ) ``` Same method, same attachment shape. The source can be a path, a URL, an IO-like object, or an Active Storage attachment. Multiple source images work too: ```ruby image = RubyLLM.paint( "Combine these references into a postcard illustration", model: "gpt-image-1", with: ["person.png", "style-reference.png"] ) ``` And if you need to constrain the edit, pass a mask: ```ruby image = RubyLLM.paint( "Replace only the background with a sunset sky", model: "gpt-image-1", with: "portrait.png", mask: "portrait-mask.png" ) ``` That's it. `paint` paints. Sometimes from scratch, sometimes from an existing image. ## Cost Tracking RubyLLM has tracked tokens since 1.0. But "this used 18,432 tokens" is only half the answer. The next question is always: how much did that cost? Calculating that was never hard. Take the input tokens, output tokens, cached tokens, maybe reasoning tokens. The pricing is already in RubyLLM's model registry. Multiply by the per-million rate. But why should every app have to write that code? RubyLLM already has the usage. RubyLLM already knows the model. RubyLLM already ships the model registry. So now it does the boring math for you. Now you can ask: ```ruby response = chat.ask("Summarize Ruby's object model.") response.cost.total chat.cost.total agent.cost.total ``` Same for images: ```ruby image = RubyLLM.paint("A small watercolor robot", model: "gpt-image-1") image.tokens.input image.tokens.output image.cost.input image.cost.output image.cost.total ``` If RubyLLM does not have pricing for part of the usage, the cost is `nil`. Better no answer than a fake one. A chat with ten messages can tell you the total. An agent can tell you the total. A generated image can tell you the total. No more handrolled sums. ## Token Counts That Mean What They Say Prompt caching made token counts messy. Some providers include cache reads in prompt tokens. Some report cache creation separately. Some don't. If you multiply the wrong number by the wrong price, your cost tracking is wrong before it starts. So 1.15 separates the different kinds of tokens before exposing them: ```ruby response.tokens.input # standard input tokens response.tokens.output # billable output tokens response.tokens.cache_read # prompt cache reads response.tokens.cache_write # prompt cache writes ``` `tokens.input` now means normal input tokens. Cache reads and cache writes are separate. `tokens.output` always mean billable output tokens. The old top-level helpers still work. New code should use `response.tokens.*`. No new Rails migration is required if you already ran the 1.9 token migration. If you display token counts directly, read the [1.15 upgrade notes](https://rubyllm.com/upgrading/#upgrade-to-115). ## Less Tool Boilerplate Tools in RubyLLM are Ruby classes. But for very simple tools, RubyLLM still made you repeat yourself: ```ruby class Weather 1.15' ``` Then: ```bash bundle update ruby_llm ``` Full release notes on [GitHub](https://github.com/crmne/ruby_llm/releases/tag/1.15.0). --- ### kamal-backup: Scheduled Rails Backups for Kamal Apps URL: https://paolino.me/kamal-backup/ Date: 2026-05-05 _Update, August 2026: [kamal-backup 1.0 is out](/kamal-backup-1-0/). The configuration below describes the early releases and is no longer current. Use the [1.0 documentation](https://kamal-backup.dev) when setting it up today._ I released [kamal-backup](https://kamal-backup.dev) today. I run [Chat with Work][] on Kamal, and I needed backups. There are already Kamal accessories for database backups. None of them also back up Active Storage. None use restic, so encryption, deduplication, and repository checks are on you. None ship a CLI with restores and drills. None produce evidence you can hand a security reviewer. So I built one. ## A gem and a Docker image `kamal-backup` is two pieces: a Ruby gem you add to your Rails app, and a Docker image you boot as a Kamal accessory. They point at a restic repository you bring yourself. The gem is your CLI. Local commands run directly on your machine using restic. Production-side commands shell out through Kamal into the accessory. The same `kamal-backup` binary covers setup (`init`, `validate`), on-demand operations (`backup`, `list`, `check`), data movement (`restore local`, `restore production`), verification (`drill local`, `drill production`), and audit (`evidence`). The Docker image (`ghcr.io/crmne/kamal-backup`) ships with `restic`, `pg_dump`, `mariadb-dump`/`mysqldump`, and `sqlite3` baked in. The default container command is `kamal-backup schedule`, a loop that fires every `backup_schedule_seconds` and writes one database snapshot and one Active Storage file snapshot per run. The restic repository is where the encrypted snapshots end up: S3-compatible object storage, a restic REST server, or a filesystem path. `kamal-backup` points at it. It doesn't run it for you. ## Why restic I didn't want to invent a backup format, and I didn't want to bolt encryption and deduplication onto shell scripts. Restic does what I needed: - encrypted repositories by default; - a tag system, so the database dump and the Active Storage tree from the same run share a `run:` and pair up at restore time; - deduplication across runs, so a year of daily backups doesn't grow linearly; - `restic forget --prune` for retention; - `restic check` for repository health; - S3-compatible storage, a restic REST server, or a local filesystem path, so you host the repository wherever fits. It's a single binary that drops cleanly into a Docker image, alongside the database client tools. Nothing extra to install on the Rails host. `kamal-backup` is the Rails- and Kamal-shaped layer on top, and restic does the cryptography, the storage, and the integrity checks. ## Setting it up Add the gem in development: ```ruby # Gemfile group :development do gem "kamal-backup" end ``` Run `init`. It creates `config/kamal-backup.yml` and prints an accessory block you paste into your Kamal deploy config: ```sh bundle install bundle exec kamal-backup init ``` `config/kamal-backup.yml` holds the backup settings: ```yaml accessory: backup app_name: chatwithwork database_adapter: postgres database_url: postgres://chatwithwork@chatwithwork-db:5432/chatwithwork_production backup_paths: - /data/storage restic_repository: s3:https://s3.example.com/chatwithwork-backups restic_init_if_missing: true backup_schedule_seconds: 86400 ``` Kamal mounts that file read-only into the accessory, so the accessory block in `config/deploy.yml` stays small. Only secrets live in `env`: ```yaml accessories: backup: image: ghcr.io/crmne/kamal-backup:latest host: chatwithwork.com files: - config/kamal-backup.yml:/app/config/kamal-backup.yml:ro env: secret: - PGPASSWORD - RESTIC_PASSWORD - AWS_ACCESS_KEY_ID - AWS_SECRET_ACCESS_KEY volumes: - "chatwithwork_storage:/data/storage:ro" - "chatwithwork_backup_state:/var/lib/kamal-backup" ``` Validate, boot, and watch the logs: ```sh bundle exec kamal-backup validate bin/kamal accessory boot backup bin/kamal accessory logs backup ``` `validate` catches missing required settings before the accessory has to be running. Once it's up, the container loops on `kamal-backup schedule`. Then run the first backup and print evidence: ```sh bundle exec kamal-backup backup bundle exec kamal-backup list bundle exec kamal-backup evidence ``` No cron glue. No separate backup host. No "remember to install restic on production." The accessory image already has it. ## Rails data, not just a database dump A Rails app has two things worth backing up: the database, and file-backed Active Storage. `kamal-backup` handles both. Postgres uses `pg_dump`. MySQL and MariaDB use `mariadb-dump` or `mysqldump`. SQLite uses `sqlite3 .backup`. File-backed Active Storage uses `restic backup` from mounted volumes. Each run writes one database snapshot and one file snapshot, both tagged with `app:`, `type:database` or `type:files`, and the same `run:`. You pair them at restore time using that timestamp. If your app stores Active Storage blobs directly in S3, there's no mounted path for `backup_paths` to capture. `kamal-backup` still covers the database. The S3 side is on your bucket lifecycle and replication settings. ## Restores are part of the product The backup script is the easy part. The restore path is where most setups fail. So `kamal-backup` ships with restore commands: ```sh bundle exec kamal-backup restore local bundle exec kamal-backup restore production ``` `restore local` pulls a production backup down to your laptop. Useful when you want to inspect real data, reproduce a production bug, or prove the backup actually comes back. `restore production` prompts before it overwrites anything. ## Restore drills The command I care about most is `drill`. ```sh bundle exec kamal-backup drill local \ --check "bin/rails runner 'puts User.count'" ``` A drill means: restore, check, record the result. Two modes: - `drill local` restores onto your machine and runs an optional check. - `drill production` restores into scratch production-side targets, never the live database. That second one matters. For Postgres and MySQL, you give it a scratch database. For SQLite, a scratch file path. For Active Storage, a scratch restore directory. The drill uses production infrastructure, without pointing at live production. That's the difference between "the backup ran" and "we restored the latest production snapshot into a scratch target on April 30, ran this check, and it passed." ## Evidence for reviews I went through a security review for [Chat with Work][] this year. The questions were fair: - What's being backed up? - Where does it go? - Is it encrypted? - When did the last backup run? - When did the last repository check run? - When was the last restore drill? - Can you prove all of that without leaking secrets? `kamal-backup evidence` prints redacted JSON: current backup settings, latest snapshots, latest restic check, latest restore drill, retention settings, tool versions. ```sh bundle exec kamal-backup evidence ``` Secrets are redacted. The output is meant to land in an internal ops record or a CASA packet. Not a screenshot of a green cron job. Actual evidence. ## Try it ```ruby # Gemfile gem "kamal-backup" ``` Docs at [kamal-backup.dev](https://kamal-backup.dev), source on [GitHub](https://github.com/crmne/kamal-backup). [Chat with Work]: https://chatwithwork.com --- ### Ruby Concurrency: What Actually Happens URL: https://paolino.me/ruby-concurrency-what-actually-happens/ Date: 2026-04-28 Since I wrote about [async Ruby][async-article] and [patched Solid Queue to support fibers][sq-article], people keep asking the same questions. What happens when a fiber blocks? Don't you still need threads? What about database transactions? What about Ractors? This post answers all of it. From the ground up. ## The four primitives Ruby gives you four concurrency primitives: processes, threads, fibers, and Ractors. They nest. Every process has an implicit "main Ractor" where your code runs by default, so you never have to think about Ractors unless you explicitly create one. Without Ractors, the hierarchy is simply process -- threads -- fibers. With Ractors, it becomes: graph TD P[Process] --> R1["Ractor 1 (GVL 1)"] P --> R2["Ractor 2 (GVL 2)"] R1 --> T1[Thread 1] R1 --> T2[Thread 2] R2 --> T3[Thread 3] T1 --> F1[Fiber A] T1 --> F2[Fiber B] T2 --> F3[Fiber C] T3 --> F4[Fiber D] T3 --> F5[Fiber E] style P fill:#4a90a4,color:#fff style R1 fill:#c084fc,color:#fff style R2 fill:#c084fc,color:#fff style T1 fill:#7fb069,color:#fff style T2 fill:#7fb069,color:#fff style T3 fill:#7fb069,color:#fff style F1 fill:#e8a87c,color:#fff style F2 fill:#e8a87c,color:#fff style F3 fill:#e8a87c,color:#fff style F4 fill:#e8a87c,color:#fff style F5 fill:#e8a87c,color:#fff Think of your computer as an office building. **Processes** are fully isolated: separate offices, each with its own locked door, furniture, and files. Each process has its own memory, its own Ruby VM, and its own GVL. When you run Puma with 3 workers, you get 3 processes. They can't corrupt each other's state because they don't share memory. The OS schedules them independently. The cost: each one loads your entire application into memory. **Ractors** sit between processes and threads: offices that share a mailroom but not their filing cabinets. Each Ractor has its own GVL, so threads in different Ractors can execute Ruby code truly in parallel, but they can only pass notes to each other -- no shared mutable objects. You communicate via message passing, copying or moving data between them. Every Ruby process has a "main Ractor" where all your code runs by default. Creating additional Ractors is opt-in. **Threads** live inside a process and share its memory: workers sharing the same office, accessing the same filing cabinets, coordinating to avoid collisions. In CRuby, they are native threads, with the GVL deciding which one can execute Ruby code at a time. You don't control when Ruby switches between them. The GVL releases during I/O, so two threads can wait on two different network calls simultaneously, but they can't crunch numbers at the same time. **Fibers** live inside a thread and are cooperatively scheduled: multiple tasks juggled by one worker at their desk. When they're waiting for something -- a phone call, a fax, a response -- they set it aside and pick up the next task. A fiber runs until it explicitly yields. When it hits I/O -- a network call, a database query, reading a file -- it yields to the reactor, and another fiber picks up. No OS thread context switch for the fiber itself, no preemption. One thread can run thousands of fibers. Here's what that means for cost: | Metric | Process | Ractor | Thread | Fiber | |---|---|---|---|---| | Memory | full app copy | ~thread + Ractor state | ~8MB virtual stack reservation | ~4KB initial virtual stack, grows as needed | | Creation time | ~ms | ~80μs | ~80μs | ~3μs | | Context switch | kernel | kernel (threads within) | ~1.3μs (kernel) | ~0.1μs (userspace) | | Isolation | Full (own memory) | Share-nothing (messages) | Shared memory | Shared thread | | Parallelism | Yes | Yes (own GVL) | No (shared GVL) | No | | I/O concurrency | Yes | Yes | Yes | Yes | | Rails compatible | Yes | No | Yes | Yes | Creation and switching benchmarks are from [Samuel Williams' fiber-vs-thread performance comparison][fiber-bench]. Fibers create 20x faster and switch 10x faster than threads. The memory row is about virtual address space reserved by the platform/runtime, not resident memory. The benchmark reports actual RSS, where the gap is much smaller than the virtual stack numbers suggest. But the shape is still real: each thread is a kernel object with scheduler state and a stack reservation, while each fiber is scheduled in userspace. Ractors give you parallelism too, but can't run Rails. Everything is a tradeoff. ## How scheduling works This is where most of the confusion lives. Let me show you what actually happens. ### Thread scheduling CRuby threads are native threads, but the GVL decides which one can run Ruby code. Your code has no say. A thread can be paused mid-calculation, mid-assignment, mid-anything. sequenceDiagram participant VM as CRuby / OS participant T1 as Thread 1 participant T2 as Thread 2 participant LLM as LLM API VM->>T1: Run T1->>LLM: Send request Note over T1: Blocks in I/O (parked) VM->>T2: Run T2->>LLM: Send request Note over T2: Blocks in I/O (parked) Note over VM: Both threads parked LLM-->>T1: Response ready LLM-->>T2: Response ready VM->>T1: Wake and run Note over T1: Processing response VM->>VM: Time slice expired VM->>T2: Preempt T1, run T2 Note over T2: Processing response VM->>VM: Time slice expired VM->>T1: Resume T1 Note over T1: Finish response VM->>T2: Resume T2 Note over T2: Finish response CRuby can switch runnable threads on a time slice, but a thread blocked in I/O is parked until the socket is ready. That part matters: threads do not spin uselessly while waiting for tokens. The switch happens when a thread is runnable -- including in the middle of response processing, object allocation, assignment, or any other Ruby code. For two threads doing I/O, this works fine. The overhead is noise. For 200 threads mostly waiting for LLM tokens, the problem is the one-operation-per-thread shape: 200 kernel threads, 200 stack reservations, 200 scheduler entries, and usually 200 copies of whatever per-thread application resources the worker holds. This is also why a worker limit means different things in Solid Queue's current thread mode and in the fiber mode from my patch. `threads: 25` is both "run 25 jobs at once" and "create 25 kernel threads." If all 25 jobs are streaming tokens, job 26 waits. `fibers: 250` is mostly an admission limit for the reactor: run up to 250 jobs as fibers on the same thread, park the ones waiting on I/O, and resume them when ready. You still need limits because APIs, sockets, memory, and databases have limits. But the cap is no longer tied to one kernel thread per job. ### Cooperative scheduling (fibers) Fibers switch only when they choose to. In practice, the [async][] gem makes this automatic: your code yields at I/O boundaries without you writing anything special. sequenceDiagram participant R as Reactor participant F1 as Fiber 1 participant F2 as Fiber 2 participant LLM as LLM API R->>F1: Run F1->>LLM: Send request Note over F1: Yields (I/O wait) R->>F2: Run F2->>LLM: Send request Note over F2: Yields (I/O wait) Note over R: Both waiting, reactor sleeps LLM-->>F1: Response ready R->>F1: Resume immediately Note over F1: Processes response F1->>R: Done LLM-->>F2: Response ready R->>F2: Resume immediately Note over F2: Processes response F2->>R: Done No OS thread context switch per fiber. No timer-based preemption between fibers. When a fiber yields, the reactor checks which fibers have I/O ready and resumes them. When nothing is ready, the reactor sleeps in the OS until something is. The kernel still does the I/O readiness work; Ruby just avoids one kernel thread per wait. ## The GVL: why threads and fibers are more similar than you think This is the part that makes thread-based Ruby less different from fiber-based Ruby than it first looks. The GVL means only one thread can execute Ruby code at a time. Threads run in parallel only during I/O, when the GVL is released. So if your workload is I/O-bound -- HTTP calls, database queries, LLM streaming -- threads give you I/O concurrency, not parallelism. Fibers give you the same I/O concurrency. One fiber yields at I/O, another picks up. The difference: fibers do it without kernel thread overhead, without the memory cost of a thread stack, and without making job concurrency itself imply one worker thread or one database slot per job. If threads only help with I/O anyway, why pay their overhead? There is one case where threads win: CPU-bound work that releases the GVL. Some C extensions (image processing, cryptographic operations) release the GVL while doing heavy computation. Multiple threads can then run those C extensions in parallel. Fibers can't do that. They share a thread. For actual Ruby-level CPU parallelism, you need processes or [Ractors](#why-not-ractors). Processes are production-ready and Rails-compatible. Ractors are lighter than processes, but still experimental. ## What happens when a fiber hits I/O This is the happy path and the most common question. ```ruby # Inside a fiber response = Net::HTTP.get(URI("https://api.example.com/v1/completions")) ``` Here's the full chain: 1. `Net::HTTP` opens a socket and sends the request 2. The socket isn't readable yet (the server hasn't responded) 3. Ruby calls `rb_io_wait` on the socket 4. The async gem's `Fiber.scheduler` intercepts this call 5. The scheduler suspends the current fiber and registers the socket with the event loop 6. The reactor runs other fibers while this one sleeps 7. When the socket becomes readable, the reactor resumes this fiber 8. `Net::HTTP` reads the response as if nothing happened Your code doesn't change. No `await`, no callbacks, no promises. The same `Net::HTTP.get` call that works in a thread works in a fiber. The yield is invisible. Bob Nystrom called this [the function color problem][function-color] in 2015. In languages with async/await, every function is either sync or async. An async function can only be called with `await`, and `await` can only live inside another async function. The color spreads upward through your entire call stack. **Python:** ```python # Python: the color spreads, and you need different libraries async def get_user(id): async with aiohttp.ClientSession() as session: # can't use requests response = await session.get(f"/users/{id}") # must await return await response.json() # must await async def handle_request(): # must be async because it calls get_user user = await get_user(1) # must await ``` You can't use `requests` in async Python without blocking the event loop. You need `aiohttp`, `httpx` in async mode, or a thread wrapper. You can't use the blocking `psycopg2` API as async I/O; you need `asyncpg` or Psycopg's async API. The ecosystem splits: sync libraries and async libraries, doing the same thing differently. **JavaScript:** ```javascript // JavaScript: same problem, less severe (Node has fewer library splits) async function getUser(id) { const response = await fetch(`/users/${id}`); // must await return await response.json(); // must await } async function handleRequest() { // must be async const user = await getUser(1); // must await } ``` **Ruby:** ```ruby # Ruby: no color def get_user(id) response = Net::HTTP.get(URI("https://api.example.com/users/#{id}")) # just a normal call JSON.parse(response) # just a normal call end def handle_request user = get_user(1) # just a normal call end ``` Same `Net::HTTP`. Same `pg`. Same call stack, as long as the library uses scheduler-aware Ruby I/O. The fiber scheduler intercepts I/O at the Ruby runtime level, below your code. Your methods don't know and don't care whether they're running in a thread or a fiber. ## What happens when a fiber does CPU-bound work ```ruby # Inside a fiber 100_000.times { Digest::SHA256.hexdigest("work") } ``` This blocks the reactor. No other fiber runs until it finishes. There's no I/O boundary to yield at, so the fiber holds the thread. sequenceDiagram participant R as Reactor participant F1 as Fiber 1 (CPU) participant F2 as Fiber 2 (I/O) R->>F1: Run Note over F1,F2: F1 doing CPU work... Note over F2: Waiting to run Note over F1,F2: F1 still computing... Note over F2: Still waiting F1->>R: Done R->>F2: Finally runs This is not a bug. It's the current tradeoff of cooperative scheduling. Fibers are designed for I/O-bound work; CPU-bound work belongs on a thread, where CRuby can preempt it. With [my fiber-mode patch for Solid Queue][sq-article], this is a configuration choice: ```yaml workers: - queues: [ chat, turbo, notifications ] fibers: 50 # I/O-bound: use fibers - queues: [ cpu ] threads: 2 # CPU-bound: use threads ``` One backend, two modes, matching the concurrency model to the workload. ## What happens when a fiber queries the database The [pg gem][] has supported `Fiber.scheduler` since v1.3.0. When a fiber executes a query, the pg gem sends it non-blockingly via `PQsendQuery`, then calls `rb_io_wait` on the PostgreSQL socket. The scheduler intercepts this, suspends the fiber, and lets others run while PostgreSQL processes the query. ```ruby # Inside a fiber user = User.find(42) # yields while waiting for PostgreSQL ``` The fiber yields. Other fibers run. When PostgreSQL responds, the reactor resumes the fiber. Your code doesn't know the difference. ### Pool size follows database work A database connection is busy until its query finishes. While PostgreSQL works, Ruby can run something else -- another thread, or another fiber on the reactor -- but that connection stays checked out. For an LLM job, most of the wall time is not database time. Read a row, call an API, stream tokens, write a status update. The database touches are short. The long waits are external HTTP. So 100 jobs in flight does not mean 100 jobs hitting PostgreSQL at the same instant. The reactor never preempts a fiber -- it only switches when a fiber yields at an I/O boundary: sequenceDiagram participant R as Reactor participant F1 as Fiber A participant F2 as Fiber B participant Pool as DB Pool (1 conn) participant PG as PostgreSQL participant HTTP as HTTP API R->>F1: Run F1->>Pool: Check out F1->>PG: SELECT * FROM users Note over F1: Yields (waiting for PG) R->>F2: Run F2->>HTTP: GET /api/data Note over F2: Yields (waiting for HTTP) PG-->>R: F1's result ready R->>F1: Resume F1->>Pool: Return F1->>R: Done HTTP-->>R: F2's result ready R->>F2: Resume F2->>Pool: Check out F2->>PG: UPDATE messages SET ... Note over F2: Yields (waiting for PG) PG-->>R: F2's result ready R->>F2: Resume F2->>Pool: Return F2->>R: Done Read this as a timeline. Fiber A uses the only connection for its query. While PostgreSQL works, Fiber B waits on HTTP. After Fiber A returns the connection, Fiber B can use it for its update. If both fibers tried to query at the same time, one would wait unless the pool had another connection. Active Record follows the same checkout rules in both cases. The current Solid Queue difference is a guardrail: thread mode expects `threads + 2` connections per process, so you don't run 50 execution threads against a 5-connection pool. Fiber mode can use a smaller baseline because `fibers: 100` means "allow 100 jobs to wait," not "create 100 execution threads." In my patch, I/O-heavy workers often start at 3 connections per process (1 execution + 2 worker overhead). If the jobs are DB-heavy, raise it. ## What happens when a fiber starts a transaction A transaction changes the timeline. The connection cannot be returned after each statement, because the transaction state lives on that connection. When a fiber starts a transaction, it keeps its checked-out connection for the entire duration -- from `BEGIN` to `COMMIT` or `ROLLBACK`. The connection is not released mid-transaction. Other fibers that need the database wait for the connection to be returned. sequenceDiagram participant R as Reactor participant F1 as Fiber A participant F2 as Fiber B participant Pool as DB Pool (1 conn) participant PG as PostgreSQL R->>F1: Run F1->>Pool: Check out F1->>PG: BEGIN F1->>PG: UPDATE accounts SET ... Note over F1: Yields (waiting for PG) R->>F2: Run F2->>Pool: Check out Note over F2: Waits (connection held by F1) PG-->>F1: Result R->>F1: Resume F1->>PG: COMMIT F1->>Pool: Return F1->>R: Done Pool->>F2: Connection available F2->>PG: SELECT * FROM accounts Note over F2: Yields (waiting for PG) PG-->>F2: Result R->>F2: Resume F2->>Pool: Return F2->>R: Done Under fiber isolation (`config.active_support.isolation_level = :fiber`), Active Support's execution state is fiber-scoped, so Active Record's lease is associated with the current fiber instead of the surrounding thread. The connection still gets a real `Monitor` lock. No other fiber can touch it during a transaction. Safe. No interleaving. Fiber B just waits. For the target workload -- LLM streaming, HTTP calls -- database touches are short reads and status updates. Transactions are brief. The wait is negligible. If your jobs run long transactions, those jobs belong on a thread-based worker. ## What happens when you have too many fibers Fibers aren't free. Each one uses memory (~4KB), and each one might hold open connections to external services. If you spawn 10,000 fibers that all hit the same API, you're opening 10,000 connections to that API. The API will not be happy. Async doesn't eliminate resource limits; it changes where they show up. With threads, the limit is explicit: 25 threads, 25 concurrent jobs. With fibers, the limit is implicit: you keep going until something else breaks. The fix is a semaphore. The `FiberPool` in my Solid Queue patch uses one: ```ruby semaphore = Async::Semaphore.new(size) # Only `size` fibers run concurrently semaphore.async do perform_job end ``` When you configure `fibers: 100` with the patch, that's not "unlimited fibers." It's a semaphore capping concurrency at 100. You control the ceiling. ## "Why not just configure more Solid Queue threads?" In plain Ruby, more threads can be reasonable. In Solid Queue thread mode, `threads: 200` means more than "allow 200 jobs to wait on I/O." **Kernel threads are the expensive unit.** Fibers don't make I/O complete faster; they let you wait on far more of it at once for a fraction of the cost. [Samuel Williams' benchmarks][fiber-bench] show fibers allocate 20x faster (~3μs vs ~80μs) and switch 10x faster (~0.1μs vs ~1.3μs) than threads. The OS can manage thousands of threads, but scheduler state, stack reservations, wakeups, and GVL coordination make that a poor default concurrency knob. **Solid Queue currently enforces a database-pool guard.** Today it expects `threads + 2` database connections per process, so 200 threads across 2 processes won't boot unless the pool is at least 404. That guard may be conservative for I/O-heavy jobs; [there's an open issue][sq-736] about making it advisory or bypassable. But it is still a guard you hit today. **A blocked job still occupies its worker thread.** The OS can park an LLM streaming thread until the socket is ready, but in Solid Queue thread mode it still consumes one of the configured thread workers. If all 25 are streaming tokens, job 26 waits. Fibers make the Solid Queue limit mean "how many jobs may wait at once" instead of "how many kernel threads should exist." They still need limits, but the limit is no longer one kernel thread per waiting job. ## "Why not Ractors?" Ractors solve a different problem. Fibers give you I/O concurrency -- many things waiting at once. Ractors give you CPU parallelism -- many things computing at once. Here's what they look like: ```ruby # Two Ractors computing fibonacci in parallel r1 = Ractor.new { fibonacci(38) } r2 = Ractor.new { fibonacci(38) } r1.value # Ruby 4.0+ r2.value # Both ran in parallel, each with their own GVL ``` Each Ractor has its own GVL, so they can execute Ruby code truly in parallel across CPU cores. The tradeoff: strict isolation. You can only share immutable (frozen) objects. Everything else gets copied or moved between Ractors via message passing. Access a mutable variable from an outer scope? `Ractor::IsolationError`. When Ractors win, they win big. Fibonacci(38) five times: 0.68s with Ractors vs 2.26s sequential. 3.3x speedup. Real parallelism. But they are not a practical answer for Rails jobs yet: - **Still experimental in Ruby 4.0.** Creating a Ractor still emits the experimental API warning. - **Many gems don't work without changes.** Gems that rely on mutable constants, global variables, class variables, or shared process state can hit `Ractor::IsolationError`. - **No Rails integration.** ActiveRecord, ActionCable, the router, the logger -- Rails is built on shared mutable state. None of it runs inside a Ractor. - **No Ractor-based job queue exists.** - **Still active bug surface.** The Ruby bug tracker still has Ractor-related issues, including recent crash reports. For I/O concurrency, Ractors don't help at all. Each Ractor still has threads constrained by its own GVL. Fibers within those threads still do the actual I/O multiplexing. Ractors add CPU parallelism, which is not what LLM streaming needs. For Rails jobs that need CPU parallelism today, processes are still the boring answer. Puma already uses that model for web workers. Ractors may become useful for isolated CPU-heavy Ruby work, but they are not the answer to this Solid Queue I/O problem. ## "Isn't this just what JavaScript does?" No. I showed the [code comparison above](#what-happens-when-a-fiber-hits-io). JavaScript's async/await is a colored concurrency model: the `async` keyword spreads upward through every caller. Ruby's fibers are colorless: your existing code works unchanged, and the scheduler handles yields below your code. There's a deeper difference too. JavaScript async/await runs on an event loop. Ruby fibers run on top of a multi-threaded runtime. You can have multiple Ruby threads, each running its own reactor with its own fibers, and mix fibers and threads in the same application. Node can run JavaScript in parallel with `worker_threads`, but that's a worker/isolate model, not the same thing as putting multiple reactors inside ordinary application threads. ## "Isn't this just what Go does?" Closer. Goroutines are lightweight, runtime-scheduled, and multiplexed across OS threads. Conceptually similar to Ruby fibers, but Go's scheduler can also preempt goroutines. Two differences: 1. **Go has true parallelism.** Goroutines run across multiple OS threads with no GVL equivalent. CPU-bound goroutines run in parallel. Ruby fibers don't. 2. **Ruby has existing code.** If you have a Rails application with hundreds of thousands of lines of Ruby, you can add fiber-based concurrency without rewriting anything. Your models, your controllers, your views, your gems -- they all work. With Go, you're rewriting. If you're starting from scratch and need both I/O concurrency and CPU parallelism, Go is a strong choice. If you have a Ruby application and need I/O concurrency, fibers give you that without a rewrite. ## "Fibers need `Async do` blocks. That's still new syntax." Someone on [Hacker News][hn-thread] called this out: I said "no async/await" but the examples show `Async do` and `.wait`. Here's the actual change: ```ruby # Before chat = RubyLLM.chat response = chat.ask("Hello") # After Async do chat = RubyLLM.chat response = chat.ask("Hello") end ``` Two lines of wrapping. Your application code inside doesn't change. Your models don't change. Your gems don't change. Nothing gets a new keyword. In Python, adopting async means rewriting every function signature in the call chain to `async def`, adding `await` to every call, and replacing or wrapping blocking libraries. `requests` becomes `aiohttp` or async `httpx`. Blocking database APIs become async database APIs. Your test framework changes. Your middleware changes. It's a rewrite. Two lines of wrapping vs. rewriting your stack. That's not even the same conversation. ## When to use what flowchart TD A[What kind of work?] --> B{CPU-bound?} B -->|Yes| C{Need parallelism?} C -->|Yes| D{Rails?} D -->|Yes| E[Processes] D -->|No| H[Ractors] C -->|No| F[Threads] B -->|No| I[Fibers] style E fill:#4a90a4,color:#fff style H fill:#c084fc,color:#fff style F fill:#7fb069,color:#fff style I fill:#e8a87c,color:#fff - **I/O-bound work** (LLM streaming, HTTP calls, webhooks, email delivery): **fibers.** Low overhead, high concurrency, database connections sized to database work rather than waiting jobs. - **CPU-bound work** (image processing, data crunching, PDF generation): **threads.** CRuby can preempt them, and C extensions can release the GVL for parallelism. - **CPU parallelism with Rails**: **processes.** Each one gets its own GVL, its own memory, its own everything. Puma already does this. - **CPU parallelism without Rails**: **Ractors** (when they graduate from experimental). Lighter than processes, true parallelism, but strict isolation means most gems don't work. - **All of them at once**: that's what a well-configured Rails app does. Puma forks processes. Each process runs threads. Fibers run inside those threads for I/O-heavy jobs. They coexist. ```yaml # Solid Queue with the fiber-mode patch: all three working together workers: - queues: [ chat, turbo ] fibers: 50 # I/O-bound: fibers processes: 2 # parallelism: processes - queues: [ pdf, images ] threads: 4 # CPU-bound: threads processes: 1 ``` No single model is universally better. The right answer is matching the model to the workload. --- This covers every "what happens when" question I've gotten so far. If I missed yours, [find me on Twitter][@paolino]; I'll either update this post or write a follow-up. [async-article]: /async-ruby-is-the-future/ [sq-article]: /solid-queue-doesnt-need-a-thread-per-job/ [async]: https://github.com/socketry/async [pg gem]: https://github.com/ged/ruby-pg [sq-736]: https://github.com/rails/solid_queue/issues/736 [hn-thread]: https://news.ycombinator.com/item?id=44516555 [function-color]: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ [fiber-bench]: https://github.com/socketry/performance/tree/adfd780c6b4842b9534edfa15e383e5dfd4b4137/fiber-vs-thread [bench]: https://github.com/crmne/solid_queue_bench [@paolino]: https://twitter.com/paolino --- ### Making the Rails Default Job Queue Fiber-Based URL: https://paolino.me/solid-queue-doesnt-need-a-thread-per-job/ Date: 2026-04-21 > **Update, July 31, 2026:** [Solid Queue 1.6.0][release] has shipped with fiber worker execution. The default Rails job queue can now run many long-running, cooperative I/O-bound jobs like LLM streaming far more efficiently, without making the queue database pool grow with the number of jobs waiting on I/O. The setup below now uses the official release. Last year I moved the LLM streaming jobs in [Chat with Work][] to [Async::Job][async-job]. It was fast. Genuinely fast. Fiber-based execution with Redis, thousands of concurrent jobs on a single thread. I was so convinced that I [wrote a whole post][async-article] about why async Ruby is the future for AI apps and recommended it to everyone. Then I started hitting walls. Async::Job doesn't persist jobs. They go into Redis and they're gone. [Mission Control][] shows nothing. Background jobs in Rails are already quieter than the rest of your application -- they fail without anyone noticing unless you go looking. Even with Honeybadger catching exceptions, I still want to see the full picture: which jobs are queued, which are running, which failed, what the system looks like right now. Without job persistence, you don't get that. Solid Queue is the default in Rails 8. Every new Rails app ships with it. When someone picks up Rails to build an LLM application and their 25-thread worker pool can only handle 25 concurrent streaming conversations, the answer shouldn't be "swap your entire job backend." It should be "change one line of config." So I [opened a PR][pr]. On July 31, it shipped in Solid Queue 1.6.0. ## Threads vs fibers, quickly If you already know this, [skip ahead to the config](#the-switch). By default, Solid Queue runs each job on its own thread. Those threads can all query the database concurrently, so conservative pool sizing assumes one connection per execution thread. There is also stack memory and kernel thread overhead. For a job that crunches data for 30 seconds, that's fine -- the thread is busy. For a job that streams an LLM response for 30 seconds but spends 99% of that time waiting for tokens, the thread is just sitting there holding resources. Fibers sidestep much of this. They are cooperatively scheduled in userspace on a single thread. When a fiber hits scheduler-aware I/O -- an [Async::HTTP][async-http] request or waiting for the next token through a compatible client -- it steps aside and another fiber picks up. One thread, hundreds of concurrent jobs. No kernel thread overhead per job, and database pools can be sized for simultaneous database work rather than every job waiting on network I/O. The [async][] gem installs the fiber scheduler. Ruby operations such as `Kernel.sleep`, scheduler-aware `IO`, and fiber-aware libraries yield without changing the job itself. This is not magic around every blocking call: a library or C extension that does not cooperate with the scheduler can still block the reactor thread. For the full deep dive -- processes, threads, fibers, the GVL, I/O multiplexing -- see [Async Ruby is the Future][async-article]. ## The switch Fiber mode ships in Solid Queue 1.6.0. Upgrade Solid Queue and add [async][] as an application dependency: ```ruby # Gemfile gem "solid_queue", "~> 1.6" gem "async" # required for fiber workers ``` Then switch that worker's execution setting: ```yaml # config/queue.yml production: workers: - queues: ["*"] # threads: 10 fibers: 100 # error register_fatal_error(error) raise end end ``` When the worker picks up jobs, it hands them to the pool. Each one becomes a fiber: ```ruby def wait_for_executions(semaphore) while execution = pending_executions.pop semaphore.async(execution) do |_execution_task, scheduled_execution| perform_execution(scheduled_execution) end end end ``` The worker poller claims only as many jobs as the pool has capacity for and pushes them into a `Thread::Queue`. Its `pop` is fiber-scheduler-aware, so the reactor can run execution fibers while it waits for more work. Each compatible I/O wait yields back to the reactor instead of occupying a dedicated execution thread. CPU-bound work gets nothing from fibers. They don't parallelize computation. A CPU-heavy job or blocking call stalls every execution fiber in that worker until it returns. In the default `fork` supervisor mode, the supervisor and other worker processes keep running, but that worker's reactor does not. Put CPU-bound or blocking jobs on a thread worker instead. ## The database connection math I [wrote about this last year][async-article]: > For 1000 concurrent conversations using traditional job queues like SolidQueue or Sidekiq, you'd need 1000 worker slots. That means 1000 kernel threads across your worker fleet, plus enough database pool capacity for whatever fraction of those jobs can hit the database at the same time. Even when the jobs are 99% idle waiting for streaming tokens, the thread resources are still reserved. That framing is about worker resources, not a special Active Record rule. The released code's pool-size check is specifically about the **Solid Queue database pool** (`SolidQueue::Record.connection_pool`), not every database your job might use. It estimates connections for polling, heartbeats, and job execution. Size any application database pools touched by the job separately. Solid Queue 1.6 does not give thread workers the same small pool estimate. It still estimates one execution connection per configured thread, plus one for polling and one for heartbeats. That's `threads + 2`. There is a separate change that makes this easy to confuse: since Solid Queue 1.5, that estimate is advisory. A thread worker can boot with a smaller pool and wait for a connection when the pool is busy, although it can still hit a checkout timeout under sustained contention. But 1.6 did not make the thread and fiber estimates equal. Here is the relevant version history: | Solid Queue version and worker | Queue pool estimate | What happens below it | |---|---|---| | 1.4.0 thread worker | `threads + 2` | Configuration is invalid; the supervisor aborts | | 1.5.x thread worker | `threads + 2` | Warning; the worker still boots | | 1.6.0 thread worker | `threads + 2` | Warning; the worker still boots | | 1.6.0 fiber worker, Active Record 7.1 | `fibers + 2` | Warning; the worker still boots | | 1.6.0 fiber worker, Active Record 7.2+ | `3` | Warning; the worker still boots | So the connection-sizing distinction is **Solid Queue 1.6 fiber workers on Active Record 7.2+ versus every thread worker**. The warning-versus-boot-error distinction is older: it changed between Solid Queue 1.4 and 1.5. For fiber workers, the estimate depends on the Active Record version. On Active Record 7.2+, Solid Queue assumes ordinary query paths release connections between queries, so it estimates one execution connection plus two worker connections regardless of the fiber count: `1 + 2 = 3`. On Active Record 7.1, it conservatively estimates one execution connection per fiber, so the estimate is `fibers + 2`. The three-connection estimate is a starting point, not a guarantee. Long transactions, `ActiveRecord::Base.connection`, `lease_connection`, direct pool checkouts, and long-lived `with_connection` blocks can pin connections across waits. If your jobs do that or generate simultaneous database work, increase the relevant pool. Here is the exact warning threshold calculated by Solid Queue 1.6 for a worker process at different concurrency levels: | Concurrent jobs | Thread worker | Fiber worker, Active Record 7.2+ | Fiber worker, Active Record 7.1 | |---|---|---|---| | 10 | 12 | 3 | 12 | | 25 | 27 | 3 | 27 | | 50 | 52 | 3 | 52 | | 100 | 102 | 3 | 102 | | 200 | 202 | 3 | 202 | On Active Record 7.2+, the thread estimate scales linearly while the fiber estimate stays flat. In the default `fork` mode, multiply the per-process pool by the number of worker processes: 6 processes with 50 execution slots means 312 configured queue connections for thread workers versus 18 for fiber workers. PostgreSQL's default `max_connections` is 100. Again, Solid Queue only calculates this estimate and warns. It does not configure the pool automatically. In supervisor `async` mode, workers share a process, so their connection needs must be added together rather than applying the per-process table independently. The benchmarks below use two pool policies. The primary Solid Queue comparison deliberately gives both modes the same pool, `DB_POOL = concurrency + 5` per worker process, so it measures the executor instead of measuring pool starvation. The stress suite uses mode-specific pools to show the operational failure envelope under higher connection demand. ## The benchmarks I reran the benchmark suite on April 28, 2026. These results were produced from the PR branch before the final 1.6.0 implementation was reorganized during review, so treat them as benchmarks of that implementation, not fresh Solid Queue 1.6.0 numbers. The architecture is the same, but a new run is required before attributing the exact deltas to the release tag. I will update this post over the next few weeks with fresh benchmarks against Solid Queue 1.6.0. The headline Solid Queue comparison covers four workloads across per-process concurrency 5, 10, 25, 50, and 100; process counts 1, 2, and 6; and both execution modes. Three runs per cell, median real run reported, with total concurrency capped at 60 so the main comparison stays about executor behavior. The workloads: - **Sleep**: 50ms `Kernel.sleep`. Pure cooperative wait. The I/O upper bound. - **Async HTTP**: HTTP request to a local server with 50ms delay via [Async::HTTP][async-http]. Real fiber-friendly I/O. - **CPU**: 50,000 SHA256 iterations. Pure computation. The control. - **RubyLLM Stream**: Actual [RubyLLM][] chat completion through a fake OpenAI SSE endpoint, with token-by-token Turbo Stream broadcasts. 40 tokens at 20ms each. The closest thing to a production AI workload you can benchmark repeatably. ### Results | Workload | Best throughput | Avg paired delta | Best paired delta | |---|---|---|---| | RubyLLM Stream | fiber, 7.01 j/s | **+11.9%** | **+21.8%** | | Async HTTP | fiber, 492.82 j/s | **+9.5%** | **+25.5%** | | Sleep | fiber, 500.50 j/s | **+7.4%** | **+15.9%** | | CPU | fiber, 110.02 j/s | +0.6% | +2.4% | RubyLLM Stream is the workload that matters. It runs an actual [RubyLLM][] chat completion with streaming, database writes, and Turbo broadcasts per token -- the same thing [Chat with Work][] does in production. Fiber wins every single paired experiment there: 9 out of 9. The CPU row is the control. Fibers don't help computation, and the average confirms it: essentially flat. That's how you know the I/O gains are real and not measurement noise. That table shows the best observed point and the paired-cell deltas. Here's the full spread. Some configurations favor threads for synthetic workloads, but the paired averages are the steadier signal: fiber wins the I/O workloads, and RubyLLM Stream always favors fiber. The newer suite also adds database-shaped workloads. With matched pools, short DB bursts still favor fiber: `db_queries` averages +12.6%, and a read/API/write mix averages +6.9%. The transaction case is the useful caveat: when each job pins a connection for the whole transaction, fiber still averages +3.5%, but the win is less consistent. That's exactly the workload where you should be more careful with pool sizing. ## Thread mode hit the wall Those benchmarks cap total concurrency at 60. I wanted to see what breaks when you push past that, so I ran a stress suite: per-process concurrency 25, 50, 100, 150, and 200; process counts 2 and 6; three runs per cell. Read this as the April PR implementation's failure-envelope test, not a Solid Queue 1.6.0 result or a universal law about threads and fibers. The result is stark. Thread mode only completed the smallest cell for each workload. Fiber mode completed every planned cell. | Workload | Thread cells completed | Fiber cells completed | |---|---|---| | Sleep | 1/10 | 10/10 | | Async HTTP | 1/10 | 10/10 | | RubyLLM Stream | 1/10 | 10/10 | PostgreSQL's default `max_connections` is 100. In this stress run, thread mode at concurrency 50 with 2 processes asked for 110 worker-pool connections. With 6 processes, even concurrency 25 asked for 180. The one surviving thread cell was the smallest: concurrency 25, 2 processes. Fiber mode in the stress suite used a smaller mode-specific pool: 6 connections per process for 2-process runs, 10 per process for 6-process runs. That is 60 worker-pool connections at concurrency 200 across 6 processes, while the benchmark's thread policy would configure 1,230. The exact constants are benchmark policy, but the shape is the point for this worker design: Solid Queue's thread estimate scales with thread concurrency; the Active Record 7.2+ fiber baseline scales with worker process overhead plus actual database concurrency. ## One backend, two modes Fiber mode isn't universally better. CPU-bound jobs get nothing from it, and blocking libraries or C extensions that do not cooperate with Ruby's fiber scheduler stall the reactor. And that's fine -- you don't have to pick one. As Trevor Turk pointed out in the PR discussion, that's the whole point: separately configured worker pools. Here's what [Chat with Work][] actually runs in production: ```yaml workers: - queues: [ chat ] fibers: 10 processes: 2 polling_interval: 0.1 - queues: [ turbo ] fibers: 10 processes: 1 polling_interval: 0.05 - queues: [ notifications, default, maintenance ] fibers: 5 processes: 1 polling_interval: 0.2 - queues: [ cpu ] threads: 1 processes: 1 ``` Almost everything uses fibers. LLM streaming, Turbo broadcasts, notifications, maintenance jobs -- all fiber-based. Only the `cpu` queue uses threads, and right now it's just one thread for the occasional heavy extraction. One backend. One deployment. [Mission Control][] shows all of it. Instead of running Solid Queue and Async::Job side by side -- two processors, two configurations, two sets of things to monitor -- you run one. I moved [Chat with Work][] to this setup, and Brad Gessler has been running it in production too. Async::Job is actually faster if you compare raw throughput against Redis. It is a backend comparison, not a Solid Queue executor comparison, but the ceiling is useful: | Workload | Solid Queue fiber best | Async::Job best | Delta | |---|---|---|---| | RubyLLM Stream | 7.01 j/s | 16.94 j/s | +141.7% | | Async HTTP | 492.82 j/s | 652.96 j/s | +32.5% | | Sleep | 500.50 j/s | 644.98 j/s | +28.9% | | CPU | 110.02 j/s | 125.75 j/s | +14.3% | If you want raw speed and don't need persistence, Async::Job is the right call. But if you want job visibility, failure tracking, retries, Mission Control, everything Rails gives you out of the box, fiber mode gets you there. Same concurrency. You can size database connections to database work instead of the number of jobs waiting on network I/O. You set `fibers: N` and keep building. --- Fiber mode is now available in [Solid Queue 1.6.0][release]. The [PR][pr] has the implementation history, and the [benchmark suite][bench] is open source. Run your own numbers, or challenge mine. [async-article]: /async-ruby-is-the-future/ [release]: https://github.com/rails/solid_queue/releases/tag/v1.6.0 [pr]: https://github.com/rails/solid_queue/pull/728 [RubyLLM]: https://rubyllm.com [Chat with Work]: https://chatwithwork.com [async-job]: https://github.com/socketry/async-job [async]: https://github.com/socketry/async [async-http]: https://github.com/socketry/async-http [Mission Control]: https://github.com/rails/mission_control-jobs [bench]: https://github.com/crmne/solid_queue_bench --- ### Your Agent's Context Window Is Not a Junk Drawer URL: https://paolino.me/your-agents-context-window-is-not-a-junk-drawer/ Date: 2026-04-07 Your agent's context window is the most precious resource it has. The more you stuff into it, the worse your agent performs. Researchers call it [context rot](https://research.trychroma.com/context-rot): the more tokens in the window, the harder it becomes for the model to follow instructions, retrieve information, and stay on task. Chroma tested 18 frontier models and found that accuracy drops up to 30% when you go from a focused 300-token input to 113k tokens of conversation history, with the task held constant. The model essentially became _dumber_. This holds true regardless of how big the window is, yet most agent setups treat the context window like a junk drawer. "Just toss it in there, the LLM will figure it out!" ## MCP: the biggest offender Don't get me wrong. MCP is a fine idea. You need to talk to a service? Grab an MCP server, plug it in, and you're running in ten minutes. For prototyping, for exploration, for answering "is this even worth building?", it's great. The problem is what happens next. Which is: nothing. People leave the MCP servers plugged in. They add more. Every MCP server you connect dumps tool descriptions, schemas, and instructions into your context. You didn't write those. You didn't optimize them. You probably haven't even read them. You're handing over a chunk of your context window to whatever some third party decided to shove in there. Say you need a tool that checks the weather. You could plug in an MCP server and get dozens of tool descriptions, parameter schemas, and whatever instructions its author decided to write. Or you could write this: ```ruby class Weather e { error: e.message } end end ``` Twelve lines of [RubyLLM](https://rubyllm.com). You wrote the description, so you know exactly what tokens are going into your context. You wrote the parameters, so the model gets precisely the interface it needs, no more. You own it, you can tune it, and nobody can inject anything into your agent's brain through it. Use MCP to prototype. Then replace it with crafted tools you actually control. ## Tool responses are context too Your RAG retrieves ten full documents when the model needs a paragraph. Your API call returns a massive JSON blob when the model needs two fields. You're paying for every one of those tokens with your agent's IQ. The fix is progressive disclosure. At [Chat with Work](https://chatwithwork.com), when the agent searches your Google Drive, we don't dump entire files into context. The search tool returns only some metadata and a single line from the file, the line that matched the search keywords. Fifty results, fifty lines. The AI reads those, decides which files actually matter, and only then reads them. If a file is too large, it reads it in chunks. At every step, the model is only looking at what it needs. The same principle applies to any tool. Don't return everything. Return enough for the model to decide what to look at next. ## Your instructions are context too Then there's the stuff you wrote yourself. Your system prompt is context. Your tool descriptions are context. Your parameter schemas are context. Every edge case, every guardrail, every overly detailed description competes for attention. You think you're being thorough. You're actually drowning the instructions that matter in a sea of instructions that don't. A focused system prompt will outperform an exhaustive one every time. ## Tool count is context too You hand-crafted 40 beautiful tools. Your agent needs 5 for this task. The other 35 sit in context doing nothing except making the model slower at picking the right one. Don't register every tool your agent might ever need. Load the tools the current task actually requires. If you're building a support agent that handles billing and technical issues, don't give it all of both. Route billing questions to a billing agent and technical questions to a technical agent. Two focused agents will outperform one bloated one. ## Every token should earn its place The context window is not a junk drawer. It's a workbench. Everything on it should be there for a reason, and you should be able to say what that reason is. So before you plug in another MCP server, add another RAG source, or write another paragraph in your system prompt, ask yourself one question: is this worth making my agent dumber? --- ### I Built a Monitor Configuration Tool for Hyprland URL: https://paolino.me/hyprmoncfg-monitor-configuration-for-hyprland/ Date: 2026-03-31 Configuring monitors in Hyprland means writing `monitor=` lines by hand. A 4K display at 1.33x scale is effectively 2880x1620 pixels, so the monitor next to it needs to start at x=2880. Vertically centering a 1080p panel against it means doing division in your head to get the y-offset right. You reload, you're off by 40 pixels, you edit, you reload again. There's no visual feedback until after you've committed to a config. Then it gets worse. You unplug your laptop, go to a conference, plug into a projector, and you're back to editing config files backstage before your talk. You come home, dock the laptop, and the layout is wrong again. I looked at what was available. The closest to what I wanted was [Monique](https://github.com/ToRvaLDz/monique): spatial editor, profiles, workspace management, a hotplug daemon. It does exactly what I need. But it's a GTK4 GUI that pulls in Python and a stack of dependencies, and the daemon was broken when I tried it. The other tools each cover parts of this: [kanshi](https://sr.ht/~emersion/kanshi/) does profiles and auto-switching but has no editor, you write config files; [nwg-displays](https://github.com/nwg-piotr/nwg-displays) and [HyprMon](https://github.com/erans/hyprmon) have spatial editors but no daemon; [HyprDynamicMonitors](https://github.com/fiffeek/hyprdynamicmonitors) has a daemon but no real layout tool, and it pulls in UPower and D-Bus. I wanted Monique's feature set without the dependency baggage, in something that works over SSH when your monitors are broken. So I built [hyprmoncfg](https://hyprmoncfg.dev). ## A real spatial editor, in your terminal The TUI is the thing I'm most proud of. It's not a config editor with a preview pane. It's a full spatial layout tool. The left side is a canvas where your monitors are drawn as rectangles, proportional to their resolution. You click one to select it, drag it to move it. Monitors snap to each other's edges as you position them, just like arranging windows in a GUI display manager. Arrow keys give you fine control: 100px per step, Shift for 10px, Ctrl for 1px. The right side is a per-monitor inspector. Pick a resolution and refresh rate from a scrollable list. Set scale, position, transform, VRR, mirroring. All inline, no dialogs within dialogs. A third tab handles workspace planning. And because it's a TUI: it works over SSH. When your monitor configuration is broken and you can't see anything, you can SSH into the machine and fix it. Try that with a GTK app. ## Safe apply with automatic revert Every apply, whether from the TUI or the daemon, follows the same path: write `monitors.conf` atomically (temp file + rename, no corruption), reload Hyprland, re-read the actual monitor state, and verify the result matches what was requested. Then it gives you 10 seconds to confirm. If you don't, maybe because the layout left you staring at a black screen, it reverts automatically. No stuck monitors. No reaching for a second machine to undo the damage. This is the same apply engine everywhere. The TUI and the daemon share identical code. If it works when you test it interactively, it works when the daemon fires at 2am because you bumped your dock cable. ## Workspace planning Monitor configuration and workspace assignment are the same problem. If you're rearranging monitors, you probably want workspaces to follow. hyprmoncfg has a workspace planner built into its third tab, with three strategies: - **Sequential**: Groups in chunks. Workspaces 1-3 on monitor A, 4-6 on monitor B. - **Interleave**: Round-robins. 1→A, 2→B, 3→A, 4→B. - **Manual**: Explicit per-workspace rules when you want full control. Workspace assignments are stored inside each profile and applied together with the layout. Switch profiles, switch workspace distribution. One operation. ## Source-chain verification Here's something no other tool does. Before writing anything, hyprmoncfg parses your `hyprland.conf` and verifies it actually sources the target `monitors.conf`. If it doesn't, it refuses to write. Other tools skip this check. They silently update a file that Hyprland never reads. You spend twenty minutes debugging why nothing changed, only to realize the file was never sourced. I lost an evening to this once. Never again. ## Dotfiles integration Profiles are stored as JSON files in `~/.config/hyprmoncfg/profiles/`, one per profile. The generated `monitors.conf` is a build artifact, you don't commit it. You commit the profiles. ```sh chezmoi add ~/.config/hyprmoncfg ``` Save a "desk" profile at home with your ultrawide. Save "conference-1080p" at one venue. Save "conference-4k" at another. Sync them across machines via your [dotfiles](https://github.com/crmne/dotfiles). The daemon matches profiles to connected hardware automatically. Arrive somewhere, plug in, and the right layout applies. This is portable. The same profile library works across machines because matching is based on the monitors you have, not on the machine you're at. ## One runtime dependency: Hyprland Two compiled Go binaries. No Python, no GTK, no GObject introspection, no D-Bus, no UPower. Install them and you're done. The only runtime requirement is Hyprland itself. ## How it compares | | hyprmoncfg | Monique | HyprDynamicMonitors | HyprMon | nwg-displays | kanshi | |---|---|---|---|---|---|---| | GUI or TUI | TUI | GUI | TUI | TUI | GUI | CLI | | Spatial layout editor | Yes | Yes | Partial | Yes | Yes | No | | Drag-and-drop | Yes | Yes | No | Yes | Yes | No | | Snapping | Yes | Not documented | No | Yes | Yes | No | | Profiles | Yes | Yes | Yes | Yes | No | Yes | | Auto-switching daemon | Yes | Yes | Yes | No (roadmap) | No | Yes | | Workspace planning | Yes | Yes | No | No | Basic | No | | Mirror support | Yes | Yes | Yes | Yes | Yes | No | | Safe apply with revert | Yes | Yes | No | Partial (manual rollback) | No | No | | Source-chain verification | Yes | No | No | No | No | No | | Additional runtime dependencies | None | Python + GTK4 + libadwaita | UPower, D-Bus | None | Python + GTK3 | None | ## Try it On Arch: ```sh yay -S hyprmoncfg ``` Or build from source: ```sh go install github.com/crmne/hyprmoncfg/cmd/hyprmoncfg@latest go install github.com/crmne/hyprmoncfg/cmd/hyprmoncfgd@latest ``` Check out the [documentation](https://hyprmoncfg.dev/) for the full guide, or browse the [source on GitHub](https://github.com/crmne/hyprmoncfg). --- ### Comb Shaped Slices URL: https://paolino.me/comb-shaped-slices/ Date: 2026-03-24 A friend who's built and shut down companies in this space sat across from me at breakfast during a conference recently. He knows what I'm building: [Chat with Work](https://chatwithwork.com), an AI tool that lets you talk to your actual work data. He wanted to know what my plan was. I think he was a bit concerned. "Add more integrations, finish the security assessment, market it well." I said. That didn't help. "All those LLM providers are going to eat the whole market. They'll ship every integration you can think of. If you want a slice of the pie, you need to pick a vertical and own it." I told him I was going to grab a T shaped slice of the pie instead. He looked at me like I'd lost it. --- Here's the thing about the "pick a vertical" advice: it's not wrong. It's just not the only way. And for a lot of small software companies, it's a trap dressed up as strategy. The conventional wisdom goes like this: the market is huge, the big players are coming, so you'd better find your little corner and defend it. Specialize. Go deep. Become the AI assistant for dentists in Luxembourg or the knowledge tool for corporate lawyers in Berlin-Brandenburg. Calculate your total addressable market. Build a defensible moat. Make investors happy. But what if you don't care about making investors happy? Most companies don't need investors. What if you just want to build something good? ## The comb I said T shaped in the moment. One horizontal, one vertical. But the more I thought about it, the more teeth it grew. Less like a T, more like a comb. Here's why. When you're OpenAI or Google, you sample from the top of the distribution. You build what most people use first, then work your way down. The result is always the same: a broad horizontal platform that serves everyone and surprises no one. When you're small, you sample from what's right in front of you. You build for yourself because no amount of user research, design thinking, or theory of mind will ever match the depth of actually needing the thing you're making. You understand your own problems in a way that connects to your emotions, your workflow, your instincts. You can't fake that. You can't interview your way to it. I chose fast onboarding over full sync, because I don't want to wait to start working. Nextcloud, Todoist, IMAP, and CalDAV: that's my stack, so that's where I'll go deep next. Then you listen to your customers. "This is cool, but I use Slack." So you build that too. A team needs to own their data, so you add on-premises installation. Someone uses Basecamp, and you build that integration because the people behind it think like you. One tooth at a time. The shape that emerges is yours. Not because you planned it on a whiteboard, but because you started from yourself and grew outward. It works for the small teams, the freelancers, the music collectives, the people who don't have an IT department and don't want one. That's the comb: not a strategy you choose, but what naturally happens when you're small and you give a damn. There's a reason people still choose Linear over Jira, or Proton over Gmail, or Plausible over Google Analytics. It's not because the small player has more features. It's because someone built it for themselves first, and that resonated. The entire market doesn't need to resonate with you. Just enough of it. So yes, the big players are coming. They're going to ship a lot of integrations. They're going to spend a lot of money. And they're going to build software that feels like it was built by a company that spends a lot of money. I'll be over here, grabbing my comb shaped slice of pie. It's [Plenty][]. _Today also happens to be the day I officially founded [Plenty][]. The papers are signed. The comb is real!_ [Plenty]: https://plenty.is --- ### Ruby Deserves Beautiful Documentation URL: https://paolino.me/ruby-deserves-beautiful-documentation/ Date: 2026-03-19 Have you ever looked at a VitePress documentation site and felt a little jealous? The sidebar navigation. The "On this page" outline on the right. The search that pops up with `/`. The homepage that actually looks like a product page, not a README with a nav bar. Dark mode that just works. Code blocks with copy buttons and language labels. It all looks like someone sat down and designed the whole experience. Because someone did. VitePress is genuinely great. And Ruby developers know it, because some of the most visible projects in our community are shipping their docs on VitePress. Not on a Jekyll theme, not on a Ruby tool. On a JavaScript static site generator built for Vue. I don't blame them. I looked at what we had in the Jekyll ecosystem and understood immediately. The best option is Just the Docs, and I've been using it for [RubyLLM](https://rubyllm.com). It's solid. But I had to patch in proper dark mode support that follows the browser setting. I had to add a copy-page button. The homepage layout is narrow and document-y. It works. It doesn't wow. So I built [Jekyll VitePress Theme](https://jekyll-vitepress.dev). ## What It Is A Jekyll theme gem that recreates the VitePress documentation experience. Everything you'd expect: - Top nav with mobile menu - Left sidebar, right "On this page" outline - Homepage layout with hero section and feature cards - Built-in local search (press `/` or `Cmd+K`) - Dark/light/auto appearance toggle - Code blocks with copy buttons, language labels, and file title bars - Doc footer with edit link, previous/next pager, and "last updated" - GitHub star widget - Rouge syntax highlighting with separate light and dark themes All configured through `_config.yml` and `_data/*.yml` files. No JavaScript toolchain. No Node.js. Just Jekyll. ## Getting Started ```ruby gem "jekyll-vitepress-theme" ``` {: data-title="Gemfile"} ```yaml theme: jekyll-vitepress-theme plugins: - jekyll-vitepress-theme jekyll_vitepress: branding: site_title: My Project ``` {: data-title="_config.yml"} ```sh bundle install bundle exec jekyll serve --livereload ``` That's it. Your docs site now looks like VitePress. Customize the nav, sidebar, colors, fonts, and everything else from the [configuration reference](https://jekyll-vitepress.dev/configuration-reference/). ## Why This Matters When I came back to Ruby in 2024, I kept finding things that could be better. There wasn't a great LLM library, so I built [RubyLLM](https://rubyllm.com). Async deserved more attention, so I [blogged about it](/async-ruby-is-the-future). And our documentation sites? They didn't look the part. In open source, looks matter. A beautiful docs site tells potential users: this project is serious, maintained, and worth your time. It lowers the barrier to adoption. It makes people want to try your library. VitePress understood this. Now Jekyll has it too. ```ruby gem "jekyll-vitepress-theme", "~> 1.0" ``` --- ### RubyLLM 1.14: From Zero to AI Chat App in Under Two Minutes URL: https://paolino.me/rubyllm-1-14-chat-ui/ Date: 2026-03-18 RubyLLM 1.14 ships a full chat UI generator. Two commands and you have a working AI chat app with Turbo streaming, model selection, and tool call display, in under two minutes. The demo above shows the whole thing: new Rails app to working chat in 1:46, including trying it out. ## Why This Matters RubyLLM turned one last week. [1.0 shipped on March 11, 2025](/rubyllm-1-0/) with Rails integration from day one: ActiveRecord models, `acts_as_chat`, Turbo streaming, persistence out of the box. [1.4](/rubyllm-1.4-1.5.1/) added the install generator. [1.7](https://github.com/crmne/ruby_llm/releases/tag/1.7.0) brought the first scaffold chat UI with Turbo Streams. [1.12](/rubyllm-1-12-agents/) introduced agents with prompt conventions. Each release got closer to the same thing: AI that works the way Rails works. 1.14 fully realizes that goal. A beautiful Tailwind chat UI (with automatic fallback to scaffold if you're not using Tailwind). Generators for agents and tools. Conventional directories for everything. All of it extracted from [Chat with Work](https://chatwithwork.com), where it's been running in production for months. ## What You Get Two generators. That's it. ```sh bin/rails generate ruby_llm:install bin/rails generate ruby_llm:chat_ui ``` Your app now has this structure: ``` app/ ├── agents/ ├── controllers/ │ ├── chats_controller.rb │ └── messages_controller.rb ├── helpers/ │ └── messages_helper.rb ├── jobs/ │ └── chat_response_job.rb ├── models/ │ ├── chat.rb │ ├── message.rb │ ├── model.rb │ └── tool_call.rb ├── prompts/ ├── schemas/ ├── tools/ └── views/ ├── chats/ │ ├── index.html.erb │ ├── show.html.erb │ └── _chat.html.erb └── messages/ ├── _assistant.html.erb ├── _user.html.erb ├── _tool.html.erb ├── _error.html.erb ├── create.turbo_stream.erb ├── tool_calls/ │ └── _default.html.erb └── tool_results/ └── _default.html.erb ``` Separate partials for each message role. Turbo Stream templates for real-time updates via `broadcasts_to`. A background job that handles the AI response. Tool calls and tool results each get their own rendering pipeline. A complete Tailwind chat interface, not a scaffold you need to fight with. ## Full Tutorial: New App from Scratch If you want to start from zero, this is what the demo shows. The whole thing takes just a minute. ```sh rails new chat_app --css tailwind cd chat_app bundle add ruby_llm bin/rails generate ruby_llm:install bin/rails generate ruby_llm:chat_ui bin/rails db:migrate bin/rails ruby_llm:load_models bin/dev ``` That's a new Rails app with Tailwind, RubyLLM installed, the chat UI generated, the database set up, models loaded, and the server running. Open `localhost:3000/chats` and start talking to an AI. ## Generators for Agents, Tools, and Schemas Now the fun part. You scaffold agents, tools, and schemas the same way you'd scaffold anything else in Rails: ```bash bin/rails generate ruby_llm:agent SupportAgent ``` ``` app/ ├── agents/ │ └── support_agent.rb └── prompts/ └── support_agent/ └── instructions.txt.erb ``` The agent class comes with the [1.12 DSL](/rubyllm-1-12-agents/) ready to go. The instructions file is an ERB template for your system prompt, so you can version it, review it in PRs, and template it with runtime context. ```bash bin/rails generate ruby_llm:tool WeatherTool ``` ``` app/ ├── tools/ │ └── weather_tool.rb └── views/ └── messages/ ├── tool_calls/ │ └── _weather.html.erb └── tool_results/ └── _weather.html.erb ``` Each tool gets its own partials for rendering calls and results. Show a weather widget for the weather tool, a search results list for a search tool, all through Rails partials. ```bash bin/rails generate ruby_llm:schema Product ``` ``` app/ └── schemas/ └── product_schema.rb ``` This creates a schema for structured output validation. More on all of this in the [Rails integration docs](https://rubyllm.com/rails/), and the dedicated guides for [agents](https://rubyllm.com/agents/) and [tools](https://rubyllm.com/tools/). ## Self-Registering Provider Config For people building provider gems: providers now register their own configuration options instead of patching a monolithic `Configuration` class. ```ruby class DeepSeek 1.14' ``` --- ### Ruby Is the Best Language for Building AI Apps URL: https://paolino.me/ruby-is-the-best-language-for-ai-apps/ Date: 2026-02-20 > If your goal is to ship AI applications in 2026, Ruby is the best language to do it. ## The AI Training Ecosystem Is Irrelevant Python owns model training. PyTorch, TensorFlow, the entire notebooks-and-papers gravity well. Nobody disputes that. But you're not training LLMs. Almost nobody is. Each training run costs millions of dollars. The dataset is the internet! This is what AI development today looks like: ```bash curl https://api.openai.com/v1/chat/completions \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{"model": "gpt-5.2", "messages": [{"role": "user", "content": "Hello"}]}' ``` That's it. An HTTP call. The entire Python ML stack is _irrelevant_ to achieve this. What matters is everything around it: streaming responses to users, persisting conversations, tracking costs, switching providers when pricing changes. That's web application engineering. That's where Ruby and Rails shine like no other. ## "You Need a Complex Agent Framework or You're Not Doing Real AI" Bullshit. You need a beautiful, truly provider-independent API. Let me show you. ## Python vs JavaScript vs Ruby LLM Libraries ### Simple chat **Python (LangChain):** ```python from langchain.chat_models import init_chat_model from langchain.messages import HumanMessage model = init_chat_model("gpt-5.2", model_provider="openai") response = model.invoke([HumanMessage("Hello!")]) ``` You need to specify the provider, create an array of messages that need to be instantiated, etc. That's ceremony. **JavaScript (AI SDK):** ```javascript import { generateText } from 'ai'; import { openai } from '@ai-sdk/openai'; const { text } = await generateText({ model: openai('gpt-5.2'), prompt: 'Hello!', }); ``` What if you want to use a model from another provider? **Ruby ([RubyLLM][]):** ```ruby require 'ruby_llm' RubyLLM.chat.ask "Hello!" ``` Reads like it should. ### Token usage tracking If you're running AI in production, you need to track token usage. This is how you price your app. **LangChain (GPT):** ```python response = model.invoke([HumanMessage("Hello!")]) response.response_metadata['token_usage'] # {'completion_tokens': 12, 'prompt_tokens': 8, 'total_tokens': 20} ``` **LangChain (Claude):** ```python response.response_metadata['usage'] # {'input_tokens': 8, 'output_tokens': 12} ``` Different key and different structure! **LangChain (Gemini):** ```python response.response_metadata # ...nothing... ``` It's not even there! [RubyLLM][]: ```ruby response.tokens.input # => 8 response.tokens.output # => 12 ``` Same interface. Every provider. Every model. ### Agents This is where it gets fun. **Python (LangChain):** ```python from langchain_openai import ChatOpenAI from langchain.agents import create_agent model = ChatOpenAI(model="gpt-5-nano") graph = create_agent( model=model, tools=[search_docs, lookup_account], system_prompt="You are a concise support assistant", ) inputs = {"messages": [{"role": "user", "content": "How do I reset my API key?"}]} for chunk in graph.stream(inputs, stream_mode="updates"): print(chunk) ``` **JavaScript (AI SDK 6):** ```javascript import { ToolLoopAgent } from 'ai'; import { openai } from '@ai-sdk/openai'; const supportAgent = new ToolLoopAgent({ model: openai('gpt-5-nano'), system: 'You are a concise support assistant.', tools: { searchDocs, lookupAccount }, }); const { text } = await supportAgent.generateText({ messages: [{ role: 'user', content: 'How do I reset my API key?' }], }); ``` **Ruby ([RubyLLM][]):** ```ruby require 'ruby_llm' class SupportAgent We had a customer deployment coming up and our Langgraph agent was failing. I rebuilt it using [RubyLLM][]. Not only was it far simpler, it performed better than the Langgraph agent. > Our first pass at the AI Agent used langchain... it was so painful that we built it from scratch in Ruby. Like a cloud had lifted. Langchain was that bad. > At Yuma, serving over 100,000 end users, our unified AI interface was awful. [RubyLLM][] is so much nicer than all of that. These aren't people who haven't tried Python. They tried it, shipped it, and replaced it. ## Go Ship AI Apps with Ruby, Rails, and [RubyLLM][] When we freed ourselves from complexity, this community built Twitter, GitHub, Shopify, Basecamp, Airbnb. Rails changed web development forever. Now we have the chance to change AI app development. Because AI apps are all about the product. And nobody builds products better than Ruby developers. [RubyLLM]: https://rubyllm.com --- ### RubyLLM 1.12: Agents Are Just LLMs with Tools URL: https://paolino.me/rubyllm-1-12-agents/ Date: 2026-02-17 "Agent" might be the most overloaded word in tech right now. Every startup claims to have one. Every framework promises to help you build them. The discourse has gotten so thick that the actual concept is buried under layers of marketing. So let's start from first principles. ## What's an Agent? An agent is an LLM that can call functions. That's it. When you give a language model a set of tools it can invoke -- a database lookup, an API call, a file operation -- and the model decides when and how to use them, you have an agent. The model reasons about the problem, picks the right tool, looks at the result, and continues reasoning. Sometimes it calls several tools in sequence. Sometimes none. There's no special "agent mode." No orchestration engine. No graph of nodes. It's just a conversation where the model can do things besides talk. ## RubyLLM Always Had This Tool calling has been a core feature of [RubyLLM][rubyllm] since 1.0: ```ruby class SearchDocs { chat.user.display_name_or_email } end ``` This renders `app/prompts/work_assistant/instructions.txt.erb` with `display_name` available as a local. Namespaced agents map naturally: `Admin::SupportAgent` looks in `app/prompts/admin/support_agent/`. Your prompts are ERB templates. Version them in git. Review them in PRs. Treat them like the application code they are. ## Rails Integration The `chat_model` macro activates Rails-backed persistence: ```ruby class WorkAssistant 1.12' ``` [rubyllm]: https://rubyllm.com --- ### Dictation Is the New Prompt (Voxtype on Omarchy) URL: https://paolino.me/dictation-is-the-new-prompt/ Date: 2026-01-07 Typing every prompt feels backwards in 2026. You can speak faster than you can type. Hold a hotkey, speak, your OS types it for you. If you care about flow, dictation is the most underrated upgrade you can make. In the [Omarchy](https://omarchy.org/) world, [Hyprwhspr](https://github.com/goodroot/hyprwhspr) is getting a lot of attention after a recent DHH tweet: He's right: local dictation is _shockingly_ good now. The catch is Hyprwhspr uses Python virtual environments, which don't mix well with [mise](http://mise.jdx.dev/). Fortunately [Pete Jackson](https://github.com/peteonrails) [saw that and created](https://github.com/basecamp/omarchy/discussions/3872) [Voxtype](https://github.com/peteonrails/voxtype/) to solve exactly this issue! EDIT: five minutes after I posted this, DHH confirmed that Voxtype ships will ship with Omarchy 3.3! 🎉 ## Why Voxtype Voxtype is built in Rust, so you don't need Python virtual environments which means it works well with mise. It's fast, it just works, and when [I opened an issue asking for an Omarchy theme](https://github.com/peteonrails/voxtype/issues/26), [the author shipped it immediately](https://github.com/peteonrails/voxtype/releases/tag/v0.4.4). Now it looks *stunning* in my setup. With Vulkan enabled, transcription is almost instant on my Ryzen AI 9 HX370. The video at the top is not sped up. Longer text also transcribes instantly. If you want to copy my exact configuration, here it is. ## Install ```bash sudo pacman -S wtype ydotool wl-clipboard vulkan-icd-loader # last only if you want to use your GPU sudo yay -S voxtype voxtype setup --download voxtype setup gpu # if you want to use your GPU voxtype setup systemd ``` Restart Waybar after the changes: ```bash pkill -SIGUSR2 waybar ``` ## Voxtype config `~/.config/voxtype/config.toml` ```toml state_file = "auto" [hotkey] enabled = false [audio] device = "default" sample_rate = 16000 max_duration_secs = 600 [audio.feedback] enabled = true # Sound theme: "default", "subtle", "mechanical", or path to custom theme directory theme = "default" volume = 0.7 [whisper] model = "base.en" language = "en" translate = false on_demand_loading = true # saves your GPU until it's needed [output] mode = "type" fallback_to_clipboard = true # Delay between typed characters in milliseconds # 0 = fastest possible, increase if characters are dropped type_delay_ms = 1 [output.notification] on_recording_start = false on_recording_stop = false on_transcription = true [text] replacements = { "hyperwhisper" = "hyprwhspr" } [status] icon_theme = "omarchy" ``` ## Waybar integration `~/.config/waybar/config.jsonc` ```jsonc "custom/voxtype": { "exec": "voxtype status --follow --format json", "return-type": "json", "format": "{}", "tooltip": true }, ``` And add it to `modules-right`: ```jsonc "modules-right": [ "group/tray-expander", "custom/voxtype", "bluetooth", "network", "pulseaudio", "cpu", "battery" ] ``` `~/.config/waybar/style.css` ```css @import "voxtype.css"; @import "../omarchy/current/theme/waybar.css"; ``` `~/.config/waybar/voxtype.css` ```css #custom-voxtype { margin: 0 16px 0 0; font-size: 12px; font-weight: bold; border-top: 2px solid transparent; border-bottom: 2px solid transparent; transition: color 150ms ease-in-out, border-color 150ms ease-in-out; } #custom-voxtype.recording { color: #ff5555; animation: pulse 1s ease-in-out infinite; } #custom-voxtype.transcribing { color: #ff5555; } #custom-voxtype.stopped { color: #6272a4; } @keyframes pulse { 0% { opacity: 1; } 50% { opacity: 0.5; } 100% { opacity: 1; } } ``` ## Keybinding In your Hyprland config: ```ini # Voxtype bindd = SHIFT, XF86AudioMicMute, Transcribe, exec, voxtype record toggle ``` That's it. Use your voice whenever possible. It's faster, more natural, and keeps you in flow. --- ### Nano Banana with RubyLLM URL: https://paolino.me/nano-banana-with-rubyllm/ Date: 2025-10-23 Google wired Nano Banana into the chat interface `generateContent`, not the image API's `predict`. Counterintuitive if you're using RubyLLM, which makes you think in terms of _actions_ like [`paint`](https://rubyllm.com/image-generation/) instead of [`chat`](https://rubyllm.com/chat/). Once you know that quirk, it's straightforward. Only caveat: you need the latest trunk or v1.9+, because that's where we taught RubyLLM to unpack inline file data from chat responses. ## Wire It Up ```ruby chat = RubyLLM .chat(model: "gemini-2.5-flash-image") .with_temperature(1.0) # optional, but you like creativity, right? .with_params(generationConfig: { responseModalities: ["image"] }) # also optional, if you prefer the model to return only images response = chat.ask "your prompt", with: ["all.png", "the.jpg", "attachments.png", "you.png", "want.jpg"] image_io = response.content[:attachments].first.source ``` That `StringIO` holds the generated image. Stream it to S3, attach it to Active Storage, or keep it in memory for a downstream processor. Want a file? ```ruby response.content[:attachments].first.save "nano-banana.png" ``` That's it. Chat endpoint, one call. Ship the image feature and go enjoy the rest of your day. --- ### RubyLLM 1.4-1.5.1: Three Releases in Three Days URL: https://paolino.me/rubyllm-1.4-1.5.1/ Date: 2025-08-01 Three releases in three days. Wednesday, Friday, and Friday again. Each one shipped as soon as it was ready. ## 1.4.0: The Structured Output Release (Wednesday) Getting LLMs to return data in the format you need has always been painful. We all had code like this: ```ruby # The old struggle response = chat.ask("Return user data as JSON. ONLY JSON. NO MARKDOWN.") begin data = JSON.parse(response.content.gsub(/```json\n?/, '').gsub(/```\n?/, '')) rescue JSON::ParserError # Hope and pray end ``` Now with structured output: ```ruby # Define your schema with the RubyLLM::Schema DSL class PersonSchema {"name" => "Yukihiro", "age" => 59, "skills" => ["Ruby", "C", "Language Design"]} ``` No more regex. No more parsing. Just data structures that work. Oh, and Daniel Friis released [RubyLLM::Schema](https://github.com/danielfriis/ruby_llm-schema) just for the occasion, but you can use any gem you want with RubyLLM, or even write your own JSON schema from scratch. ## Rails Generators: From Zero to Chat We didn't have Rails generators before. Now we do: ```bash rails generate ruby_llm:install ``` This creates everything you need: - Migrations - Models with `acts_as_chat`, `acts_as_message`, and `acts_as_tool_call` - A clean initializer Your Chat model works like any Rails model: ```ruby chat = Chat.create!(model: "gpt-4.1-nano") response = chat.ask("Explain Ruby blocks") # Messages are automatically persisted with proper associations ``` From `rails new` to working chat in under 5 minutes. ## Tool Call Transparency New callback to see what your AI is doing: ```ruby chat.on_tool_call do |tool_call| puts "🔧 AI is calling: #{tool_call.name}" puts " Arguments: #{tool_call.arguments}" Rails.logger.info "[AI Tool] #{tool_call.name}: #{tool_call.arguments}" end chat.ask("What's the weather in Tokyo?").with_tools([weather_tool]) # => 🔧 AI is calling: get_weather # Arguments: {"location": "Tokyo"} ``` Essential for debugging and auditing AI behavior. ## Direct Parameter Provider Access Need that one weird parameter? Use `with_params`: ```ruby # OpenAI's JSON mode chat.with_params(response_format: { type: "json_object" }) .ask("List Ruby features as JSON") ``` No waiting for us to wrap every provider option. ## Critical Bug Fixes and Other Improvements in 1.4.0 - **Anthropic multiple tool calls**: Was only processing the first tool call, silently ignoring the rest - **Streaming errors**: Now handled properly in both Faraday V1 and V2 - **Test fixtures**: Removed 60MB of unnecessary test data - **Message ordering**: Fixed race conditions in streaming responses - **JRuby support**: Now officially tested and supported - **Direct access to raw responses**: Get the raw responses from Faraday for debugging - **GPUStack support**: A production-ready alternative to Ollama [Full release notes for 1.4.0 available on GitHub.](https://github.com/crmne/ruby_llm/releases/tag/1.4.0) ## 1.5.0: Two New Providers (Friday) ### Mistral AI 63 models from France, from tiny to massive: ```ruby RubyLLM.configure do |config| config.mistral_api_key = ENV['MISTRAL_API_KEY'] end # Efficient small model chat = RubyLLM.chat(model: 'ministral-3b-latest') # Their flagship model chat = RubyLLM.chat(model: 'mistral-large-latest') # Vision with Pixtral vision = RubyLLM.chat(model: 'pixtral-12b-latest') vision.ask("What's in this image?", with: "path/to/image.jpg") ``` ### Perplexity Real-time web search meets LLMs: ```ruby RubyLLM.configure do |config| config.perplexity_api_key = ENV['PERPLEXITY_API_KEY'] end # Get current information with web search chat = RubyLLM.chat(model: 'sonar-pro') response = chat.ask("What are the latest Ruby 3.4 features?") # Searches the web and returns current information ``` [Full release notes for 1.5.0 available on GitHub.](https://github.com/crmne/ruby_llm/releases/tag/1.5.0) ### Rails Generator Fixes - Fixed migration order (Chats → Messages → Tool Calls) - Fixed PostgreSQL detection that was broken by namespace collision - PostgreSQL users now get `jsonb` columns instead of `json` ## 1.5.1: Quick Fixes (Also Friday) Found issues Friday afternoon. Fixed them. Shipped them. That's it. Why make users wait through the weekend with broken code? - Fixed Mistral model capabilities (was a Hash, should be Array) - Fixed Google Imagen output modality - Updated to JRuby 10.0.1.0 - Added JSON schema validation for model registry [Full release notes for 1.5.1 available on GitHub.](https://github.com/crmne/ruby_llm/releases/tag/1.5.1) ## The Philosophy: Ship When Ready Three days. Three releases. Each one made someone's code work better. We could have bundled everything into one release next week. But every moment we wait is a moment someone's dealing with a bug we already fixed. The structured output in 1.4.0? People needed that since before RubyLLM existed. The PostgreSQL fix in 1.5.0? Someone's migrations were failing Thursday. The Mistral fix? Breaking someone's code Friday morning. When code is ready, you ship. ## What You Can Build Now With structured output and multiple providers, you can build real features: ```ruby # Extract structured data from any text class InvoiceSchema 1.5' ``` Full backward compatibility. Your 1.0 code still runs. These releases just made everything better. --- ### Async Ruby is the Future of AI Apps (And It's Already Here) URL: https://paolino.me/async-ruby-is-the-future/ Date: 2025-07-09 I spent a decade in Python's async ecosystem. When I came back to Ruby, I couldn't find the async revolution. SolidQueue, Sidekiq, and GoodJob were all thread-based. Where Python had reorganized its entire world around `asyncio`, Ruby seemed stuck. Then I started building [RubyLLM][] and [Chat with Work][], and it clicked. LLM communication is async Ruby's killer app. Long-lived connections, token-by-token streaming, and thousands of concurrent conversations: exactly where threads fall apart. Here's the thing: Ruby's approach to async is actually *superior* to Python's. Python forced everyone to rewrite their entire stack. Ruby didn't. Your existing code just works. No syntax changes. No library migrations. Just better performance when you need it. [Samuel Williams][] and the [async][] community have been building this for years. We just needed the right use case to see it. ## Threads don't work well for LLM streaming in Ruby Streaming LLM responses hit every weak spot in thread-based concurrency at once: ### 1. Slot Starvation Configure any thread-based job queue with 25 workers: ```ruby class StreamAIResponseJob < ApplicationJob def perform(chat, message) # This job occupies 1 of your 25 slots... chat.ask(message) do |chunk| # ...for the ENTIRE streaming duration (30-60 seconds) broadcast_chunk(chunk) # Thread is 99% idle, just waiting for tokens end # Slot only freed here, after full response end end ``` Your 26th user? They're waiting in line. Not because your server is busy, but because all your workers are occupied by jobs waiting for tokens. ### 2. Resource Multiplication Each background job worker thread brings its own: - Potential database demand (25 workers can hit the database at once) - Stack memory allocation - OS thread management overhead For 1000 concurrent conversations using traditional job queues like SolidQueue or Sidekiq, you'd need 1000 worker slots. That means 1000 kernel threads across your worker fleet, plus enough database pool capacity for whatever fraction of those jobs can hit the database at the same time. Even when the jobs are 99% idle waiting for streaming tokens, the thread resources are still reserved. ### 3. Performance Overhead Real benchmarks show[^1]: - Creating a thread: ~80μs - Thread context switch: ~1.3μs - Maximum throughput: ~5,000 requests/second When you're handling thousands of streaming connections, these microseconds add up to real latency. ### 4. Scalability Challenges Try creating 10,000 threads and the kernel thread overhead starts to dominate. Yet modern AI apps need to handle thousands of concurrent conversations. These aren't separate issues. They're all symptoms of the same mismatch: LLM communication is fundamentally different from traditional background jobs. [^1]: [Samuel Williams][]' [fiber-vs-thread performance comparison](https://github.com/socketry/performance/tree/adfd780c6b4842b9534edfa15e383e5dfd4b4137/fiber-vs-thread) ## Understanding Concurrency: Threads vs Async To understand why, we need to build up from first principles. ### The Hierarchy: Processes, Threads, and Fibers Think of your computer as an office building: - **Processes** are separate offices -- each with its own locked door, furniture, and files. They can't see into each other's spaces. - **Threads** are workers sharing the same office -- they can access the same filing cabinets but need to coordinate to avoid collisions. - **Fibers** are multiple tasks juggled by one worker at their desk -- switching between them when waiting for something, like a phone call. ### Scheduling: The Core Difference The fundamental question in concurrency is: who decides when to switch between tasks? #### Threads: Preemptive Multitasking With threads, the operating system is the boss. It forcibly interrupts running threads to give others a turn: ```ruby # You start threads, but the OS controls them threads = 10.times.map do |i| Thread.new do # This might be interrupted at ANY point expensive_calculation(i) fetch_from_api(i) # Each thread blocks individually here process_result(i) end end ``` Each thread: - Gets scheduled by the OS kernel - Can be interrupted mid-execution (in Ruby, after 100ms) - Blocks individually on I/O operations - Requires OS resources and kernel data structures - Can need its own database connection while doing database work #### Fibers: Cooperative Concurrency With fibers, switching is voluntary -- they only yield at I/O boundaries: ```ruby # Fibers yield control cooperatively Async do fibers = 10.times.map do |i| Async do expensive_calculation(i) # Runs to completion fetch_from_api(i) # Yields here, other fibers run process_result(i) # Continues after I/O completes end end end ``` Each fiber: - Schedules itself by yielding during I/O - Never gets interrupted mid-calculation - Managed entirely in userspace (no kernel involvement) - Shares resources through the event loop ### Ruby's GVL: Why Fibers Make Even More Sense Ruby's Global VM Lock (GVL) means only one thread can execute Ruby code at a time. Threads are preempted after a 100ms time quantum. This creates an interesting dynamic: ```ruby # CPU work: Threads don't help much due to GVL threads = 4.times.map do Thread.new { calculate_fibonacci(40) } end # Takes about the same time as sequential execution! # I/O work: Threads do parallelize (GVL released during I/O) threads = 4.times.map do Thread.new { Net::HTTP.get(uri) } end # Takes 1/4 the time of sequential execution ``` But here's the thing: if threads only help with I/O anyway, _why pay their overhead_? ### The I/O Multiplexing Advantage This is where fibers truly shine. Threads use a "one thread, one I/O operation" model: ```ruby # Traditional threading approach thread1 = Thread.new { socket1.read } # Blocks this thread thread2 = Thread.new { socket2.read } # Blocks this thread thread3 = Thread.new { socket3.read } # Blocks this thread # Need 3 threads for 3 concurrent I/O operations ``` Fibers use I/O multiplexing -- one thread monitors *all* I/O: ```ruby # Async's approach (simplified) Async do # One thread, many I/O operations task1 = Async { socket1.read } # Registers with selector task2 = Async { socket2.read } # Registers with selector task3 = Async { socket3.read } # Registers with selector # Event loop uses epoll/kqueue to monitor ALL sockets # Resumes fibers as data becomes available end ``` The kernel (via `epoll`, `kqueue`, or `io_uring`) can monitor thousands of file descriptors with a single system call. No thread-per-connection needed. ### Why Fibers Win: The Complete Picture Let's look at real benchmark data comparing fibers to threads[^1]: **Performance Advantages (Ruby 3.4 data)**: - **20x faster allocation**: Creating a fiber takes ~3μs vs ~80μs for a thread - **10x faster context switching**: Fiber switches in ~0.1μs vs ~1.3μs for threads - **15x higher throughput**: ~80,000 vs ~5,000 requests/second But the real advantage is **scalability**: 1. **Fewer OS Resources**: Fibers are managed in userspace, avoiding kernel overhead 2. **Efficient Scheduling**: No kernel involvement means less overhead 3. **I/O Multiplexing**: One thread monitors thousands of I/O operations via `epoll`/`kqueue`/`io_uring` 4. **GVL-Friendly**: Cooperative scheduling works naturally with Ruby's concurrency model 5. **Resource Sizing**: Database pools can be sized to actual database concurrency instead of the number of jobs waiting on I/O While memory usage between fibers and threads is comparable, fibers don't depend on OS resources. You can create vastly more fibers than threads, switch between them faster, and manage them more efficiently while monitoring thousands of connections -- all from userspace. ## How Async Solves Every LLM Challenge Remember those four problems? Here's how async addresses each one: 1. **No More Slot Starvation**: Fibers are created on-demand and destroyed immediately. No fixed worker pools. 2. **Shared Resources**: One process with a correctly sized database pool can handle thousands of mostly-waiting conversations. 3. **Improved Performance**: 20x faster to create, 10x faster to switch, 15x less scheduling overhead (synthetic upper bound). 4. **Massively Improved Scalability**: 10,000+ concurrent fibers? No problem. The OS doesn't even know they exist. ## Ruby's Async Ecosystem The beauty of Ruby's [async][] is transparency. Python requires `async`/`await` everywhere. Ruby code just works: ### The Foundation: The [async][] Gem ```ruby require 'async' require 'net/http' # This code handles 1000 concurrent requests # Using ONE thread and minimal memory Async do responses = 1000.times.map do |i| Async do uri = URI("https://api.openai.com/v1/chat/completions") # Net::HTTP automatically yields during I/O response = Net::HTTP.post(uri, data.to_json, headers) JSON.parse(response.body) end end.map(&:wait) # All 1000 requests complete concurrently process_responses(responses) end ``` No callbacks. No promises. No async/await keywords. Just Ruby code that scales. ### Why RubyLLM Just Works™ [RubyLLM][] gets async performance *for free*. No async version of the library. No code changes. No configuration. RubyLLM uses `Net::HTTP` under the hood. Wrap your calls in an Async block and `Net::HTTP` automatically yields during network I/O. Thousands of concurrent LLM conversations on a single thread. ```ruby # This is all you need for concurrent LLM calls Async do 10.times.map do Async do # RubyLLM automatically becomes non-blocking # because Net::HTTP knows how to yield to fibers message = RubyLLM.chat.ask "Explain quantum computing" puts message.content end end.map(&:wait) end ``` Libraries that follow conventions get superpowers without even trying. That's Ruby at its best. Check out [RubyLLM's Scale with Async guide](https://rubyllm.com/guides/async) to learn more. ### The Rest of the Ecosystem - **[Falcon][]**: Multi-process, multi-fiber web server built for streaming - **[async-job][]**: Background job processing using fibers - **[async-cable][]**: ActionCable replacement with fiber-based concurrency - **[async-http][]**: Full-featured HTTP client with streaming support ... and many more available from [Socketry](https://github.com/orgs/socketry/repositories). ## Migrate your Rails app to Async The migration requires almost no code changes: ### Step 1: Update Your Gemfile ```ruby # Gemfile # Comment out thread-based gems # gem "puma" # gem "sidekiq" / "good_job" / "solid_queue" # gem "solid_cable" # Add async gems gem "falcon" gem "async-job-adapter-active_job" gem "async-cable" ``` ### Step 2: Configure Your Application ```ruby # config/application.rb require "async/cable" # config/initializers/async_job.rb require 'async/job/processor/inline' Rails.application.configure do config.async_job.define_queue "default" do dequeue Async::Job::Processor::Inline end config.active_job.queue_adapter = :async_job end ``` ### Step 3: There's No Step 3! Your existing jobs work unchanged. Your channels don't need updates. Just deploy with Falcon and watch. You'll get more performance, more capacity, and better response times. #### Note on Puma The above configuration works out of the box with Falcon. If you're using Puma, you'll need additional setup for concurrent job processing. See the [RubyLLM Async Guide](https://rubyllm.com/guides/async#note-on-puma) for Puma configuration details. ### Mixing Job Adapters: Best of Both Worlds You don't have to go all-in. Use async-job only for LLM operations while keeping your existing job processor for everything else: ```ruby # Keep your existing adapter as default config.active_job.queue_adapter = :solid_queue # or :sidekiq, :good_job, etc. # Base class for all LLM jobs class LLMJob < ApplicationJob self.queue_adapter = :async_job end # LLM jobs inherit the async adapter class ChatResponseJob < LLMJob def perform(conversation_id, message) # Runs with async-job - perfect for streaming response = RubyLLM.chat.ask(message) # ... end end # Regular jobs use your default adapter class ImageProcessingJob < ApplicationJob def perform(image_id) # Runs with solid_queue - better for CPU work # ... end end ``` This approach lets you optimize each job type for its workload without disrupting your existing infrastructure. ## When to Use What Let's be practical -- async isn't always the answer: **Use threads for:** - CPU-intensive work - Tasks needing true isolation - Legacy C extensions that aren't fiber-safe **Use async for:** - I/O-bound operations - API calls - WebSockets, SSE, and other forms of streaming - LLM applications ## Ruby got this right Python forced its entire community to rewrite everything for `asyncio`. Libraries fragmented. Codebases split. Every library needed an async twin. Ruby didn't do that. [Samuel Williams][] and the [async][] community built something that works with the code you already have. No syntax changes. No library migrations. Just better performance when you need it. LLM apps are where this pays off. Long-lived connections, streaming responses, and thousands of concurrent conversations: exactly the workload where fibers beat threads. And your existing code doesn't have to change to benefit. --- *[RubyLLM][] powers [Chat with Work][] in production with thousands of concurrent AI conversations using [async][].* *Thanks to [Samuel Williams][] for reviewing this post and providing the [fiber-vs-thread benchmarks](https://github.com/socketry/performance/tree/adfd780c6b4842b9534edfa15e383e5dfd4b4137/fiber-vs-thread).* *I'll be speaking about async Ruby and AI at [EuRuKo 2025](https://2025.euruko.org/), [San Francisco Ruby Conference 2025](https://sfruby.com/), and [RubyConf Thailand 2026](https://rubyconfth.com/).* [RubyLLM]: https://rubyllm.com [Chat with Work]: https://chatwithwork.com [Samuel Williams]: https://github.com/ioquatix [async]: https://github.com/socketry/async [Falcon]: https://github.com/socketry/falcon [async-job]: https://github.com/socketry/async-job [async-http]: https://github.com/socketry/async-http [async-cable]: https://github.com/socketry/async-cable --- ### RubyLLM 1.3.0: Smarter Attachments, Multi-Tenancy, and No More Manual Model Tracking URL: https://paolino.me/rubyllm-1-3/ Date: 2025-06-03 RubyLLM 1.3.0 ships three things: attachments that figure themselves out, isolated configuration contexts for multi-tenant apps, and the end of manually tracking model capabilities. ## Attachments Before, you had to tell RubyLLM what kind of file you were sending: ```ruby chat.ask "What's in this image?", with: { image: "diagram.png" } chat.ask "Describe this meeting", with: { audio: "meeting.wav" } chat.ask "Summarize this document", with: { pdf: "contract.pdf" } ``` Now just hand it the file: ```ruby chat.ask "What's in this file?", with: "diagram.png" chat.ask "Describe this meeting", with: "meeting.wav" chat.ask "Summarize this document", with: "contract.pdf" # Multiple files, mixed types chat.ask "Analyze these files", with: [ "quarterly_report.pdf", "sales_chart.jpg", "customer_interview.wav", "meeting_notes.txt" ] # URLs work too chat.ask "What's in this image?", with: "https://example.com/chart.png" ``` RubyLLM detects the type and does the right thing. You shouldn't have to think about file types when the computer can figure it out. ## Configuration Contexts Global config is fine until you need different API keys per customer. Passing config objects everywhere is tedious. So we built contexts: ```ruby tenant_context = RubyLLM.context do |config| config.openai_api_key = tenant.openai_key config.anthropic_api_key = tenant.anthropic_key config.request_timeout = 180 end response = tenant_context.chat.ask("Process this customer request...") # Global configuration stays untouched RubyLLM.chat.ask("This still uses your default settings") ``` Each context is isolated, thread-safe, and garbage-collected when you're done with it. Works for multi-tenancy, A/B testing providers, or anything where you need scoped configuration. ## Ollama Your dev machine shouldn't phone home to OpenAI every time you want to test something: ```ruby RubyLLM.configure do |config| config.ollama_api_base = 'http://localhost:11434/v1' end chat = RubyLLM.chat(model: 'mistral', provider: 'ollama') response = chat.ask("Explain Ruby's eigenclass") ``` Same API, local model. Good for development, testing, compliance, costs. ## OpenRouter One API key, hundreds of models: ```ruby RubyLLM.configure do |config| config.openrouter_api_key = ENV['OPENROUTER_API_KEY'] end chat = RubyLLM.chat(model: 'anthropic/claude-3.5-sonnet', provider: 'openrouter') ``` ## No More Manual Model Tracking *Update: RubyLLM has since moved from Parsera to [models.dev](https://models.dev) for model data.* We've been maintaining model capabilities and pricing by hand since 1.0. Every time a provider changes pricing or ships a new model, someone updates a file. That's over. We partnered with [Parsera](https://parsera.org) to build a [continuously updated API](https://api.parsera.org/v1/llm-specs) that scrapes model information from provider docs. `RubyLLM.models.refresh!` now pulls from that API. Context windows, pricing, capabilities, and modalities are always current. We kept our `capabilities.rb` files for older models that providers don't document well anymore. Between the two sources, virtually every model worth using is covered. [More on the Parsera API here](/standard-api-llm-capabilities-pricing-live/). ## Rails ActiveStorage now works properly with attachments: ```ruby class Message < ApplicationRecord acts_as_message has_many_attached :attachments end chat_record.ask("Analyze this upload", with: params[:uploaded_file]) chat_record.ask("What's in my document?", with: user.profile_document) chat_record.ask("Review these files", with: params[:files]) ``` Full parity with the plain Ruby implementation. ## Also in 1.3.0 - **Custom embedding dimensions**: `RubyLLM.embed("text", model: "text-embedding-3-small", dimensions: 512)` - **Enterprise OpenAI**: Organization and project ID support - **Ruby 3.1–3.4, Rails 7.1–8.0**: Officially tested - **13 new contributors** across foreign key fixes, HTTP proxy support, and more Thanks to @papgmez, @timaro, @rhys117, @bborn, @xymbol, @roelbondoc, @max-power, @itstheraj, @stadia, @tpaulshippy, @Sami-Tanquary, and @seemiller. ```ruby gem 'ruby_llm', '1.3.0' ``` Full backward compatibility. [GitHub](https://github.com/crmne/ruby_llm). --- ### The LLM Capabilities and Pricing API is Live URL: https://paolino.me/standard-api-llm-capabilities-pricing-live/ Date: 2025-05-13 *Update: The Parsera API has been sunsetted. RubyLLM now uses [models.dev](https://models.dev) for model capabilities and pricing.* The [LLM Capabilities API](/standard-api-llm-capabilities-pricing) I announced last month is live. [Browse the models](https://llmspecs.parsera.org/) or [hit the API directly](http://api.parsera.org/v1/llm-specs). ```yaml id: gpt-4o-mini name: GPT-4o mini provider: openai context_window: 128000 max_output_tokens: 16384 modalities: input: - text - image output: - text capabilities: - function_calling - structured_output - streaming - batch pricing: text_tokens: standard: input_per_million: 0.15 output_per_million: 0.6 cached_input_per_million: 0.075 ``` [Parsera][parsera] scrapes provider docs and keeps the data current. Context windows, pricing, capabilities, and modalities are all in one place. ## Already in RubyLLM [RubyLLM 1.3.0][rubyllm-release] pulls from this API directly: ```ruby RubyLLM.models.refresh! model = RubyLLM.models.find("gpt-4.1-nano") puts model.context_window # => 1047576 puts model.capabilities # => ["batch", "function_calling", "structured_output"] puts model.pricing.text_tokens.standard.input_per_million # => 0.1 ``` The API is open to everyone: any language, any framework. Found a missing model? [Report it](https://github.com/parsera-labs/llm-specs/issues). Providers should expose this data themselves. Until they do, this works. [rubyllm]: https://rubyllm.com [rubyllm-release]: /rubyllm-1-3 [parsera]: https://parsera.org --- ### A Standard API for LLM Capabilities and Pricing URL: https://paolino.me/standard-api-llm-capabilities-pricing/ Date: 2025-04-01 *Update: The Parsera API has been sunsetted. RubyLLM now uses [models.dev](https://models.dev) for model capabilities and pricing.* It's 2025, and no LLM provider exposes basic model information through their API. Context window? Pricing per token? Function calling support? You're reading documentation pages that change without notice and look different for every provider. I've been maintaining this data by hand in [RubyLLM][rubyllm] since the beginning. Every pricing change, every new model: someone updates a file. It doesn't scale. And every other LLM library is doing the same thing independently. So I partnered with [Parsera][parsera] to build what should have existed from the start: a single API that returns capabilities and pricing for every major LLM. ## The schema ```yaml id: gpt-4.5-preview # Matches the provider's API display_name: GPT-4.5 Preview provider: openai family: gpt45 context_window: 128000 max_output_tokens: 16384 knowledge_cutoff: 20231001 modalities: text: input: true output: true image: input: true output: false audio: input: false output: false pdf_input: false embeddings_output: false capabilities: streaming: true function_calling: true structured_output: true batch: true reasoning: false pricing: text_tokens: standard: input_per_million: 75.0 cached_input_per_million: 37.5 output_per_million: 150.0 batch: input_per_million: 37.5 output_per_million: 75.0 ``` Context windows, token limits, modalities, capabilities, pricing for standard and batch operations. Everything you need to programmatically pick a model and estimate costs. Parsera handles the scraping. They expose a public GET endpoint. RubyLLM integrates on day one. But this isn't just for RubyLLM; any library in any language can use it. We're finalizing the schema now: [check out the draft][gist]. Starting with OpenAI, Anthropic, Gemini, and DeepSeek. Feedback welcome in the [Gist comments][gist] or on [GitHub Discussions](https://github.com/crmne/ruby_llm/discussions). [gist]: https://gist.github.com/crmne/301be1d38ff193e7274a69833947139a [rubyllm]: https://rubyllm.com [parsera]: https://parsera.org --- ### RubyLLM 1.0 URL: https://paolino.me/rubyllm-1-0/ Date: 2025-03-11 I released [RubyLLM][rubyllm] 1.0 today. When I started building [Chat with Work](https://chatwithwork.com), I wanted to write this: ```ruby chat = RubyLLM.chat chat.ask "What's the best way to learn Ruby?" ``` And have it work regardless of model or provider. No provider-specific client classes, no different response formats, no ceremony. Just a conversation. That's RubyLLM. One API for OpenAI, Claude, Gemini, DeepSeek, and more. ## What it looks like ```ruby chat = RubyLLM.chat embedding = RubyLLM.embed("Ruby is elegant") image = RubyLLM.paint("a sunset over mountains") ``` Switch models whenever you want. Don't specify one and you get a sensible default: ```ruby chat = RubyLLM.chat(model: 'claude-3-5-sonnet') chat.with_model('gpt-4o-mini') ``` Tool calling is a Ruby class, not JSON Schema gymnastics: ```ruby class Search < RubyLLM::Tool description "Searches our knowledge base" param :query, desc: "Search query" param :limit, type: :integer, desc: "Max results", required: false def execute(query:, limit: 5) Document.search(query).limit(limit).map(&:title) end end chat.with_tool(Search).ask "Find our product documentation" ``` Streaming works the same way everywhere: ```ruby chat.ask "Write a story about Ruby" do |chunk| print chunk.content end ``` Token tracking is built in: ```ruby response = chat.ask "Explain Ruby modules" puts "This cost #{response.input_tokens + response.output_tokens} tokens" ``` Rails is a first-class citizen: ```ruby class Chat < ApplicationRecord acts_as_chat end chat = Chat.create!(model_id: 'gemini-2.0-flash') chat.ask "Hello" # Everything persisted automatically ``` Vision, PDFs, and audio through the same interface: ```ruby chat.ask "What's in this image?", with: { image: "photo.jpg" } chat.ask "Summarize this document", with: { pdf: "contract.pdf" } chat.ask "Transcribe this recording", with: { audio: "meeting.wav" } ``` Dependencies: Faraday, Zeitwerk, and a tiny event parser. That's it. RubyLLM already powers [Chat with Work](https://chatwithwork.com) in production. `gem install ruby_llm` and [rubyllm.com][rubyllm] has the rest. [rubyllm]: https://rubyllm.com --- ### Building Cluster Headache Tracker from a Hospital Bed URL: https://paolino.me/cluster-headache-tracker/ Date: 2024-08-19 Cluster headaches are called "suicide headaches" for a reason. The worst pain you can imagine, behind one eye, up to eight times a day. I've had them for years. During my last bout I spent two weeks in hospital. They handed me a paper form, one line per day. One line. For up to eight attacks, each with different intensity, duration, location, and medication. I looked at the form, looked at the nurses, and started coding. That's how [Cluster Headache Tracker][cht] happened. Built between attacks, in a hospital bed, while I was also supposed to be running [Freshflow][freshflow]. ## The app It started simple: log an attack, note the pain level, track what you took for it. The kind of thing that should have existed already. The headache apps out there are built for migraines. Different condition, different needs. Cluster headaches have their own patterns, their own triggers, their own treatments. Nobody had built something specific. So I did, and I open-sourced it. People started using it. Some told me it made their doctor visits actually productive for the first time. They could show real data instead of trying to remember through the fog. A few said it helped them get oxygen therapy approved, which can be a fight. Here's a demo: If you get cluster headaches, [try it][cht]. If you're a developer who wants to help, it's on [GitHub][github]. [freshflow]: https://freshflow.ai [cht]: https://clusterheadachetracker.com [github]: https://github.com/crmne/cluster-headache-tracker --- ### Bye Freshflow URL: https://paolino.me/bye-freshflow/ Date: 2024-08-04 I co-founded [Freshflow][freshflow] three and a half years ago. We built an AI system that tells supermarkets how much fresh produce to order so less of it ends up in the bin. We shipped fast, scaled to many stores, and assembled a team I'd work with again in a heartbeat. Now I'm moving on. Not because something went wrong. The company is in a good place and I'm staying on as a shareholder. But I want to build something different. Something bootstrapped, something mine. The best part of Freshflow was always the people. A small remote team that actually trusted each other, communicated asynchronously, and shipped work they were proud of. That's the thing I'll carry with me. Not a playbook, not a set of "key learnings," just the proof that a small group of people who give a damn can build something real. More on what's next soon. [freshflow]: https://freshflow.ai --- ## Links - About: https://paolino.me/about/ - GitHub: https://github.com/crmne - LinkedIn: https://www.linkedin.com/in/carminepaolino - Twitter: https://x.com/paolino - Instagram: https://www.instagram.com/crmne/ - Soundcloud: https://soundcloud.com/crimsonlakemusic - Crimsonlake: https://crimsonlake.live