When I worked at Code for America, I helped design and deliver two different technical interviews.

The first interview, which I feel confident I can take credit for wholly, was security related. Me, the interviewer, would first have the candidate read about Cross Domain Referer Leakage, and then we’d talk about it:

• How would you summarize the vulnerability in your own words?
• Describe to me a scenario for how an attacker would exploit this.
• Can you think of any apps or services you’ve used or worked on that could be vulnerable to this?
• What are some various ways that we might mitigate this? Are there any non-code changes, or things that you would estimate as more or less effort? Thinking of a typical app you’ve worked on, how might we address it?

…and then we’d go into an actual Ruby on Rails codebase, pinned to a specific commit from whence the codebase had that specific class of vulnerability, and talk through making a code change together to mitigate it, if not actually making the change then and there with tests and so forth (time permitting).

I particularly liked this interview because it involved some technical reading comprehension, the implication that project responsibility was broad (hey, you’re gonna have to be aware of things!), and the particular problem itself I find very technically satisfying because it involves weird browser security model stuff (but still simple, relatively speaking) and also the solution involves booping around redirects, which I think is fun.

As background, Code for America at that time was largely a Ruby on Rails shop, though we tried to be programming language and framework agnostic during the hiring process (to my mild disagreement). That said, we did emphasize experience with full-stack frameworks similar to Rails, like Laravel or Django or Java Spring, and especially emphasized full-stack web development. We were also, at the time, heavily pair-programming oriented (lots of former Pivotal Labs peeps), so these interviews were performed in front of dual-monitors+keyboards (or remotely with remote control) and the emphasis was on clearly communicating intent as much as it was on manipulating the code.

The second interview, which I don’t remember if it was my original design, or if I simply spent a lot of time shaping it over the years, was more of a traditional “here’s a feature ticket, let’s do it together.” That feature was like “As a user of the dashboard, I want to be able to filter [this list of records] by those authored by a specific user within our organization.” I think the provided design was maybe a picture of the page and a “filter widget goes here”-arrow below the heading and above the list of records.

During this interview, the main beats were like:

• For the visual affordance, what makes sense? And hopefully they’d ask about the design system and we’d talk about how many potential users there might be and I’d guide them towards like “a select dropdown”.
• How is it manipulated by a user and how does the select option value get transported to the backend server? (does it auto submit, is there a submit button)
• How do we filter it on the backend and display the result? Are there any validation or security or record lifecycle considerations we should make?

The last question always had a lot of substance to it, because it would be like “well, if we use their username as the value, what happens when they change their username?”, or “What happens if someone slaps in a user that isn’t a member of the org?” and so forth. All the sorts of things you’d hopefully enjoy nerding out about with a (potential) longterm colleague.

I’d always finish the second interview with an open ended question of like: Now that you’ve seen some of our administrative interface and we’ve talked about the data and uses… and thinking of your own experience building and using applications generally… what ideas come to mind for how we might further improve the functionality and usability of this interface?

And this is the bit where I think about information technology. Because a lot of the thing I’m trying to evaluate for people (engineers, but also designers and product managers and pretty much anybody working adjacent to “tech”) is What would you say we are even doing here? Like in the big sense?

And to answer that myself, it’s sorta like the Pushing Patrick I’ve mentioned before: we’re taking a data-based representation of something from over here… and then we’re putting it over there. And along the way the thing we’re building is:

• First off, we’re being reductive by taking a conceptual thing and reducing it to some limited representation of it in the data
• Then it’s being collected and aggregated so that some subset (or all of it, universally) is in our system
• It’s being enriched in some way by adding additional internal or external data to it. And maybe being conflated or deduplicated as well. And maybe this is happening explicitly by additional data entry, or maybe implicitly by some behavior of the system. Maybe immediately or over some time horizon, short, long or in perpetuity
• And then it’s being listed and filtered or scoped and then specific parts of the data are being projected or displayed in lots of different ways.
• And all together it’s ultimately driving towards some sort of monitoring and decisional process that leads to an action either within or outside the system.

And then for those of us building it, we’re trying to figure out how to do it in a way that is functional (it works), and usable, and ultimately adds value by being faster or cheaper or better or simply more possible than it would be if people did this without the Information Technology thing being built (e.g. than the status quo).

And you might be thinking: hey Ben, that sounds like more of a business, product, and design problem than a software engineering problem. And yes, I would agree with you somewhat, but I sorta think the “Software Engineering” domain is pretty uninteresting when untethered from the rest of it. As well as the thought: any backend, non-user facing system or library or object or function still has users: those users are the developers who have adopted or integrated or wrote and maintain the code that calls it. Same same.

I can’t help but think, nearly every time I write a predicate? method of Umbrella Today (good and bad).

I’ll offer that there is a dimension I have been thinking a lot about how it integrates into that previous definition of Information Technology, and that’s stickiness and engagement. On one hand, if you’re getting value out of one side of it, then it shouldn’t be soul-crushing to use. But if you’re using it purely for the delight and high and the actual value or benefit is fleeting… that seems… bad?

One reason these things come to mind is that I’m coming to the 15 year anniversary of writing Civic Tech Patterns, which has been a longterm project to bully a very small group of very bright people into not wasting their time (I choose these words very specifically; it’s not an approach I would choose from scratch today). At the heart is this same project of trying to engage people in “no really, what are you trying to do here?”

And with AI agents and generative chatbot agents and the like too, where many of the experiences of “Information Technology” (as I define it above) is put into a black box and what comes out is often… bad? But also sticky and delightful? For some people? Arguably? I dunno.

If anything, in addition to these thoughts pervading my own direct work, I think of this moment where:

• More and more people are ostensibly building “Information Technology”-style tools (with generative tools) and I see a potential gulf between looks like a thing that has historically performed Information Technology and (what I believe) is the actual goal of performs the function of, really well, Information Technology.
• Maybe my conceptualization of Information Technology is lacking something, or doesn’t quite encompass this new Generative+Agent thing many folks are trying to make happen. Of which I think Information Technology is ultimately about converge and Generative+Agent stuff seems more like diverge but I dunno.

Hence my writing this out, cause on that last bit, I dunno. And given how rarely I seemed to interview people who got it (at least according to me) for the last generation of Information Technology, I can’t say that it’s much of a change if this next generation doesn’t get it either. And probably that’s ok.

Random note: While looking through my (other) notebook, I found shorter commentary along similar lines that I commented on Reddit in reply to someone being grumpy about producing a portfolio of side projects:

I’m not the commenter you’re responding to. But I am also a hiring manager and want to try to meet both of you in the middle.

The thing that I want to evaluate, by seeing some non-school projects, is that the candidate has an understanding of how Information Technology can be used to solve a problem. Some of that is technological (does it work), and some of it is telling the story (eg input or take data and transform it in a new or novel way that solves a human-understandable problem). It’s “design” to the end of demonstrating you understand the overall problem space of how technology is applied to a problem. And calibrated to what’s expected of someone coming out of a (effective) bootcamp.

And when I’ve got 150 candidates for an entry level position, the candidates who can communicate that effectively will stand out.

It’s fairly well known that for the longest time, GitHub has not had a programmatic interface for uploading images and attachments. It’s possible in the browser, by dragging-and-dropping your image, but not via something you can script or curl. That’s been a bummer for lots of GitHub Action-powered systems you might imagine, like: automatically attaching demo screenshots or videos on PRs for new features, or attaching failure screenshots or visual diffs from browser tests.

Until now, I guess. Here’s the unofficial, undocumented endpoint:

FILE=cat.jpg
MIME=image/jpeg
REPOSITORY=bensheldon/good_job
TOKEN=$(gh auth token)

curl -s "https://uploads.github.com/user-attachments/assets?name=$FILE&content_type=$MIME&repository_id=$(gh api "repos/$REPOSITORY" --jq .id)" \
  -X POST -H "Authorization: Bearer $TOKEN" \
  -H "Accept: application/json" --data-binary "@$FILE"

# that outputs a JSON response containing the attachment URL, which you can then embed in an image tag into the GitHub content.

That’s it! I mean, you can be more clever than curl or whatever, but that’s the idea. It’s equivalent to dragging-and-dropping an image into an Issue, PR, or Comment.

I don’t know if this endpoint always accepted an Auth Token and no one tried or noticed or talked about it, or if that’s new. My memory was that this endpoint only previously accepted Cookie Auth: circa 2018 I explored different Visual Diff services and the lack of a programmatic API for GitHub attachments meant that to DIY you’d have to use a service like S3, and then a service to authorize S3, and that’s annoying to set up when GitHub Actions could do everything else. I’m pretty sure I tried to fuzz the attachment interface at the time and it didn’t work.

I recently learned all of this was possible because Justin Gordon of Shakacode shared a link to Intercom’s 2x-skills attach-github-assets and I was… skeptical… and then I tried it and it worked!

It’s very useful, I hope it doesn’t go away.

Here we go…

Via server do blocks

Rails 6.1 added a Railtie#server hook that can be invoked in application.rb or a gem’s engine.rb. It’s invoked when the config.ru file is evaluated.

Warning: the invocation of the hook in config.ru is missing from a lot of Rails projects. I once lackadaisically tried to address this upstream, but doublecheck your config.ru actually contains Rails.application.load_server as it should.

In your config.ru

The Rackup configuration file only gets evaluated by a webserver, so that is a natural place to invoke your webserver-only code.

Aside: like boot.rb, I either find interesting things lurking around in configu.ru , or it’s one of those files that hasn’t been touched since rails new was first run.

As a Puma plugin

If you’re dedicated to Puma, write a simple plugin for it. Puma plugins also allow hooking into more sophisticated places than simply “before it starts”.

tbh, I wasn’t aware of Puma plugins as an integration point until Solid Queue shipped a Puma plugin for its in-webserver mode. I like it!

By searching the callstack

I just mentioned Solid Queue has having an admirable integration point. My GoodJob has an… ok one: searching the callstack for webserver fingerprints. I chose this path at the time for the broadest zero-configuration compatibility with older versions of Rails, different webservers, and JRuby, but I wouldn’t recommend it very strongly today because it feels brittle:

def in_webserver?
  return @_in_webserver unless @_in_webserver.nil?

  @_in_webserver = Rails.const_defined?(:Server) || begin
    self_caller = caller
    self_caller.grep(%r{config.ru}).any? || # EXAMPLE: config.ru:3:in `block in <main>' OR config.ru:3:in `new_from_string'
      self_caller.grep(%r{puma/request}).any? || # EXAMPLE: puma-5.6.4/lib/puma/request.rb:76:in `handle_request'
      self_caller.grep(%{/rack/handler/}).any? || # EXAMPLE: iodine-0.7.44/lib/rack/handler/iodine.rb:13:in `start'
      (Concurrent.on_jruby? && self_caller.grep(%r{jruby/rack/rails_booter}).any?) # EXAMPLE: uri:classloader:/jruby/rack/rails_booter.rb:83:in `load_environment'
  end || false
end

In a Rack middleware

This is the simplest answer: if you only want code to run in the webserver, invoking it on the first (or any) request makes sense to me. In Ruby on Rails, I think the development environment’s “are there pending migrations?” check is a neat example because Rails does the check in the request, and then raises a special kind of ActionableError that causes the “Run pending migrations” button to appear.

Aside: I think people are sometimes too quick to reach for Rack middleware instead of doing a Controller before_action; consider that too.

Other thoughts

There is a lot of trickiness to this problem. Mastodon is sorta special because it’s an application that is distributed. As opposed to your own specially customized Rails application you’re wholly developing, or a narrow gem/engine that you’re distributing to be installed in someone else’s application.

It’s tough to recommend a single best option. For a gem or engine, like Solid Queue or GoodJob, you’re probably focused most on the simplest and trouble-free installation with the widest compatibility. For your own application, the goal is probably not leaving a forgotten footgun to be stumbled over in the future. Good luck!

I’ll cover the most common stumbles I’ve seen, and end with a little bit of advice for doing better. This is not intended to be an essay, or some something comprehensive like Good Strategy, Bad Strategy. Just fun stuff I occassionally try to describe to others.

I do recognize that a lot of these things stem from context like “toxic workplaces are bad and warp your sense of normalcy long after you leave them”, so if any of these things I write about seems self-evidently obvious, consider yourself lucky.

Rule of three

A simple heuristic for identifying bad planning, either from yourself or others, is how closely it hews to the Rule of Three. For example, if you’re fitting your planning to OKRs, and you end up with three Objectives, each with three Key Results, maybe someone in that process is a little bit lazy.

People like threes. If you turn in a one item plan, maybe it looks like you didn’t do the assignment; if you submit 5 or more, maybe you don’t look focused enough. I would suggest you play with it to see if that’s actually the case rather than always trying to shape everything down to 3: try defensibly submitting just one priority, or twelve, and see what happens.

When I worked at Code for America, there was a period where they adopted a planning framework called OGSM: Objectives, Goals, Strategies and Measures. Unfortunately, its application led to teams creating massive spreadsheets of 3 goals by 3 objectives by 3 Strategies by 3 Measures each. 3-to-the-4th-power is 81. Yikes. This was eventually reined in.

So to the extent that planning habitually hovers around 3 items per category, that’s the simplest sign that “planning” is a box-checking and template-filling exercise. Either by you, or by the people reviewing it.

Operational Planning and Strategic Planning

What’s the point of planning?

• To capture the entirety of people’s work and responsibilities and ensure there is proper direction and resources to achieve it all …or…
• To apply an unnatural focus upon a critical opportunity or risk that might not otherwise get the attention it needs to be addressed.

I call the former Operational Planning, and the latter Strategic Planning. I do Strategic Planning consultation, and a good 20% of my time is spent ensuring that the thing we’re doing together is Strategic rather than Operational.

Both Operational Planning and Strategic Planning are important and necessary. But you gotta keep them distinct. Especially as planners not-infrequently pad out ostensibly “Strategic” plans with activities that would happen anyways due to momentum and natural care (gotta get to three!).

Most of this post is about my experience trying to do Strategic Planning well.

Who appears in a plan? Not necessarily everyone.

A common problem I see with Strategic Planning is a necessity that everyone’s work show up in the plan. I experienced this a lot working in matrixed, cross-functional teams, when every function (Design, Engineering, Support) wanted—if not needed, to justify themselves to others—their work to show up in the strategy for the cross-functional team as a whole.

In good Strategic Planning, not everyone’s primary focus will show up in the overall strategy all the time. Sometimes that looks like this: we’re going to continue operating effectively on our own goals in order to create space for other parts of the team to have a strategic focus. That should be ok!

Roll up or roll down? Don’t wait.

At nearly every tech job I’ve worked, there was the intent that company strategy would be set at the top, by leadership, and then that strategy would roll down to sub-organizations and departments who would do a planning process to align with the company strategy.

That “roll-down” never happened effectively. Instead, the status quo was that individual teams would kinda take a temperature check, read a few recent company memos, and write their own plans based on their own internal goals, and wash those team goals through the contemporary language of the company. Then the department would take those and kind of generalize them into three-ish themes, and that would continue rolling up and never be heard from again.

The most disfunctional times was when teams would wait to plan in the expectation of some company-wide process that never came. And then would be caught flat-footed and disorganized when they were inevitably asked “What is your focus right now? What would extra resources help you achieve? How might you absorb a resource cut?”

The point being here, I would never recommend waiting for inputs to planning. Just do it, and then align with whatever the official process is.

Planning should be sufficient

Some of the most annoying things in my career is seeing my own leadership, or an adjacent team, be casually asked by someone even higher up some flavor of: “what are you doing to work towards [new company priority]?” and the sub-leader immediately kicks off some new planning process to preempt the old one.

Have some backbone, and simply remind them of current plan. I think a lot of the time no one honestly remembers what was previously planned and they’re just making chit-chat. If you have a quarterly planning process, then any new company priority gets fed as input into the next planning cycle (unless it’s actaully pants-on-fire/company-runs-out-of-money-otherwise situation, which is rare),

Measures and Targets are different

I’ve observed people try to find a nihilism between the statements “You can’t manage what you don’t measure” and “When a measure becomes a target, it ceases to be a good measure” (Goodhart’s Law). Whereas I think you just have to allow both to be true at once and allow “target” and “measure” to mean different things.

The Target is what you aim directly at. A Measure is just something you look at occassionally or from the side of your eye and say “huh, interesting”.

This distinction isn’t particularly important, other than to be able to distinguish one from the other, and also recognize when someone with power over you is “huh, interesting” a little too hard and you should pay attention.

Progress and percentage targets

For anything that you’re asked to track as “percentages to completion” you need to ask whether the intent is that you reach 100% or there’s some neg-style “if you hit 100% that means you weren’t audaciaous enough in setting your target” and thus you need to take your actual target and inflate/deflate it the expectation.

The outcome you want is both that you know what you’re trying to achieve (regardless of what you publicly commit to) and you look good doing it.

My advice for anyone in charge is to simply make 100% the achievable target and avoid all of this triangulative nonsense.

Percentages are also annoying when you’re measuring progress towards some boolean done-or-not-done where there’s no benefit until it’s done. 80% of zero value is still zero value.

In my opinion, the entire reason to report on progress is to have some shared context for if or when you need to ask for more resources to finish it out. Same with stoplights. If the only shared context is “things were green but now they’re yellow” that’s not useful; in the same way as “if some projects aren’t yellow, that means we’re not working hard enough” sucks too. Also when percentages or stoplights get averaged and rolled-up 🙄.

Writing a plan vs doing planning

The most common pattern I see for“planning enablement” is the company provides a template for the final document. e.g. fill out this template and turn it in and that’s your plan.

Template-first planning means that a planning process then becomes:

1. First, filling in the “what are you going to do?” section
2. Second, making up something narratively consistent to fill out everything else

I see this happen everywhere. People already have stewing what they want to do, and then everything else is backwards justification.

This backwards-justification is one of the key dynamics that we’re going to try to overcome as I now shift towards how I think you should plan instead.

How I think you should plan, instead

The main idea in how I think you should plan is that you break the planning down into stages where you try to get farther and farther away from “what people want to do” and towards a more complex “what needs to be done to overcome the barriers that make it hard for people to do the things they want to do”. Got all that? Good.

Aside: None of this is my new idea. Creating conceptual distance is core part of Synectics (yes, Steve Krug once mentions it) and a lot of this process comes from taking lots of trainings in Technology of Participation, which has entire books on gestalt theory and such.

So of separate stages, that means that you force yourself to do each piece separately, rather than starting at the end (“what you want to do”) and working backwards or all at once. The stages are:

1. What’s going on? (the dry name here is “Environmental Scan”) Laundry list any new themes, priorities, developments that have happened inside the company and outside it since the last planning process. Go look at the last plan too. “We said we were gonna do X” is a good thing to list here. The thing to be aware of is that people are dying to shout “what we are going to do” so everything at this stage is still somewhat suspect. But that’s fine, we recognize thatand move forward.
2. What are we going to do? That’s right, get it out there. Shake it out. List the things you want to do. Yes, you’re going to do them. They are very important. They are! I think it’s really important at this stage to let people linger and make sure they see those things they want to do captured. These are the things that are going to happen.
3. What makes it hard to do those things? (“Barriers”) Like, really, why are they such big things that you felt the need to articulate them so badly. There’s lots of important work that just happens, so why spotlight those things? What are the problems you’re trying to overcome by spotlighting them? This is usually like resource problems, or distractions, or actual disagreements between leaders, or some other problem that constantly steals focus. Remember, you’re not trying to justify the thing from the previous stage, you’re trying to explain why you would ever need to justify them in the first place. Why can’t the work just happen?
4. Of those things that make it hard, what do you have control over? It’s possible to work this question into the previous stage, but I like to allow people to let loose with all their grievances before trying to narrow them down slightly. This is a good spot though to fully divorce people from the “what I want to do”-brainspace by engaging them with something else to problem solve around….
5. Of the barriers, what can you do to overcome or route around them? Here we go, this is what we’ve been working towards: solving problems. What actually would address the problems we identified in the previous stage? Write that down.
6. Of those solutions, what are you going to do? (“The Strategics”) This is the decisional commitment. Remember, we haven’t replaced the initial stage’s “what are we going to do’s”—those are still happening—but rather worked towards a new list of “everything would sure be a lot better/faster/stronger if we did this first or in-addition-to”.

…and then you write that into the planning template to report it out. And in this case, the “justification” for doing The Strategics is listing all of the stuff that you were going to do anyway.

But you can do this badly too, right?

Sure of course, we can do anything badly. It’s possible to wash the initial “what are are we going to do’s” through this entire process and end up exactly where you started, but with several hundred words of retcon around it. That happens.

…but at least I believe this process affords some potential of not doing that, whereas I think most default planning processes (fill out the template) almost guarantee you end up where you started.

Therefore, I’ve been working in a fresh Ruby on Rails app full-time now for 1 year. One of the Rails pillars is “convention over configuration”, so I thought it would be fun to share what has so far accumulated in my app’s /config directory: monkeypatches, extensions, and appwide behaviors.

Let’s start with the most controversial one.

Object#not_nil? and boolean extensions

# config/extensions/ext_object_boolean_nil.rb

class Object
  def not_nil?
    !nil?
  end

  def false?
    false
  end

  def true?
    false
  end
end

class FalseClass
  def false?
    true
  end
end

class TrueClass
  def true?
    true
  end
end

I realize there are PhDs written about the evils of null. I DISAGREE. I am continually confronted with the need to distnguish between “Yes”, “No”, and “has not answered the question yet” in my my wide models. I want a simple predicate pair like present?/blank? for nil , hence nil? /not_nil? .

And then you can see where that led me wanting to distinguish between truthy and falsey and true true and false false. Predicates are great!

timestamptz default

# config/initializers/active_record_timezones.rb

ActiveSupport.on_load(:active_record_postgresqladapter) do
  ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.datetime_type = :timestamptz
end

I write my database migrations using table.datetime and it generates timestamptz columns. I guess this is fine.

Capybara.threadsafe

# config/initializers/capybara.rb

Capybara.threadsafe = true

I use Capybara as my harness for webdriving automations, rather than writing raw Selenium (never again!).

Custom Types

# config/initializers/custom_types.rb

Rails.application.config.to_prepare do
  ActiveRecord::Type.register(:phone_number, PhoneNumber::Attribute)
  ActiveModel::Type.register(:phone_number, PhoneNumber::Attribute)
end

module Kernel
  def PhoneNumber(value) # rubocop:disable Naming/MethodName
    value.is_a?(PhoneNumber) ? value : PhoneNumber.new(value)
  end
end

There are several annoyances here:

• I have to register custom attributes twice: once for Active Record and once for Active Model.
• I want my types to live in /app, and be autoloaded, but there isn’t a lifecycle hook for that

And one celebration: I love Ruby and being able to coerce withPhoneNumber(something)

dom_target

# config/initializers/extend_action_view_record_identifier.rb

module ActionView
  module RecordIdentifier
    def dom_target(*objects)
      objects.map do |object|
        if object.is_a?(Symbol) || object.is_a?(String)
          object
        elsif object.is_a?(Class)
          dom_class(object)
        else
          dom_id(object)
        end
      end.join(JOIN)
    end
  end
end

I write a lot of Turbo and I want to be able to chain together an unlimited list of identifiers like (:admin, client, :outbound, Message, :new) to produce something like admin_client_24_outbound_message_new to pair up Turbo Streams and Broadcasts. I upstreamed this into Rails 8.1 so this is unecessary now🎉

field_error_proc

# config/initializers/form_errors.rb

ActiveSupport.on_load(:action_view) do
  ActionView::Base.field_error_proc = proc do |html_tag, _instance_tag|
    html_tag
  end
end

Still necessary 🫥

GoodJob and UUIDv7

# config/initializers/good_job.rb

Rails.application.configure do
  config.good_job.execution_mode = :inline if Rails.env.test?
end

GoodJob.preserve_job_records = true
GoodJob.on_thread_error = ->(exception) { Appsignal.send_error(exception) }

ActiveSupport.on_load(:action_mailer) do
  ActionMailer::MailDeliveryJob.retry_on StandardError, wait: :polynomially_longer, attempts: Float::INFINITY
  ActionMailer::MailDeliveryJob.discard_on ActiveJob::DeserializationError
end

# **SNIP** Lots of GoodJob cron config

module ActiveJobUUIDv7
  def initialize(*args)
    super
    @job_id = SecureRandom.uuid_v7
  end
  ruby2_keywords(:initialize)
end

ActiveSupport.on_load(:active_job) do
  include ActiveJobUUIDv7
end

ActiveSupport.on_load(:good_job_execution) do
  before_create { self.id ||= SecureRandom.uuid_v7 }
end

ActiveSupport.on_load(:good_job_process) do
  before_create { self.id ||= SecureRandom.uuid_v7 }
end

ActiveSupport.on_load(:good_job_batch_record) do
  before_create { self.id ||= SecureRandom.uuid_v7 }
end

ActiveSupport.on_load(:good_job_setting) do
  before_create { self.id ||= SecureRandom.uuid_v7 }
end

A surprising misconception I’ve run across with folks using GoodJob is that using more configuration is better. Less less less.

Specifically at the bottom, I’ve patched GoodJob and Active Job to use UUIDv7 to see if it’s any better. The verdict is still out. It’s not worse!

I18n verification

# config/initializers/i18n_verify

return unless Rails.configuration.i18n.raise_on_missing_translations

IGNORED_I18N_KEYS = %w[
  activerecord.attributes.client.reports
  ...
].freeze

module I18n
  class << self
    alias original_translate t

    def translate(key, **options)
      begin
        original_fallbacks = I18n.fallbacks
        I18n.fallbacks = I18n.available_locales.index_with { |locale| [locale] }

        ignored = IGNORED_I18N_KEYS.any? { |k| [options[:scope], key].compact.join(".").start_with?(k) }
        unless ignored
          available_locales.without(I18n.locale).each do |locale|
            with_locale(locale) { original_translate(key, **options, raise: true) }
          end
        end
      ensure
        I18n.fallbacks = original_fallbacks
      end

      original_translate(key, **options)
    end
  end
end

This is a fun patch that causes I18n.raise_on_missing_translations to raise on any missing translation across all available locales, not just the current locale.

i18n_tasks / Psych YAML mangling

# config/initializers/i18n_tasks_normalize.rb

# Drop this at the top of config/i18n_tasks.yml too:
# <% require "./config/i18n_tasks_normalize" %>

module I18nTaskYamlExt
  UNMASKED_EMOJI = /
    (?:
      (?:\p{Emoji_Presentation}|\p{Emoji}\uFE0F)   # base emoji
      (?:\u200D(?:\p{Emoji_Presentation}|\p{Emoji}\uFE0F))* # + ZWJ parts
    )
  /ux

  def dump(tree, options)
    builder = Psych::Visitors::YAMLTree.create
    builder << tree
    ast = builder.tree
    _process_node(ast)
    strip_trailing_spaces(restore_emojis(ast.to_yaml(nil, options || {})))
  end

  private

  def _process_node(node)
    case node
    when Psych::Nodes::Scalar
      node.plain = false
      node.quoted = true
      node.style = node.value.include?("\n") ? Psych::Nodes::Scalar::LITERAL : Psych::Nodes::Scalar::DOUBLE_QUOTED # <== THE ENTIRE PURPOSE OF THIS MONKEYPATCH 🫠
      node.value = _mask_emoji(node.value) if node.style == Psych::Nodes::Scalar::LITERAL # pre-mask emoji because otherwise libyaml will double-quote when we want literal style
    when Psych::Nodes::Mapping
      # only process the values, not the keys
      node.children.each_slice(2) { |_key, value| _process_node(value) }
    when Psych::Nodes::Stream, Psych::Nodes::Document, Psych::Nodes::Sequence
      node.children.each { |node| _process_node(node) }
    else
      raise "not handling #{node.inspect}"
    end
  end

  # libyaml will do this, but we want to do it first so that libyaml doesn't _also_
  # mark the node as unprintable and thus prevent it from being in a literal
  # https://github.com/yaml/libyaml/issues/279
  # "Hello 👋 world 🌍!" => "Hello \\u0001F44B world \\u0001F30D!"
  def _mask_emoji(string)
    string.gsub(UNMASKED_EMOJI) do |emoji|
      emoji.codepoints
        .map { |cp| format('\\u%08X', cp) }
        .join
    end
  end
end

I18n::Tasks::Data::Adapter::YamlAdapter.singleton_class.prepend I18nTaskYamlExt

Ok, i18n_tasks is absolutely essential for doing localization. But it works by roundtripping your YAML through Psych. This patch does 2 things:

• It makes Psych output every string as strictly either a doublequoted string or, if it contains a newline, as literal-block (key: |). Without the patch, Psych will swap around single and doublequoted strings and convert literal-blocks to double-quoted strings with \ns
• Emojis don’t get mangled.

Custom mailers

# config/initializers/mailers.rb

ActiveSupport.on_load(:action_mailer) do
  ActionMailer::Base.add_delivery_method :twilio_sms, TwilioSmsDelivery
  ActionMailer::Base.add_delivery_method :null, NullDelivery
end

I’m one of the rare true-fans of Action Mailer who wants to deliver messages to more channels via Action Mailer, not none. Our app uses Action Mailer to construct and deliver SMS messages too via a custom delivery method. I’ve touched on this previously.

Markdown

# config/initializers/markdown.rb

module Markdown
  def self.convert(text = nil, **options)
    raise ArgumentError, "Can't provide both text and block" if text && block_given?

    text = yield if block_given?
    return "" unless text

    text = text.to_s.strip_heredoc
    options = options.reverse_merge(
      auto_ids: false,
      smart_quotes: ["apos", "apos", "quot", "quot"], # disable smart quotes
      input: 'GFM',
      hard_wrap: true # turn single newlines into <br />
    )
    Kramdown::Document.new(text, options).to_html
  end

  def self.inline(text = nil, **)
    # Custom input parser defined in Kramdown::Parser::Inline
    convert(text, input: "Inline", **).strip
  end

  module HtmlSafeTranslationExt
    def html_safe_translation_key?(key)
      key.to_s.end_with?("_md") || super
    end
  end

  module I18nBackendExt
    def translate(locale, key, options)
      result = super
      # Rails missing key returns as MISSING_TRANSLATION => (2**60) => -1152921504606846976
      if key.to_s.end_with?("_md") && result.is_a?(String)
        if result.include?("\n")
          Markdown.convert(result)
        else
          Markdown.inline(result)
        end
      else
        result
      end
    end
  end
end

ActiveSupport::HtmlSafeTranslation.prepend Markdown::HtmlSafeTranslationExt
ActiveSupport.on_load(:i18n) do
  I18n.backend.class.prepend Markdown::I18nBackendExt
end

# Generate HTML from Markdown without any block-level elements (p, etc.)
# http://stackoverflow.com/a/30468100/241735
module Kramdown
  module Parser
    class Inline < Kramdown::Parser::Kramdown
      def initialize(source, options)
        super
        @block_parsers = []
      end
    end
  end
end

There’s two features here:

• Markdown.convert(string) and Markdown.inline(string) are simple helpers that turn a string of markdown formatted text into html
• The entirety rest of the code duplicates Rails i18n behavior for html-safing _html keys. Any i18n key that ends in _md will convert the value from markdown to html, and then mark it as html safe. I love authoring complex i18n keys in markdown (especially when i18n_tasks/Psych doesn’t mangle the newlines, see previously).

rbtrace

# config/initializers/rbtrace.rb
if ENV['RBTRACE']
  require 'rbtrace'

  $stdout.puts "Enabled rbtrace. Example commands:"
  $stdout.puts "- Show all method calls: $ bundle exec rbtrace --pid #{Process.pid} --firehose"
  $stdout.puts "- Debug Rails deadlock: $ bundle exec rbtrace --pid #{Process.pid} --eval \"puts output = ActionDispatch::DebugLocks.new(nil).send(:render_details, nil); output\""
  $stdout.puts <<~TEXT
    - Show all threads: $ bundle exec rbtrace --pid #{Process.pid} --eval "ObjectSpace.each_object(::Thread).to_a.each { puts(it, it.backtrace_locations, '------') }; nil"
  TEXT
  $stdout.puts "- Heap Dump: $ bundle exec rbtrace --pid #{Process.pid} --eval 'Thread.new{require \"objspace\"; GC.start; io=File.open(\"tmp/ruby-heap.\#{Time.now.to_i}.dump\", \"w\"); ObjectSpace.dump_all(output: io); io.close }'"
  $stdout.puts 'Press Enter to continue...'
  $stdin.gets
end

I have a terrible memory for debugging commands. so this is how I document them. I pair this with a bin/productionrails script that allows me to boot a Rails in production-mode locally by setting up all the necessary environment variables with placeholder values, and disabling https and so forth.

Sprocket Reproducible Assets

# config/initializers/sprockets.rb

Rails.application.configure do
  config.assets.gzip = false
end

Gzipped Sprocket include the build time and thus are uncacheable across builds. I’m waiting for someone to look at my PR.🤷

Turbo Stream jobs

# config/initializers/turbo.rb

ActiveSupport.on_load(:active_job) do
  Turbo::Streams::BroadcastJob.queue_name = "turbo"
  Turbo::Streams::ActionBroadcastJob.queue_name = "turbo"
  Turbo::Streams::BroadcastStreamJob.queue_name = "turbo"
end

Somone should make this a configuration option 🫵

View Components that can broadcast_later

# config/initializers/view_component.rb

# Instantiate a ViewComponents that is (optionally) serializable by Active Job
# but otherwise behaves like a normal ViewComponent. This allows it to be passed
# as a renderable into `broadcast_action_later_to`.
#
# To use, include the `ViewComponent::Serializable` concern:
#
#  class ApplicationComponent < ViewComponent::Base
#    include ViewComponent::Serializable
#  end
#
# And then call `serializable` instead of `new` when instantiating:
#
#   Turbo::StreamsChannel.broadcast_action_later_to(
#     :admin, client, :messages,
#     action: :update,
#     target: ActionView::RecordIdentifier.dom_id(client, :messages),
#     renderable: MessageComponent.serializable(message: message)
#   )
#
module ViewComponent
  module Serializable
    extend ActiveSupport::Concern

    included do
      attr_reader :serializable_args
    end

    class_methods do
      def serializable(*args)
        new(*args).tap do |instance|
          instance.instance_variable_set(:@serializable_args, args)
        end
      end
      ruby2_keywords(:serializable)
    end
  end
end

class ViewComponentSerializer < ActiveJob::Serializers::ObjectSerializer
  def klass
    ViewComponent::Base
  end

  def serialize?(argument)
    argument.is_a?(ViewComponent::Base) && argument.respond_to?(:serializable_args)
  end

  def serialize(view_component)
    super(
      "component" => view_component.class.name,
      "arguments" => ActiveJob::Arguments.serialize(view_component.serializable_args),
    )
  end

  def deserialize(hash)
    hash["component"].safe_constantize&.new(*ActiveJob::Arguments.deserialize(hash["arguments"]))
  end

  ActiveJob::Serializers.add_serializers(self)
end

I only use View Components in the complex parts of my app, and the complex parts of my app are complex because they do a lot of Turbo Streaming and asynchronous updating. I shared this upstream with View Component and it’s happening, though I already can imagine ways to do this better.

Git Worktree Support

This set of things is rather sprawling. I have a GitWorktree class that fetches the current Git Worktree name, and then uses either the string, or a deterministic integer, to prevent collisions during development and testing (itself with parallel_tests) across multiple worktrees concurrently:

# database.yml
development:
  database: frontdoor_development<%= GitWorktree.db_suffix %>
test:
  development: database: frontdoor_test<%= GitWorktree.db_suffix %><%= ENV["TEST_ENV_NUMBER"].then { it.present? ? "_#{it}" : "" } %>

# puma.rb
port ENV.fetch("PORT", GitWorktree.integer(3000..3999))

# application.rb
config.x.session_prefix = "frontdoor"
if Rails.env.development?
  port = ENV.fetch("PORT", GitWorktree.integer(3000..3999))
  config.x.session_prefix = "frontdoor_#{port}"
end
config.session_store :cookie_store, key: "_#{config.x.session_prefix}_session"

# spec/support/capybara.rb
Capybara.server_port = GitWorktree.integer(4000..4990, stride: ENV.fetch("PARALLEL_TEST_GROUPS", 1).to_i) + ENV.fetch("TEST_ENV_NUMBER", 0).to_i

Enumerating states Turbo can be in

Turbo adds some attributes to the webpage DOM that we can look for in our tests to ensure that things are in the right state. But it’s tricky. Let’s start with the trickiest thing first.

Turbo sets aria-busy when navigation is happening:

• Turbo Drive sets html[aria-busy] when navigating
• Turbo Frames set turbo-frame[aria-busy] when navigating
• Turbo forms set form[aria-busy] when submitting

…but when doing a very typical Submit-a-Form-and-Redirect operation, Turbo drops the [aria-busy] for a brief moment after the form submit returns a redirect response but before navigating to the redirected page sets the [aria-busy] again. That sucks.

The other things we need to be aware of and wait for are more pedestrian:

• The first time a page is loaded and Turbo hasn’t been loaded at all yet. This probably wouldn’t matter because, of course, we’re all using progressive enhancement so nothing is functionally inaccessible when javascript isn’t loaded. But our tests should be deterministic. And we have the non-browser Rack Test driver to cover the non-javascript scenario (which are super fast too). To handle this scenario, we’ll add an attribute to our server-rendered HTML layout, and then remove it after Turbo has loaded.
• Turbo Previews. Which is like a weird cached version of the page Turbo will show, but might get yanked away when the actual response finishes and Turbo swaps them. Fortunately turbo adds a data-turbo-preview attribute we can test for.

Doing the simplest thing about it

Let’s start with the test helper, and then we’ll wire up the pieces:

def wait_for_turbo
  return unless Capybara.current_driver != :rack_test

  page.assert_no_selector "html[aria-busy], form[aria-busy], turbo-frame[aria-busy], html[data-turbo-not-loaded], html[data-turbo-loading], html[data-turbo-preview]", visible: :all
end

Not bad! It simply waits, because assert_no_selector uses Capybara’s async timeout, until the page doesn’t have any of the “I’m loading” attributes on it. Easy.

To handle the turbo-not-loaded attribute, add something like this to your layout(s):

<!-- app/views/layouts/application.html.erb -->
<html data-turbo-not-loaded="1">

And then in your javascript, at the very bottom after you’ve loaded Turbo and everything else:

// app/javascript/application.js

import "@hotwired/turbo-rails"
// ...

document.addEventListener("turbo:load", function() {
  document.documentElement.removeAttribute("data-turbo-not-loaded")
  document.documentElement.removeAttribute("data-turbo-loading")
})

document.addEventListener("turbo:submit-start", function() {
  if (!event.target.closest("turbo-frame") {
    document.documentElement.setAttribute("data-turbo-loading", "1")
  }
})
document.addEventListener("turbo:submit-end", function(event) {
   if (!event.detail.fetchResponse?.redirected) {
    document.documentElement.removeAttribute("data-turbo-loading")
  }
})

The carve-out logic for setting data-turbo-loading on turbo:submit-start and removing it on turbo:submit-end is a bit complicated:

• We don’t want to set data-turbo-loading on Turbo Frames because Turbo Frames don’t emit turbo:load events so the attribute would never be removed. More importantly and fortunately, Turbo Frames don’t suffer from an aria-busy gap.
• And we don’t want to remove data-turbo-loading if it’s a redirect, to bridge the aria-busy gap mentioned earlier. We’ll wait for the subsequent turbo:load to clean it up.

There it is. Now you just need to put wait_for_turbo before any action that could potentially engage Turbo e.g.

# ...
click_on "Do something"
wait_for_turbo
click_on "The result"
#...

I hate it

Yeah, that’s awful. We want our system tests to invoke declarative behavior, not be littered with “wait for it….”. So… we’re going to patch Capybara. Here we go…:

module SystemTurboClick
  def click(...)
    unless session.driver.is_a?(Capybara::RackTest::Driver)
      assert_no_ancestor("[aria-busy], html[data-turbo-not-loaded], html[data-turbo-loading], html[data-turbo-preview]", visible: :all)
    end
    super
  end
end

Capybara::Node::Element.prepend SystemTurboClick

Now anytime that Capybara does a click event, it will first wait for Turbo to do its thing.

I spent that Spring Break week on campus alone with my thoughts, as any romantic 20-year old I imagine would be. The woman I had just begun dating, who I had met in that same university class, was traveling in Germany. That relationship would last another three years; the Iraq war would last eight.

I did not sign up. My best friend had been exercising with the ROTC cadets. I remember him explaining that the army’s current slogan “Army of ONE” was terrible. He not sign his paperwork. He lives in Germany now.

Another member of our same high school friend group did serve. While on leave later that year, he would kill a man outside his hotel, presumed to have been caught stealing from his car, and then would kill himself.

My grandfather served in the Army, in Alaska, during World War 2. The family lore is that while courting my grandmother, he briefly snuck off base for a date. My grandfather being the model of a loyal, hardworking, reliable private, caused the base commander to exclaim: “AWOL? Not Jerome!”

My other grandfather served during the last days of World War One in his native Hungary, before Trianon. He would go to Spain as an American communist to report on the civil war against fascism. He came back a liberal anti-communist and wrote a book about it. The US government would later prevent him from returning to Hungary to cover the revolution against the Soviets for Newsweek.

My dad fell off a cable car as a teenager and messed up his knees. That led to a draft deferment. His knees still cause him pain.

My uncle was drafted and served in Vietnam. We don’t talk about it.

I can check online to see that yes, I did register for Selective Service a week after my 18th birthday.

My father-in-law served in the US Army in Germany after Korea and before Vietnam, paving the way for his US citizenship. His father served in the Chaco War, which would kill 2% of the population of his home nation, Bolivia.

When I was 5, we hosted a family from the Soviet Union through MEND, Mothers Embracing Nuclear Disarmament. One of their gifts was a matryoshka nesting doll. It sits on my piano today.

During my last year working at UMass Boston’s College of Public and Community Service, 2011-ish, I started taking my lunch on a newly-installed bench overlooking Dorchester Bay. The bench had a plaque remembering a US soldier killed in Kandahar Province, Afghanistan.

In the acknowledgements of Spencer Ackerman’s book, Reign of Terror, he writes:

Yes, I know this book is incomplete.

There’s not enough in Reign about the pivotal role of the media in manufacturing consent for the War on Terror. There’s nowhere near enough about the economic forces driving the war, both specific and structural. Ultimately, I found that attempting either analysis swallowed the actual narrative events of the war and still felt superficial. These subjects require books in their own right—particularly one I’m tossing around in my head called Capitalism and Terrorism. (That book is the forum to go into the U.S.–Saudi relationship and its place in all this, I found . . . after failing to get it into Reign satisfactorily.) I chose to attempt to do one thing well instead of three things badly.

That leads to the second way Reign is incomplete. I had to make a lot of cuts. They were a function of Reign’s particular critique, as well as the unavoidable reality that I have a twenty-year palette of events and a contractually stipulated word limit. (Someone have me on their podcast to talk about Michelle Malkin bullying Dunkin’ Donuts because she was mad Rachael Ray appeared in a commercial wearing a scarf that looked kind of like a kaffiyeh.) Even after all those cuts, at all times writing this book, a voice in my head objects that I’m presenting the “We Didn’t Start the Fire” version of the Forever War. So if I didn’t dwell long enough on events you think are crucial, know that I tried, and please research and write your own versions that outdo mine.

The third way Reign is incomplete is that much of what I write about remains an official secret. The façade of the War on Terror has been cracked, but we won’t know what the war truly was for decades. I frankly don’t think anything we discover will undermine the critique of Reign. I expect what we learn to reinforce my critique. Consider that at least eighteen hundred photographs of military torture have been barred from public release after a yearslong legal battle. Not even Dan Jones’s torture report could tell the story of CIA renditions, operations that we know applied to more people than the CIA directly jailed and tortured. Some aspects of the War on Terror are likely lost to history forever.

Seriously. If you are blogging, a prompt is good enough. It’s great! If you’ve got a thought or idea that you want to share with the world, just post that idea. It’s enough.

Jon’s post caused me to reflect on my own writing process. Over the course of the nearly 20 years (!) I’ve been writing here, I’ve become much more comfortable incorporating my initial “gosh, I have this thing to say in my head and I want to get it out”-compulsion into the the thing I evetually write. Just spit it out.

These abstracts statements are more “I want to tell you about …” and less “I want to write a longform blog post that tells you about…”

• I want to tell you about this thing I’m proud of, and make you be proud of it too.
• I want to tell you about the shape of a thing I see, so I can ask if you see it too.
• I want to tell you a story in the canon of my life, and I want you to understand why it’s meaningful to me rather than simply dumb or sad or cruel.

The form doesn’t matter to the want. The da-dum da-dum rhythm of heading and sentences and commas that results is just how I’ve come to spit out my written words. Don’t worry about that. Post your want.

What I mean to say is: this is working for me; I recommend you try it.

Becoming aware

I guess this is a post about writing. When you want to become a better writer, you get told to read more. I don’t think that’s quite right, or not the point I’d emphasize. Instead I’d say: When you want to write better, you should pay more attention when you’re reading to whether what you’re reading works for you and then pause a beat, and become aware of that feeling and, momentarily or however long you want, try to figure out why. …or why not.

Sure, self-awareness is a skill that develops alongside reading more, but I don’t see the point to recommend reading more if you aren’t also becoming more aware of whether you like it or are responding to what you’re reading.

That sort of awareness can be hard because the vast majority of what shows up in front of me is intended to be informational. Work memos, and news reports, and the like. It’s not intended to hit or feel, and usually is intended specifically not to make me feel. But I can ask myself: was this boring-ass memo more or less effective than those other boring-ass memo at achieving its purpose? And why. And recognizing that the purpose isn’t clear at all is a feel too. What is this shit?

…and on that last negative reaction: I personally find it harder to split the mildy good writing from the mildy bad writing. Really good and really bad is obvious, but also rare; most stuff is mid. A mild disgust or boredom or frustration is easier for me to identify than mild understanding or mild respect. “Oh, I wasn’t aware of that before and I now I am” or “oh, this person knows what they’re talking about in an everyday way” is harder for me to sus than “huh?” and “what?” and “ok, whatever that was”

Mildly good itself is a totally reasonable target. I should be so lucky to be surrounded by and produce myself mildly good writing. Mildly good!

My answer for mildly good

So, the entire previous section was me recommending that you develop your own personal sense of awareness and taste for writing that works for you.

This section is where I tell you what works for me: Specificity and economy. Say the things themselves, and do it directly without a lot of leadup or padding out.

• “Specificity is the soul of narrative”, attributed to John Hodman via “Why Olivia Rodrigo’s “Drivers License” Had the Biggest Debut Since “WAP”.
• “If I had more time, I would have written a shorter letter.” No one knows where that one comes from, but it’s true: writing clearly, directly, and efficiently is the hard part of writing well.

Because this is the section where I laundry list the things I really want to share. And because everything else was a leadup to explain why you might spend time reading it… here is a personal example. This is the kind of thing my dad posts, all the time, on Facebook. It’s mildly perfect.

Yesterday morning Constance and I watched the funeral of Prince Philip starting at 6:30am. She has this button of him, and I have this souvenir of the Royal Visit to Whitehorse in 1959. My father was to cover it for Newsweek. He got press credentials for both of us from Ottawa. I was to be his assistant. I was 12 years old.

We flew down to Whitehorse from Fairbanks on a Lions Club charter flight. We were at the airport for the arrival. The press plane came first, and then the plane with Queen Elizabeth and Prince Philip. The stairs were all folded up inside the front door of the plane. They were deployed and the Queen and Prince Philip descended. A First Nations (that phrase was not used then) girl gave the Queen some flowers, and local dignataries were presented. By now the press were pretty jaded, and they just stood and watched. Other noteworthy events were the ride on the White Pass & Yukon train and the church service. After a visit to the McBride Museum the Royal Couple boarded the train and stood on the rear platform, waving to the crowded. They were talking to each other out of the corner of their mouths. The press boarded and the train departed. The royal party was in the last car and the press were in the leading 3 or 4 cars. We stopped at a cross road outside of town and Queen Elizabeth and Prince Philip got into a waiting car to go back into town.

The train continued on to Carcross, where there was an open bar to the press. Someone offered to take my father and me for a boat ride on the lake. We did that. On another day there was to be a trip to Dawson City. The Queen did not feel well and only Philip went. When he returned he was piloting, and landed the plane. There were episodes on “The Crown” where he was learning to fly. When they returned to London it was announced she was pregnant. That was Prince Andrew.

The other event I remember was the service at the log church. There were a lot of Canadian army officers standing around in their uniforms with all their medals. There were some with service medals going back to George V. For some reason I crossed a security line. An army officer stopped me, and I pointed to my badge. He studied it, and said “I don’t know why they are giving press credentials to 12 year olds now.” He let me through.

Detail, detail, detail, and a joke at the end. It’s arguably a shaggy dog story leading up to the humor, but it works because there isn’t extraneous structure or padding or filler. In the comments, my dad even found a film archive that has the two of them in it (at 8:54). It means something.

Back to the prompt

I want to read what is personally meaningful to you. At a minimum I’m curious, and more often, I care. I wouldn’t bother reading it otherwise.

So don’t fuck around having an LLM wrap what you intend to say in meaninglesless and next-token statistical noise. Go at it with a ruthless knife. Say what you want to say. Write it down. Tell me. I’ll put it in my RSS reader.

And thank you Jon for writing the much longer, more thoughtful piece that inspired this, and the discussion in the Ruby on Rails Performance Slack where I mostly spat all this out already.

Due to what might poetically be called “preconceived notions” baked into a coding model’s neural network (more technically, statistical semantic associations), it can be difficult to get AI agents to create truly novel things, even if you carefully spell out what you want.
For example, I spent four days trying to get Claude Code to create an Atari 800 version of my HTML game Violent Checkers, but it had trouble because in the game’s design, the squares on the checkerboard don’t matter beyond their starting positions. No matter how many times I told the agent (and made notes in my Claude project files), it would come back to trying to center the pieces to the squares, snap them within squares, or use the squares as a logical basis of the game’s calculations when they should really just form a background image.

To get around this in the Atari 800 version, I started over and told Claude that I was creating a game with a UFO (instead of a circular checker piece) flying over a field of adjacent squares—never once mentioning the words “checker,” “checkerboard,” or “checkers.” With that approach, I got the results I wanted.

I’ve experienced something similar in my own AI-assisted game design. In my case, it’s my perennial desire to have a Simcity-like city builder that operates on “parcels” rather than tiles or zones and, oh boy, the LLM really chafes at it. Is it weird enough to work?

In this post, I’m writing my personal perspective on how you, dear developer, might decide what background job backend to choose for Rails and Active Job. But everything in context. And oh boy, it’s all context.

A sober look at how technical decisions are actually made

I’ve worked in software engineering a long time and seen a lot of bullshit justifications for… doing stuff.

• It’s “more modern”, meaning newer, I guess.
• It has more features, or a unique architecture.
• Everybody’s doing it. Or maybe, this particular someone is doing it. Or nobody is doing it, except for the successful ones.

All of this has some flavor of: I want to be different than everyody else, but also the same as the people who are succeeding.

There’s an assumption that the generation of knowledge is efficient. Meaning that all possible problems and solutions have been rigorously measured and stress-tested and their upsides and downsides unearthed; that people of equivalent smarts have spent equivalent time running out equivalent unflawed methodologies and sharing the results across equivalent channels. And our work is to place that preexisting knowledge on the platters of a scale and note which way it tips.

Also true: familiarity breeds contempt. The grass is always greener. And despite things being inherently complex, where the goal should be less “work your way out of the job” and more “don’t work yourself into an early grave”, your managers—whether actual people or your own intrusive thoughts—will not accept that.

Alternatively, I know, from such evergreen classics like Glass’s Facts and Fallacies of Software Engineering and Weinberg’s The Psychology of Computer Programming, it’s rare to deeply track our own technical and operational bottlenecks, the full business contexts, the skills and resources of the team and the broader talent market, and integrate those details into the decision-making process. Most folks start with the solution they want, and work backwards in their argumentation through whatever technical decision process their engineering organization requires. No one shows their work ’cause there’s no work to show; they read the assignment and the answer just popped out.

But not you, of course.

There’s another dimension to this that I think is really important to note: Decision making for new projects is vastly different than existing projects.

• With New Projects, the decisions don’t really deeply matter cause nobody knows how it’s going to go and everything is speculative. But also because of bikeshedding and the Law of Triviality those decisions become the most forcefully argued over. It’s really hard for a room of technical people to plainly say they just want to do it one way cause they know it, or try something different for the hell of it and then figure out an effective social compromise instead of debating to death some hardboiled technical justification.
• With Existing Projects, you have some data and some context and continuity to base your needs and your decisions on. But there is also an inexorable pressure to do something different and you end up picking a solution-at-hand rather than more deeply digging into a problem area where the solutions are more elusive in the sense of “I’m searching for my keys under the streetlight not because I lost them nearby but because the light is better”.

Bringing it back to Rails and Active Job, let’s look at a really big differentiator.

Omakase

Solid Queue is a default gem in Rails. If you run rails new, you get Solid Queue. That’s a strong signal that it will work, and that the Rails team confidently stands behind it as much as anything else in the Rails framework. Solid Queue is Ruby on Rails’ chefs’ choice made for you; it’s omakase.

“Eliminate valueless choices” is a powerful principle, as part of the Rails Manifesto. Solid Queue is a good choice, especially if you don’t have to choose. This alone is an incredibly powerful reason to Solid Queue. You probably chose Rails for the lore. There’s a manifesto. And it’s actually good. Conceptual integrity is important. Make the center hold.

But if you’ve been following the Rails discourse for a while, you’ll know not everyone agrees that omakase is best, or best for them. Lots of people quietly (and loudly!) go their own way. Callbacks are the worst, they comment underneath every single Reddit post. Nobody actually uses ActionMailer. Why isn’t there a Services directory? That pattern encourages methods that are too long! All those Concerns, are you for real? Nobody does it that way anymore. It’s not modern.

More recently 37 Signals has started releasing source code for several of their applications, and yes, they’re for real and yes, they do still do it that way. But Rails is a big framework (“Batteries included”) and you don’t have to use everything (“No one paradigm”). You can swap something out and remain lore canon.

And really, Solid Queue slots into a real sweet spot. It’s both new and it’s the default. Solid Queue can occupy the unicorn stall that’s both excitingly cutting edge and boringly conservative all at once. This will be a different discussion 5 years from now even if nothing has changed other than the year on the calendar.

But, maybe you do need to order off the menu! And we should dig into that.

Incomparables

Given the infinite rainbow of the human experience, there has to be somebody who does not collapse their categories. I have yet to meet them.

People want a simple decision tree that starts with “Postgres, MySQL, SQLite or Redis?” It’s not so simple.

• Postgres on Heroku? Digital Ocean? Neon? Fly’s umpteenth attempt at it? Self-managed? How? Co-located?
• MySQL? MariaDB? Behind Vitess? Planetscale?
• SQLite? Litestream? Did you apply the good patches? How’s your IO?
• Redis? HA? How? You didn’t choose “Flex” did you?

And any one of these can have the DBA who swears by that one deep-cut config and won’t budge. And while there is wisdom in sinking engineer-years into completely rearchitecting your system to sidestep a problematic technical peer—oh, I’ve done it—there’s no way you’re gonna write those words in the justification doc. You’re slapping down “more modern” and your junior (or junior-minded) colleague is tweeting “more modern” and HackerNews will cite that a decade from now.

Knowledge is not efficient. You’ll always find another subfloor and several sheer walls of complexity below and beyond whichever one you just pried open. This stuff is not enumerable, so any good decision tree you’ll receive will, on its surface, appear to be a gross and useless simplification despite the experience that went into it.

In conclusion

As we’ve deeply explored together, you find yourself in one of two scenarios regarding background jobs:

• The backend doesn’t matter. Pick whatever, or pick nothing and use the default. Give whatever nonsense justification you want that serves you best. No shame, all respect if you can recognize this for yourself. You can stop reading now (I know, we love the lore).
• It does actually matter, a lot. And successfully navigating this will be the crowning achievement of your career.

If you, dear developer, find yourself in the second bucket, here’s my advice.

• If performance matters, like actually matters, use Sidekiq Enterprise or Karafka. But it’s complicated, you comment. It costs money. No one uses it anymore. It’s not modern. Shush. Sidekiq Enterprise and Karafka are, hands down the best for performance outside of building it in-house (a legit choice we won’t explore here). You can make it complicated by trying to precisely nail down what is meant by “performance” (throughput, latency, resources, spiky, steady, whatever), but if you use the word superlatively–“most performant”–your choice is made.
• Otherwise:
  • If you’re using MySQL or SQLite, use SolidQueue. it’s your best option for those databases, and it’s really good in an absolute sense too. And it’s getting even better. it should have GoodJob-esque batches soon. No shade, all love.
  • If you’re using Postgres, use GoodJob. It’s full featured, and those features are implemented simple-like because they use Postgres-only features; there’s no workaround complexity for other databases. Want to break all the rules of encapsulation and fiddle with the job data? It’s one table. FANG uses GoodJob. It’s modern.

Because I’m close to GoodJob (familiarity!) I know the limitations too. You’ll have to go around PgBouncer. One guy profiled Postgres several years ago and pointed out where in the performance stratosphere Advisory Locks fall apart. It’s all true.

And there is a lot of relational-database arcana that I don’t think GoodJob nor Solid Queue have documented that you could do. Like vacuuming your tuples and stuff like that. I don’t do it, but some people swear by it. Every system falls apart if you run it hot or weird for long enough, and relational databases are no different than Redis or Kafka.

I could laundry-list any number of downsides and problems people have had with GoodJob… and you probably won’t experience them. Rosa and Mike and Maciej can tell you the same about their own backends. If and when you do experience something bad, it’s likely you’d experience something else bad on any other job backend and then you’d have to work through the problem and it’s likely that problem is unique to you, either because of your unique setup, or a trivial, personal failure to have simply googled it earlier.

And to solve that problem, you’ll probably do some research. And ideally you have a strong community of peers that think like you think, and work on similar systems to the one you work on, and you talk to them and sometimes help solve their problems too. They won’t have your specific answer, but they’ll have some suggestions about what to look at and what to try looking at or googling for you, and maybe the change you make is to change your Active Job backend, but it probably isn’t.

So the takeaway here is that today in 2026 there are a small number of really, really good Active Job backends to choose from, and the most important consideration is that you make friends along the way.