Sorbet Journey

This is an ongoing series about adding Sorbet to a mature Rails code base. Follow me on Twitter to find out when updates are posted.

In This Episode of Sorbet Journey…

We’re a few weeks in working with Sorbet in our Rails mainline app and we’ve found there’s a natural kind of rhythm to the process. The task at hand is a refactor of the way a purchase’s line items are handled. This requires some big, wide-reaching changes so we’re taking some dev cycles to do some deep cleaning and tech-debt paydown.

(These big changes are what made Sorbet so appealing – it’s giving us more confidence around making big, wide-reaching changes.)

But First: Make Sure You Have Tests

Having a comprehensive test suite has been super important as part of adding Sorbet to our Rails application. T.must and friends raise exceptions on invalid data.

Many times we’ve gotten everything passing the srb tc typechecker beautifully only to have CI explode with hundreds of failures because T.must got a nil or some typed method got passed something from an untyped part of the code base.

This is a good thing, since Sorbet is catching undefined behaviour and forcing explicit code. But without comprehensive tests we would be having a bad time.

The Usual Steps

So, these are the steps happen pretty much every time we start adding Sorbet to a part of the app.

Step: Increase Strictness, Check for Breakage

We chose an area, set typed: true if it wasn’t already and then add one or two sig {} declarations in key locations. We’ll often start by typing the attr_readers and initializers since they affect most of the file.

class ParamsParser
+  extend T::Sig
+
+  sig { returns(T.any(LineItem::RelationType, T::Array[LineItem])) }
   attr_reader :line_items
+
+  sig { params(line_items: T.any(LineItem::RelationType, T::Array[LineItem])).void }
   def initialize(line_items)
     @line_items = line_items
   end
end

Step: Run sorbet-rails on Affected Models

We’re using sorbet-rails to get Rails and Sorbet playing nicely together. sorbet-rails lets you generate RBI files for models, controllers and other Rails files. In order to make our migration to Sorbet as low-impact as possible, we didn’t run sorbet-rails on many files to start. Just enough to get a feel for things.

Because Sorbet needs to know something about those files, it created basic stubs for all our models in the hidden-definitions RBI file. The auto-generated hidden-definitions are pretty good, but sorbet-rails has Rails-specific knowledge that make enums and relations work properly.

So, we’ll run sorbet-rails on specific models as we come across them as part of our daily work. This will generate a proper RBI file (e.g. sorbet/rails-rbi/models/your_model.rbi) for each model with enum types, relation types and knowledge of some of our model mixins like money-rails (more on that in a future post).

Step: Rewrite Callbacks

Our single most frequent change is with ActiveRecord callbacks. Sorbet doesn’t like Rails callbacks using procs (you can’t use instance methods in proc callbacks) so we rewrite them with actual methods.

class RedemptionCode < ApplicationRecord
-  before_save do
-    self.code = code.to_s.downcase
-  end
+  before_save :normalize_before_save
+
+  private
+
+  def normalize_before_save
+    self.code = code.to_s.downcase
+  end
end

Easy enough.

Step: Add Nil Guards

When we first tried Sorbet about a year ago our code ended up littered with T.must everywhere (you wrap T.must around code to promise Sorbet that it is not null). This time around I spent a bit of time learning about what Sorbet can know about code, specifically about what might be nil. Funny how learning the rules makes things easier…

# Here's an example...
class MyModel
  def ticket_count
    # we know this will _probably_ return the same value each time, but Sorbet doesn't
    get_from_database(ticket_count)
  end
end

record = MyModel.new

# This won't work with Sorbet nil checking since
# Sorbet calls `record.ticket_count` two times and
# doesn't do a nil check the second time
if record.ticket_count && record.ticket_count > 100
    # ...
end

# Assigning a variable and doing a T.must assertion is the
# way to Sorbet happiness
ticket_count = T.must(record.ticket_count)
if ticket_count > 100
# ...
end

Some code like this is still inevitable:

def not_started?
-    starts_at? && starts_at > Time.now
+    starts_at? && T.must(starts_at) > Time.now
end

But for many cases a well-placed variable assignment can limit the T.must clutter.

-  result = quantity_per_item[item_id]&.seat_ids.take(quantity)
+
+  seat_ids = T.must(quantity_per_item[item_id].seat_ids) if quantity_per_item[item_id]&.seat_ids
+  result = item.seat_ids.take(quantity)

Interesting Code Changes

Refactoring Hashes to a Proper Data Structure

One of the core refactors we have been tackling is taking a Hash of user input (modifying the Items in an Order) and turning it into a strictly defined data structure. We could have done this without Sorbet, but having Sorbet typecheck during development and throw errors as part of our test runs made this refactor much easier.

The data structure we ended up with was like this:

# typed: strict

# A container for parsed and sanitized "modify cart" requests.
# Usually created via ParseCartQuantityParams.
class CartItemParams < T::Struct
  extend T::Sig
  prop :quantity, Integer
  prop :seat_ids, T::Array[Integer], default: []
  # various other props and methods
end

Which were then created like this:

+ sig { params(item_id: Integer, param: T::Hash[Symbol, T.untyped]).returns(CartItemParams) }
  def parse_quantity_param(item_id, param)
+    # all the parsing and sanitizing rules for CartItemParams in one place
  end

And in dozens of files, we can now made changes like this:

module Seats
  class UpdateForOrderByQuantity
+   sig { params(item_quantity: CartItemParams).void }
    def update(item_quantity)
-      # lots of unconfident code like...
-      quantity = item_quantity.fetch(:quantity, nil)&.to_i
-      seat_ids = Array(item_quantity.fetch(:seat_ids, []]).compact.map(&:to_i).uniq
+
+      # and instead we can have our nice object
+      item_quantities.quantity
+      item_quantities.seat_ids.empty?
  end
end

Just to re-iterate, all of the above is possible without Sorbet. Sorbet just made it easier and gave us much more confidence that it actually worked.

View Helpers (Yuck)

Rails’ view helpers have always been a bit of a nasty part of our code base. That continues. At least now they’re nasty with types.

The problem is that our view helpers are unstructured. Our ApplicationHelper has code that deals with routing, timezones, currency formatting, date formatting and meta tags. I’m not sure what’s in BaseHelper, CoreHelper or StoreHelper. Maybe spiders?

Many of our helpers also call global controller helper methods, like current_user and current_store.

Adding typed: true to the top of most view helpers caused dozens of errors that we weren’t ready to tackle yet. Instead we did a bit of reorganizing. We moved the methods we were ready to add types to new, narrowly defined helper modules left the other ones as typed: false for now.

--- /dev/null
+++ b/app/helpers/currency_helper.rb
+module CurrencyHelper
+   sig { params(amount: T.nilable(Money), precision: Integer).returns(String) }
+   def currency(amount, precision: 2)
+     amount ||= Money.zero
+     number_to_currency(amount.round, precision: precision) || ''
+   end
+end

Unfortunately we currently use some of these methods in non-view parts of code. For example, that currency formatting method is called in some models’ to_json methods. So for now we have code like this:

class SomeModel
+  include CurrencyHelper
end

sorbet-rails creates a module with all of your helpers in it but we weren’t able to get it working. We’ll dig deeper when we start working on more code that uses helpers.

This whole area is a work in progress. It feels like Sorbet just surfaced a bunch of unpleasant code that already existed, so we don’t blame Sorbet for all this, but the best way forward isn’t immediately clear.

Converting Request Params to Hashes

Right near the end of this refactor we finally touched the boundary where some user input comes in: a JSON API endpoint. Our internal code had all been operating happily on a Hash with Symbol keys (i.e. T::Hash[Symbol, T.untyped]). but our API endpoint had ActionController::Parameters.

My first instinct was to update our signatures to accept parameters:

- sig { params(params: T::Hash[Symbol, T.untyped]) }
+ sig { params(params: T.any(T::Hash[Symbol, T.untyped], ActionController::Parameters)) }

But then I remembered HashWithIndifferentAccess and hashes with string keys and remebered all the subtle issues that come up with dealing hash types (e.g. to_unsafe_h, deep_symbolize_keys, etc…).

So instead we took the opportunity to lock down the hashes we accept. We left the Sorbet signature unchanged and turned ActionController::Parameters into a hash earlier:

- purchase_params = ActionController::Parameters.new(params).permit(*PURCHASE_PARAM_KEYS)
+ purchase_params = ActionController::Parameters.new(params).permit(*PURCHASE_PARAM_KEYS).to_h

We haven’t added Sorbet to any controllers yet, so this may change. And we may change the place where to_h gets called. But at least we have this defined somewhere for once. Eliminating vague behaviour is exactly what we wanted to get from Sorbet.

Relations, Arrays and RelationType

Throughout our app we had many places where arrays or ActiveRecord relations could be passed, often calling to_a just to be safe when we didn’t know what we actually add. Sorbet has forced us to be explicit about those. It seems like an improvement in this early days, but it takes away a bit of the Rails magic of just iterating on whatever objects you find.

Be sure to use the RelationType alias from sorbet-rails when dealing with relations so you don’t end up with long T.any(...) signatures for your relation types.

RelationType combined with ActiveRecord#none for empty results is that way to happiness.

Summing Up

Things are going well. We’ve got about 25% of our models that have been specifically typed. We’ve found some bugs and gotten a bit of refactoring done with increased confidence.

sorbet-rails has been solid. We haven’t run tapioca yet but expect that’s coming soon. In a future post we’ll talk about adding types for an ActiveRecord plugin (money-rails).