Pragma::Migration

Pragma::Migration is an experiment at implementing Stripe-style API versioning.

This gem is highly experimental and still under active development. Usage in a production environment is strongly discouraged.

Installation

Add this line to your application's Gemfile:

gem 'pragma-migration'

And then execute:

$ bundle

Or install it yourself as:

$ gem install pragma-migration

Next, you're going to create a migration repository for your API:

module API
  module V1
    class MigrationRepository < Pragma::Migration::Repository
      # The initial version isn't allowed to have migrations, because there is nothing
      # to migrate from.
      version '2017-12-17'
    end
  end
end

And configure the gem:

# config/initializers/pragma_migration.rb or equivalent in your framework
Pragma::Migration.configure do |config|
  config.repository = API::V1::MigrationRepository
  config.user_version_proc = lambda do |request|
    # `request` here is a `Rack::Request` object.
    request.get_header 'X-Api-Version'
  end
end

Finally, you need to mount the migration Rack middleware. In a Rails environment, this means adding the following to config/application.rb:

module YourApp
  class Application < Rails::Application
    # ...

    config.middleware.use Pragma::Migration::Middleware
  end
end

Usage

When you start working on a new API version, you should define a new version in the repository:

module API
  module V1
    class MigrationRepository < Pragma::Migration::Repository
      version '2017-12-17'

      # We will give this a date very far into the future for now, since we don't know the release
      # date yet.
      version '2100-01-01', [
        # Add migrations here...
      ]
    end
  end
end

Suppose you are working on a new API version and you decide to remove the _id suffix from association properties. In order to support users who are on an older version of the API, you will need to do the following:

• remove the _id suffix from their requests;
• add the _id suffix back to their responses.

To accomplish it, you might write a new migration like this:

module API
  module V1
    module Migration
      class RemoveIdSuffixFromAuthorInArticles < Pragma::Migration::Base
        # You can use any pattern supported by Mustermann here.
        apply_to '/api/v1/articles/:id'

        # Optionally, you can write a description for the migration, which you can use for
        # documentation and changelogs.
        describe 'The _id suffix has been removed from the author property in the Articles API.'

        # The `up` method is called when a client on an old version makes a request, and should
        # convert the request into a format that can be consumed by the operation.
        def up
          request.update_param 'author', request.delete_param('author_id')
        end

        # The `down` method is called when a response is sent to a client on an old version, and
        # should convert the response into a format that can be consumed by the client.
        def down
          parsed_body = JSON.parse(response.body.join(''))
          Rack::Response.new(
            JSON.dump(parsed_body.merge('author' => parsed_body['author_id'])),
            response.status,
            response.headers
          )
        end
      end
    end
  end
end

Now, you will just add your migration to the repository:

module API
  module V1
    class MigrationRepository < Pragma::Migration::Repository
      version '2017-12-17'

      version '2100-01-01', [
        API::V1::Migration::ChangeTimestampsToUnixEpochs,
      ]
    end
  end
end

As you can see, the migration allows API requests generated by outdated clients to run on the new version. You don't have to implement ugly conditionals everywhere in your API: all the changes are neatly contained in the API migrations.

There is no limit to how many migrations or versions you can have. There's also no limit on how old your clients can be: even if they are 10 versions behind, the migrations for all versions will be applied in order, so that the clients are able to interact with the very latest version without even knowing it!

Using migrations to contain side effects

In some cases, migrations are more complex than a simple update of the request and response.

Let's take this example scenario: you are building a blog API and you are working on a new version that automatically sends an email to subscribers when a new article is sent, whereas the current version requires a separate API call to accomplish this. Since you don't want to surprise existing users with the new behavior, you only want to do this when the new API version is being used.

You can use a no-op migration like the following for this:

module API
  module V1
    module Migration
      class NotifySubscribersAutomatically < Pragma::Migration::Base
        describe 'Subscribers are now notified automatically when a new article is published.'
      end
    end
  end
end

Then, in your operation, you will only execute the new code if the migration has been executed (i.e. the user's version is greater than the migration's version):

require 'pragma/migration/hooks/operation'

module API
  module V1
    module Article
      module Operation
        class Create < Pragma::Operation::Create
          step :notify_subscribers!

          def notify_subscribers!(options)
            return unless migration_rolled?(Migration::NotifySubscribersAutomatically)

            # Notify subscribers here...
          end
        end
      end
    end
  end
end

Implementing complex version tracking

It is possible to implement more complex tracking strategies for determining your user's API version. For instance, you might want to store the API version on the user profile instead:

Pragma::Migration.configure do |config|
  # ...

  config.user_version_proc = lambda do |request|
    current_user = UserFinder.(request)
    current_user&.api_version # nil or an invalid value will default to the latest version
  end
end

The possibilities here are endless. Stripe adopts a hybrid strategy: they freeze a user's API version when the user performs the first request. They allow the user to upgrade to newer versions either permanently (you are not allowed to go back after a grace period) or on a per-request basis, which is useful when doing partial upgrades.

This strategy can be accomplished quite easily with the following configuration:

Pragma::Migration.configure do |config|
  # ...

  config.user_version_proc = lambda do |request|
    request.get_header('X-Api-Version') || UserFinder.(request)&.api_version
  end
end

FAQs

Why are the migrations so low-level?

Admittedly, the code for migrations is very low-level: you are interacting with requests and responses directly, rather than using contracts and decorators. Unfortunately, so far we have been unable to come up with an abstraction that will not blow up at the first edge case. We are still experimenting here - ideas are welcome!

What are the drawbacks of API migrations?

If you are used to ActiveRecord migrations, then you might be tempted to use this very freely. However, API migrations are very different from DB migrations: DB migrations are run once and then forgotten forever, API migrations are executed on every request as long as clients are running on an outdated version of your API. This means that API migrations should be considered an active, evolving part of your codebase that you will have to maintain over time.

Why should I keep the /v1 prefix?

The main reason for keeping the /v1 prefix and the API::V1 namespace in your API is that you might want to introduce a change so disruptive that it warrants a separate major version, like migrating from REST to GraphQL or introducing one alongside the other. In this case, you won't be able to use migrations to contain the change, so you will need to create a completely separate codebase and URL scheme.

What is the impact on performance?

We have a simple benchmark that runs 2,000 migrations in both directions. You can check out benchmark.rb for the details. Improvements are welcome!

Here are the results on my machine, a MacBook Pro 2017 i7 @ 3.1 GHz:

$ ruby -v benchmark.rb

ruby 2.4.2p198 (2017-09-14 revision 59899) [x86_64-darwin16]

Running 2k migrations, up and down:

       user     system      total        real
  0.090000   0.010000   0.100000 (  0.097414)

Are you out of your mind?

Possibly, but we're not the only ones.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/pragmarb/pragma-migration.

License

The gem is available as open source under the terms of the MIT License.