Sneaking Crystal in Wego

We recently needed to extract a feature from a Ruby on Rails application into a full-fledge microservice of its own. We could have done it again in Ruby on Rails but decided to use another tech stack this time around. The company is currently experiencing technological renaissance with recent successful adoption and deployment of Elixer and Golang stacks in our backend from doing mostly everything between Java and Ruby.

Choosing the next tech stack for the project, esp. the language is treading on dangerous waters. You don't want to force colleagues to learn new stuff that they might not be able to utilize outside the project. We need people to be able to read the code and contribute or point out gaping hole in our logic.

Having previously smuggled Crystal into Wego ecosystem through small standalone shell utility for generating Terraform files for our CDN and Nameserver configs, it's a joy to write code in that the next project was decided to be with Crystal again and it doesn't hurt that Crystal apps are mainstays of top 50 TechEmpower benchmarks ranking.

Features We Liked

• compiles to native code
• static type checking
• generics and macros
• reads like ruby

Compiles to Native Code

Crystal programs compiles to native code in a single file. We build the project and deploy only the executable in the final container. This greatly cleans up the deployment by not having the source code.

Static Type Checking

Most of the runtime error can be eliminated by type checking the variables used in the program. Crystal will not allow us to pass a different variable type than what we expect to receive.

Generics and Macros

The term generics and macro will scare most devs but in Crystal they're easy and fun to deal with. Here's a sample of a simple validator we have that's implemented as a generic using macro programming.

class Validator(T)
  annotation Check; end

  property data : T

  def initialize(data : T)
    @data = data
  end

  def validate!
    errors = Array(String).new
    {% for method in @type.methods %}
      {% for ann in method.annotations(Check) %}
        if !{{ method.name }}
          errors << {{ ann[0] }}
        end
      {% end %}
    {% end %}

    unless errors.empty?
      raise T::Exception.new(errors)
    end
  end
end

class ApplicationParams < JsonParams
  property name : String?
  property email : String?
  property expires_in : String?
  property token_validity : Int32?
  property rate_limits : JSON::Any?
end

module Applications
  class CreateValidator < Validator(ApplicationParams)

    @[Check("name is required")]
    def has_name : Bool
      !@data.name.nil?
    end

    @[Check("email is required")]
    def has_email : Bool
      !@data.email.nil?
    end

    @[Check("expires_in is required")]
    def has_expires_in : Bool
      !@data.expires_in.nil?
    end
  end
end

Macros can be found inside the {% %} and {{ }}blocks. Crystal gives us access to the AST during compile time. The code above creates a dedicated validator for JSON objects which gives it ultimate freedom on how to validate it's properties. The CreateValidator is using the Validator generics.

Stack

Web Framework

After testing out and making POC Todos project to find out the best stack we should pick, we went with the modular approach rather than the single-framework-batteries-included route. In fairness to the web frameworks in Crystal is that most of the them share common underlying libraries but hardwired for the specific framework use case. The most popular frameworks that we shortlisted and played with are Amber, Lucky, Kemal, Grip and Spider-Gazelle.

Amber and Lucky are full-stack web frameworks which we didn't consider at this point as they're overkill for the microservice applications we're doing in the company. Grip was promising until we can't do something very important to how we do things in the company. (eg. we had a hard time figuring out setting up logging and redirecting it to where we wanted). Spider-Gazelle is a clone the repo and add your stuff type of framework which we figured out will be a pain to upgrade later on after we added our custom stuff in. That brings us to Kemal.

Kemal is actually the most popular web framework in Crystal. It's works similarly to Sinatra in Ruby or Express in Node. It gives you all the things you need to put out a page in a browser - cookie/session, logging, builtin app server provided by the language, templates, and routing. Middlewares can easily be written by following Crystal's app server API.

ORM

Database access is the most common things you do with web applications. Most ORMs from the full-stack frameworks can be installed as shards into your project. We went with Jennifer ORM. Jennifer behaves like RoR's Activerecord complete with necessary features like migration, serialization, relations, validations, callbacks, scopes... Database connection configuration are done in YAML like in RoR.

Issues

We've use Crystal in production for a few months now without a major issue besides unexpected input params which is not the fault of the language. There's currently one thing that we wish we can figure out is how to to see the error log stack trace in Sentry.