What started as an Elixir project over the weekend to support our mobile team with their QA. UltronEx grew into the playground to have introduced many new things to our stack including elixir. The initial design of the project upon which this was built to provide additional support for the QA was stream messages directly into Slack. It soon hit limit on the Slack side with calls being rate limited. This caused thousands of messages not being delivered to Slack for QA to work with. We were getting hundreds of event types rate-limited some over 10000 times since the project. One way would have been to retry but in a stream system you are only queuing messages again to fail and when a message is delayed or out of sequence it loses its value. How bad the rate limit was would be evident from the image below.

As a fun way to monitor the system, I started keeping stats of how many messages were processed, how many attachments were downloaded and scanned for matches, how many messages were sent using the service. Keeping in mind it all started as a project to not only provide value but to also assess the ability of Elixir as a robust technology. Along with the slack bot, the API and now a real-time Dashboard built using WebSocket nothing has changed on the resource limit. It still runs on a 1vCPU 2GB RAM instance in the cloud (application container memory limit is set to 350MB only) . As of this writing, it has read over 1.3 million messages from slack, downloaded over 1.2 million attachments and scanned them and has sent close to 1 million messages to slack using its API endpoint from other services.

The Dashboard is divided up in 3 verticals showing messages as SUCCESS , DANGER and WARNING . It had been long since I did work on the frontend. Last I worked with CSS and JS on the client-side Bootstrap used to be the hip kid on the block. So my first instinct was to Google for Bootstrap and to my surprise, it is active and doing well and keeping jQuery alive. With little Bootstrap styling, the dashboard was looking acceptable.

The performance of the dashboard has been smooth and stable. Internally we have tested it up to 30 users with no noticeable delay on a single box and the application is only allocated 350MB of memory with only 165MB utilised on average. This is a one-way communication where messages are sent to the dashboard. One could argue that Server-Sent Events would have sufficed the use case but as this is a project with expanding scope upgrading it to bidirectional later didn’t seem worth the effort.

The code that handles the message pushes from the server side is compact and readable thanks to Elixir

ahsandar/ultronex

RTM Slack bot written in Elixir. Contribute to ahsandar/ultronex development by creating an account on GitHub.

GitHubahsandar

defmodule Ultronex.Server.Websocket.SocketHandler do
  use Appsignal.Instrumentation.Decorators

  @behaviour :cowboy_websocket

  def init(request, _state) do
    state = %{registry_key: request.path}
    {:cowboy_websocket, request, state}
  end

  def websocket_init(state) do
    Registry.UltronexApp
    |> Registry.register(state.registry_key, {})

    {:ok, state}
  end

  @decorate transaction()
  def websocket_handle({:text, json}, state) do
    payload = Jason.decode!(json)
    message = payload["data"]["message"]
    websocket_send_msg(message, state)
    {:reply, {:text, message}, state}
  end

  @decorate transaction()
  def websocket_info(info, state) do
    {:reply, {:text, info}, state}
  end

  @decorate transaction()
  def websocket_send_msg(message, state) do
    Registry.UltronexApp
    |> Registry.dispatch(state.registry_key, fn entries ->
      for {pid, _} <- entries do
        if pid != self() do
          Process.send(pid, message, [])
        end
      end
    end)
  end
end

Real-time web apps seem to be the trend now and with Phoenix Live view it could be the next revolution. After how Ruby on Rails modernized the web development it is going to be Elixir and Phoenix.