Is My CDN Working? Announcing Fastly on Engine Yard

Fastly on Engine Yard

CDN integration is a request we've been hearing from our customers at Engine Yard for a long time now. So we are proud to announce the immediately availability of Fastly as an Engine Yard Add-on.

Fastly is built using the open-source Varnish Cache running in a network of data centers around the world. While supporting a number of advanced options for caching and cache-invalidation, Fastly is also super easy to setup as a basic asset cache.

This post will talk about setting up Fastly on Engine Yard and how to use Fastly Surrogate-Keys for purging individual objects from the cache. You may also be interested in the announcement from Fastly: Ruby on Rails on Fastly.

Setting up Fastly

The simplest, most bare-bones way to use a CDN is to just use it as an asset host. This means things like your javascript, css, and image files can be served by the CDN, leaving more resources available for your Engine Yard web stack to serve requests that run your app-specific code.

The example TODO application is a basic app using Rails 3, and so makes a good candidate for a simple Fastly integration.

I created a branch of the app, and deployed it running on a single c1.medium at http://is-my-cdn-working.engineyard.com.

I enabled the Add-on by going to https://addons.engineyard.com/addons/fastly, clicking "Log In", selecting my todo application, and clicking "Activate".

activating fastly

After activating, Fastly still needs to know a few more pieces of information about our app, so we click on the activated Add-on to SSO over to a page hosted by Fastly and enter the app name and URL.

We enable asset caching in our Rails app with a few lines of code.

config.action_controller.asset_host = EY::Config.get(:fastly, "FASTLY_CDN_URL")
config.static_cache_control = "public, max-age=2592000"
config.assets.digest = true

After a deploy, our app is up and running and serving all assets via Fastly.

We can see this is working by using the chrome inspector:

chrome inspector

Notice that application.js is now being served from a URL that starts with "http://is-my-cdn-working-engineyard-com.global.ssl.fastly.net" (meaning it's coming from Fastly).

Speed Testing with Blitz

By using Blitz as an Engine Yard Add-on, we can test the performance of our newly improved TODO app.

Blitz load tests using direct HTTP and not a web browser, so the tests won't download all of our assets. But, we can get a rough approximation of the effect by performing a sequential test. We'll have Blitz first hit the homepage (not cached), and then hit on our assets (cached).

Here's what the setup looks like in Blitz:

blitz test setup

We'll do the test 2 times, once hitting our app directly, and once going through Fastly.

Caveat

Ideally our second load test would hit our engineyard.com address on the first hit instead of fastly.net to better simulate what web browsers would do. However, Blitz doesn't support this. So what we're really doing with this test is more a simulation of what performance would be like if we passed all of our traffic through Fastly (not just assets).

Results (Uncached)

uncached test result

Results (Cached using Fastly CDN)

cached test result

The results show (as we expected) lower response times when going through Fastly. The average response time is 256 ms when using the cache vs. 474 ms when uncached.

Taking the next step with Fastly, Dynamic Edge Caching

So far we've only been using Fastly as an asset host. But, we could start to see even more performance gains is if we cached the whole site. The first step in more caching is getting all requests to go through Fastly. We do this by setting up a CNAME record in DNS.

After we setup the CNAME, all traffic to our site will be passing through Fastly. This means that we can delete that asset host line from production.rb if we want (doesn't matter either way).

Setup DNS CNAME

We'll have to delete our DNS entry for is-my-cdn-working.engineyard.com and re-add it as an CNAME to global.prod.fastly.net. And then so we can still compare against the un-cached version of the site, we'll create uncached.is-my-cdn-working.engineyard.com.

DNS entry

By the way, because we use Dyn DNS to manage DNS and we are able to publish the full set of changes simultaneously and avoid any downtime. We'd love to offer DNS as an Add-on, so please tweet at Dyn and tell them! (Add-on Partner docs here)

Back in the Fastly dashboard, we also need to change our "Origin Settings" to uncached.is-my-cdn-working.engineyard.com. When I did this at first I got a 503 error with:

Loop occurred: cachesv63SJC3cachelax1432LAX

But after a few seconds it seemed to resolve itself and work.

Cache Control Headers

Now all of our rails app responses are passing through Fastly, and we almost have our requests cached. However, there's a particular header being returned by app that's preventing caching.

Cache-Control: max-age=0, private, must-revalidate

The Cache-Control header tells your web browser not to cache this page (which we want). And, accordingly, the Varnish cache server running on Fastly will interpret this header to never cache this page. (not exactly what we want)

Fortunately, there is another header we can use for communicating caching information to Varnish without affecting our directive to web browsers: Surrogate-Control.

With an application-wide before filter, we can implement a 1 second cache on all GET requests.

before_filter :set_cache_control_headers

def set_cache_control_headers(max_age = 1.second.to_s)
  if request.method == "GET"
    request.session_options[:skip] =
      true  # removes session data
    response.headers['Cache-Control'] =
      'public, no-cache'
    response.headers['Surrogate-Control'] =
      "max-age=#{max_age}"
  end
end

If you are using Rails 4, the Cache-Control header is set in a slightly different way.

response.cache_control[:public] = true
response.cache_control[:extras] = ['no-cache']

Now let's do another Blitz load test. This time we'll just hit the homepage with a GET request. Also, let's ramp up the intensity by changing the peak concurrent user count from 30 to 300.

Results (Uncached)

uncached test result

Results (Cached using Fastly CDN)

cached test result

Now we're seeing order of magnitude improvements!

Cache Invalidation and Fastly Instant Purging

A 1 second cache is nice, but imagine how much load we could alleviate if we had a 1 minute cache, or even a 1 hour cache! The problem with caching every page at 1 minute, however, is that when users add or remove items the changes won't appear for up to 1 minute. And that's an unacceptable user experience. We need a way to invalidate the cache whenever there's an update.

Fortunately, there is another special header that Varnish will notice: Surrogate-Key. This header can be set with any values (space separated), and the idea is to use it to specify unique identifiers for your content.

By tagging cached pages in Fastly with these identifiers, we can now make API calls to Fastly to purge content.

For our simple example, we will just use the same identifier for every page. And then purge the whole cache whenever there is a non-GET request. We just need to make an authenticated POST to https://api.fastly.com/service/<id>/purge/<key>.

before_filter :cache_or_purge

def cache_or_purge(max_age = 1.minute.to_s)
  if request.method == "GET"
    request.session_options[:skip] =
      true  # removes session data
    response.headers['Cache-Control'] =
      'public, no-cache'
    response.headers['Surrogate-Key'] =
      "blah"
    response.headers['Surrogate-Control'] =
      "max-age=#{max_age}"
  else
    api_key =
      EY::Config.get(:fastly, "FASTLY_API_KEY")
    service_id =
      EY::Config.get(:fastly, "FASTLY_SERVICE_ID")
    Excon.new('https://api.fastly.com').post(
      :headers => {'Fastly-Key' => api_key},
      :path => "/service/#{service_id}/purge/blah")
  end
end

New Relic

Blitz was great for seeing the performance impact of our Fastly caching under specific controlled conditions, but to monitor ongoing real-world performance we can use New Relic as an Engine Yard Add-on.

One such Engine Yard customer sQuidd.io did just that and sent us this screenshot of performance improvements:

squidd.io new relic

Further Reading

Engine Yard customer Hotel Tonight has been successfully using Fastly to cache API requests, and blogged about both basic caching and cache invalidation with surrogate keys.

For more information about how cache invalidation works with Fastly, you should checkout their posts about using surrogate keys and how surrogate keys are implemented.