This past Sunday I was part of the Acquia team working behind the scenes to help support the GRAMMY Awards website crew, whose talented members make the GRAMMY.com experience such a success. I was on the ground at the Nokia Theater L.A. Live, getting an inside look at the GRAMMY team in action, and serving as the communications link through Google Hangout back to Acquia operations and support. The challenges of a GRAMMY Awards night are nearly unmatched as we had to ensure that the website would scale to meet incredible demand.
Whenever LL Cool J sent the nation to GRAMMY.com as the broadcast went to break, the site had to be ready to receive that traffic, humming along without missing a beat. ;)
The GRAMMY Digital Media team is under pressure to deliver on awards night. The GRAMMY digital properties receive the lion’s share of the year’s traffic in the short span of days surrounding the annual show. Supporting the team that produces GRAMMY.com are several technology partners in addition to Acquia, which provides site resiliency and performance with Acquia Cloud along with Drupal expertise and one-to-one support. Lullabot helped build and launch the beautifully responsive site. Akamai provides the CDN and their Site Shield DDoS protection. Ooyala provides the video-on-demand platform.
Under the Hood
We started load testing GRAMMY.com a few weeks ago in preparation of the 56th annual award show. We do everything to simulate award night traffic as best we can. We did need to make some late adjustments based on our testing. We discovered that the addition of an Elastic Load Balancer following last year’s show introduced some new technical challenges for us to implement Akamai’s Site Shield protection. Some of what we learned during testing while working with the GRAMMY’s ELB is now incorporated into Acquia Cloud for all of our customers’ benefit.
We also identified potential PHP performance issues that the GRAMMY’s Drupal developers were able to address quickly. Given some of the late changes, we continued to test the technology stack throughout the week leading up to the show, sometimes staying up until well past midnight to complete our tests.
Testing the entire stack and ensuring performant code is key to any successful site with high traffic. With demand spiking to GRAMMY proportions, even a fraction of a percentage point in degraded performance can have a dramatic impact on site health.
We also moved GRAMMY.com from a shared Solr search cluster to a dedicated cluster for their search system. That migration happened completely transparently with no configuration changes required. The sheer volume of searches happening on GRAMMY.com this past weekend would have degraded search performance for the GRAMMYs and for other sites on the shared infrastructure.
Acquia Cloud allows us to offer our clients elastic platform capacity during the year based on need. For the GRAMMYs, each year we expand the capacity prior to the event. During the show, we continue to monitor incoming traffic and are able to add additional capacity on the fly to handle the load. This year we more than doubled the GRAMMY’s capacity during the course of the award night.
For us, the biggest source of pride is that our work is transparent in all affect. Our job well done is invisible when your experience on GRAMMY.com is flawless. And with a few years of show experience under our belts, we’re battle tested. Without any fires to chase at the close of the show, we enjoyed the The Recording Academy’s aftershow party at the LA Convention Center.
