Horizontally Scaling Drupal

Sun, Feb 10, 2013 2-minute read

Few months ago something terrible happened. One corporate site (written in Drupal) crashed after just couple of hundreds concurrent users visited site. Since that was in time of some very important promotions we were tasked to fix problem asap (10:05am).

Our finding –> Reason #1 (10:15am): We had very basic server (6GB RAM, 2CPU) hosting inside same box: apache, all caches, and MySQL. Our top-of-the-art Drupal app that required just few Cray machines crashed 100% :-).

So what did we do. Since time was critical we did:

  1. Moved site into maintenance mode, so we can access choking server (10:30).
  2. Increased our VM machine resources to all what we can get (40GB RAM, 16 CPU cores).  Required time to contact provided, get changes applied some 60min.
  3. After that site performed more or less as requested (starting from 11:45am).

Few days after that experience, during some quiet period of time, I did load test of this machine and found that we cannot handle more than 800 concurrent users even with this configuration. Little work on some code tweaks allowed extra few hundred users, but at all nothing more than 1000. Pretty low number…

And in that moment I started to investigate how to scale Drupal site horizontally. I know I was supposed to do that before, but this is corporate site and… (add text according to personal preferences). I started with detaching Apache and MySQL servers, adding extra web servers, and finally moving complete solution to the Cloud.

In next few weeks I will present here some of my findings and as much as I can step-by-step guides how to implement them.

So be tuned.