APC opcode caching of multiple sites

APC performs two functions, one as an object cache and the other as an opcode cache. In this post I’m talking specifically about the opcode caching functionality when used on a server hosting multiple websites.

What is an opcode cache?

I’m sure most of you know but I’ll just cover it quickly for those that don’t. Each time a PHP web page is requested the web server has to compile the human readable PHP code into a language the processor can understand, this is called opcode. APC will cache opcode in RAM so that subsequent requests for that file do not have to go through that same process of opening up the file from disk and compiling it into opcode. So not only does it save on compiling the code it also saves the disk access. If your website’s files are located on a distributed filesystem such as NFS then opcode caching will give you upwards of 100% improvement in performance any day of the week.

Configure APC with enough memory for your needs

APC’s default configuration is probably fine if you’re hosting a single website with hardly any plugins and a basic theme. Otherwise you should change the configuration, especially If you have more than a one website hosted on the server or your website has lots of PHP files.

In its default state APC will allocate 30MB of shared memory. The PHP files of a pretty moderate WordPress website will need more than 30MB space for the opcode cache (there are a lot of PHP files). You’ll probably want to look at budgeting 40MB for each site (so 10 websites would wipe out the RAM of a 500MB small cloud server). If APC runs out of space to store its cached PHP files then it will totally expunge the cache and start over and if that happens on every page load then you can say goodbye to your performance increase. There are some settings you can tweak to improve things a little but really you just need to allocate enough memory to APC.

Use apc.php to keep track of your APC usage and make sure it has enough free space to fit everything in. If you use APC as an object cache then you will need to allocate even more space to every website you host.

The benefits of using the APC opcode cache on a WordPress website

The graph on the left is without APC enabled and the graph on the right has APC enabled. The initial load test done without the APC opcode cache nearly crashed the server at 75 clients so the load test had to be stopped. Once APC was enabled requests per second are pretty stable with the server now easily doubling its performance.

More bang for your buck

I’ve thought for some time now that if you run multiple sites on your web server and those sites use a common codebase then disk space, RAM and processor time could be saved by creating a symlink to those common files. Rather than each website loading up and caching its own copy of the same code it would be much better if that code could be cached once and accessible to all sites.

In my case the codebase is WordPress but this could also apply to other frameworks such as CodeIgniter or CakePHP.

So to test this and confirm my initial thoughts I created a single line PHP file called simon.php, then created 2 symlinks and 2 hard links to that PHP file. Then went through a process of clearing the APC system cache, executing the symlinks, checking with apc.php to see if there were entries in the system cache for either the links or the file they referenced.

No matter which of the 4 files I accessed in a web browser there was only a single entry in the APC system cache. Each time I refreshed a page the system cache hit count incremented by one, whichever of the 4 files I accessed that same counter increased. It’s the path which is initially used to access simon.php that is recorded by APC then any subsequent requests that ultimately resolve to the simon.php file get attributed to the same entry in APC.

There is one slight difference between using hard and soft links in this situation (besides the usual differences). If the first request to a PHP file is through a hard link then the path to the hard link is stored by APC. If however the first request to a PHP file is through a soft link (symlink) then it’s the ultimate path (simon.php) that is recorded.

So you should be able to share a collection of PHP files among an unlimited number of sites using soft/hard links and those files will only be put in the opcode cache once and therefore only be taking up space in RAM once.

Let’s just confirm at a lower level that the opcode is being cached

I still felt like I needed further proof, so I forced Apache to just use a single process and then I watched that process to see exactly what it was doing each time it processed a request for my links. I’ve included those interactions between Apache and the OS in the gists below, both when Apache should have been opcode caching the file and when it should have been reading the file out of the cache rather than compiling it all over.

For those of you that can’t be bothered to read through those gists (120-180 lines each) It pretty much goes like this:

Initial request to a PHP file using the hard link

  1. Apache uses the “stat” command to get some information about the file requested
  2. Apache tries to open .htaccess files all the way up the directory tree
  3. Apache uses the “lstat” command to get information about the link and every directory leading to that link file (goes through this exact process 3 or 4 times for some reason)
  4. Then it opens up the file from the disk
  5. The file is then added to the opcode cache in shared memory
  6. Serves the resulting file to the browser
  7. Logs the request

Subsequent requests to a PHP file using the hard link

  1. Apache uses the “stat” command to get some information about the file requested
  2. Apache tries to open .htaccess files all the way up the directory tree
  3. Apache reads the file from the opcode cache in shared memory
  4. Serves the resulting file to the browser
  5. Logs the request

The only difference I can see when doing the same with a soft link (symlink) is that once Apache has used “stat” to get file information it then follows the link and switches to using the ultimate file name/path for further “lstat” checks.

————————————–

So the opportunity to squeeze out further performance in this situation is there. I’m definitely on the lookout for the best way to run separate WordPress installs with a shared core codebase (Multitenancy)

————————————–

Gists for the hard linked file
Apache + APC opcode cache : initial request for a hard linked PHP file

Apache + APC opcode cache : subsequent request for a hard linked PHP file (it’s coming from the opcode cache)

Gists for the soft linked file
Apache + APC opcode cache : initial request for a soft linked PHP file (symlink)

Apache + APC opcode cache : subsequent request for a soft linked PHP file (it’s coming from the opcode cache)

Advertisements