Updates from simonthepiman Toggle Comment Threads | Keyboard Shortcuts

  • simonthepiman 11:52 pm on August 27, 2012 Permalink
    Tags: apc, opcode cache, performance   

    APC opcode caching of multiple sites 

    APC performs two functions, one as an object cache and the other as an opcode cache. In this post I’m talking specifically about the opcode caching functionality when used on a server hosting multiple websites.

    What is an opcode cache?

    I’m sure most of you know but I’ll just cover it quickly for those that don’t. Each time a PHP web page is requested the web server has to compile the human readable PHP code into a language the processor can understand, this is called opcode. APC will cache opcode in RAM so that subsequent requests for that file do not have to go through that same process of opening up the file from disk and compiling it into opcode. So not only does it save on compiling the code it also saves the disk access. If your website’s files are located on a distributed filesystem such as NFS then opcode caching will give you upwards of 100% improvement in performance any day of the week.

    Configure APC with enough memory for your needs

    APC’s default configuration is probably fine if you’re hosting a single website with hardly any plugins and a basic theme. Otherwise you should change the configuration, especially If you have more than a one website hosted on the server or your website has lots of PHP files.

    In its default state APC will allocate 30MB of shared memory. The PHP files of a pretty moderate WordPress website will need more than 30MB space for the opcode cache (there are a lot of PHP files). You’ll probably want to look at budgeting 40MB for each site (so 10 websites would wipe out the RAM of a 500MB small cloud server). If APC runs out of space to store its cached PHP files then it will totally expunge the cache and start over and if that happens on every page load then you can say goodbye to your performance increase. There are some settings you can tweak to improve things a little but really you just need to allocate enough memory to APC.

    Use apc.php to keep track of your APC usage and make sure it has enough free space to fit everything in. If you use APC as an object cache then you will need to allocate even more space to every website you host.

    The benefits of using the APC opcode cache on a WordPress website

    The graph on the left is without APC enabled and the graph on the right has APC enabled. The initial load test done without the APC opcode cache nearly crashed the server at 75 clients so the load test had to be stopped. Once APC was enabled requests per second are pretty stable with the server now easily doubling its performance.

    More bang for your buck

    I’ve thought for some time now that if you run multiple sites on your web server and those sites use a common codebase then disk space, RAM and processor time could be saved by creating a symlink to those common files. Rather than each website loading up and caching its own copy of the same code it would be much better if that code could be cached once and accessible to all sites.

    In my case the codebase is WordPress but this could also apply to other frameworks such as CodeIgniter or CakePHP.

    So to test this and confirm my initial thoughts I created a single line PHP file called simon.php, then created 2 symlinks and 2 hard links to that PHP file. Then went through a process of clearing the APC system cache, executing the symlinks, checking with apc.php to see if there were entries in the system cache for either the links or the file they referenced.

    No matter which of the 4 files I accessed in a web browser there was only a single entry in the APC system cache. Each time I refreshed a page the system cache hit count incremented by one, whichever of the 4 files I accessed that same counter increased. It’s the path which is initially used to access simon.php that is recorded by APC then any subsequent requests that ultimately resolve to the simon.php file get attributed to the same entry in APC.

    There is one slight difference between using hard and soft links in this situation (besides the usual differences). If the first request to a PHP file is through a hard link then the path to the hard link is stored by APC. If however the first request to a PHP file is through a soft link (symlink) then it’s the ultimate path (simon.php) that is recorded.

    So you should be able to share a collection of PHP files among an unlimited number of sites using soft/hard links and those files will only be put in the opcode cache once and therefore only be taking up space in RAM once.

    Let’s just confirm at a lower level that the opcode is being cached

    I still felt like I needed further proof, so I forced Apache to just use a single process and then I watched that process to see exactly what it was doing each time it processed a request for my links. I’ve included those interactions between Apache and the OS in the gists below, both when Apache should have been opcode caching the file and when it should have been reading the file out of the cache rather than compiling it all over.

    For those of you that can’t be bothered to read through those gists (120-180 lines each) It pretty much goes like this:

    Initial request to a PHP file using the hard link

    1. Apache uses the “stat” command to get some information about the file requested
    2. Apache tries to open .htaccess files all the way up the directory tree
    3. Apache uses the “lstat” command to get information about the link and every directory leading to that link file (goes through this exact process 3 or 4 times for some reason)
    4. Then it opens up the file from the disk
    5. The file is then added to the opcode cache in shared memory
    6. Serves the resulting file to the browser
    7. Logs the request

    Subsequent requests to a PHP file using the hard link

    1. Apache uses the “stat” command to get some information about the file requested
    2. Apache tries to open .htaccess files all the way up the directory tree
    3. Apache reads the file from the opcode cache in shared memory
    4. Serves the resulting file to the browser
    5. Logs the request

    The only difference I can see when doing the same with a soft link (symlink) is that once Apache has used “stat” to get file information it then follows the link and switches to using the ultimate file name/path for further “lstat” checks.

    ————————————–

    So the opportunity to squeeze out further performance in this situation is there. I’m definitely on the lookout for the best way to run separate WordPress installs with a shared core codebase (Multitenancy)

    ————————————–

    Gists for the hard linked file
    Apache + APC opcode cache : initial request for a hard linked PHP file

    Apache + APC opcode cache : subsequent request for a hard linked PHP file (it’s coming from the opcode cache)

    Gists for the soft linked file
    Apache + APC opcode cache : initial request for a soft linked PHP file (symlink)

    Apache + APC opcode cache : subsequent request for a soft linked PHP file (it’s coming from the opcode cache)

     
    • household cleaning supplies organizer 11:33 pm on December 20, 2012 Permalink | Reply

      Does your website have a contact page? I’m having problems locating it but, I’d like to
      send you an e-mail. I’ve got some ideas for your blog you might be interested in hearing. Either way, great site and I look forward to seeing it expand over time.

    • Watch Spartacus War of the Damned Season 1 Episode 3 10:54 pm on February 8, 2013 Permalink | Reply

      I think this is one of the most significant information for
      me. And i’m glad reading your article. But should remark on some general things, The site style is wonderful, the articles is really nice : D. Good job, cheers

    • Leigh 4:58 am on March 4, 2013 Permalink | Reply

      Excellent blog you have here but I was wanting to know if you knew of any
      message boards that cover the same topics discussed in this article?
      I’d really like to be a part of group where I can get opinions from other experienced people that share the same interest. If you have any suggestions, please let me know. Cheers!

    • DenverFeri 8:53 am on April 7, 2014 Permalink | Reply

      Подтверждаю. Похожее смотрел на этом сайте мудрые мысли

    • sdorttuiiplmnr 2:45 pm on June 21, 2015 Permalink | Reply

      F*ckin¦ amazing issues here. I¦m very satisfied to look your post. Thanks so much and i’m having a look ahead to contact you. Will you please drop me a mail?

  • simonthepiman 10:40 am on August 23, 2012 Permalink
    Tags: Multitenancy, shared codebase   

    Jason McCreary gives an overview of what “WordPress Multitenancy” is and the steps he took to achieve it, along with his current working solution.

    http://jason.pureconcepts.net/2012/08/wordpress-multitenancy/

    This is relevant to servers hosting a number of WordPress installs. Having a shared core codebase (not necessarily using multisite) means it’s easier to detect changes in the codebase by having it version controlled. Also less shared memory will be needed by the APC opcode cache, giving you more memory for object caching etc.

     
    • blobaugh 11:01 pm on August 23, 2012 Permalink | Reply

      Note that this bring in it’s own set of headaches, and may ultimately not be helpful to scaling out large networks of sites

    • simonthepiman 11:37 pm on August 23, 2012 Permalink | Reply

      If you have knowledge of the headaches then maybe summarise them here or contribute to that blog post or if you know of other resources that have further information on the issues someone might face when sharing the core codebase then link to them here in the comments.

      That’s what it’s all about, sharing ideas, experiences, data.

    • blobaugh 11:39 pm on August 23, 2012 Permalink | Reply

      Sure, all this was shared in the mailing list. That data should probably be ported over here and anyone new posting there also pointed to this site

    • simonthepiman 10:17 am on August 24, 2012 Permalink | Reply

      I’ve just looked at the wp-hackers list archive and not sure of the best way to get that information over here in a digestable way so I’ll just link to the archive:

      [wp-hackers] Running several WordPress sites on the same server
      http://lists.automattic.com/pipermail/wp-hackers/2012-August/044183.html

      • Jason McCreary 7:18 pm on August 25, 2012 Permalink | Reply

        First, thanks for mentioning my post here. WordPress Multitenancy is something I’ve been interested in for a while. This solution is a first attempt. As such, I welcome feedback. I’ve reviewed the mailing list link provided. However, I’d appreciate elaboration on these “headaches”.

        • simonthepiman 7:57 pm on August 25, 2012 Permalink

          I too looked through the mailing list and couldn’t really see the headaches?

    • Simon 11:39 pm on August 26, 2012 Permalink | Reply

      I can confirm that having a shared core codebase will reduce the RAM needed for hosting multiple sites when utilising the APC opcode cache.

      http://lists.automattic.com/pipermail/wp-hackers/2012-August/044230.html

      • Luke 6:45 pm on February 24, 2013 Permalink | Reply

        Currently looking into this myself for a platform of 30 odd sites, definitely reduced the RAM and so far by sharing everything except theme files and uploads we’ve reduced the deploy size about 98%, has anybody else tried this in production?

    • Simon 8:40 pm on February 24, 2013 Permalink | Reply

      I’ve got a symlinked install in production. It works really well, I’m moving everything to symlinked as soon as I have time.

      Added advantage is you can have a WordPress folder for each wp version and easily switch version by symlinking to the chosen wp directory or uploading the latest version of WP to wp-latest and upgrading all sites in one swoop.

      • Luke 2:37 pm on February 26, 2013 Permalink | Reply

        Did you ever have an issue with the server response being slower? We’ve got the wordpress core symlinked into each of our sites but with this new mutlitenancy setup its spending almost twice the time in PHP

        • Simon 10:44 am on February 28, 2013 Permalink

          No issues like that at all.

          The main issue is down to not having a run of the mill file structure. Many plugins just assume the wp-content and WordPress core files are found under the same old names in the same location but on my installs WordPress core files are in a folder called /wp/ and the wp-content folder is /wp-content/ but plugins just assume everything is in the WordPress folder so look for a non existant wp-content folder in /wp/wp-content/.

          It’s not usually too much of a problem though, just an image missing here and there but I could see how a plugin could try cache expensive to generate information in an invalid location and cause page generation times to double.

    • Eduardo 8:17 pm on March 13, 2013 Permalink | Reply

      Awesome. We’re rolling this into production for 3k websites in a couple of weeks.

    • Scott Hack 9:40 pm on April 11, 2013 Permalink | Reply

      Jason did an updated blog post on this subject that you’ll probably want to check out Eduardo.

      • Eduardo 3:11 am on July 4, 2013 Permalink | Reply

        Thanks Scott. I should mention that we were unable to get multitenancy to work yet. I’ll see if Jason’s update can help us.

c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel