from http://speedhunters.com/archive/2009/07/06/event-gt-gt-goodwood-fos-view-from-the-hill.aspx

In my previous post  I spoke about how to fix the preloading in order to take advantage of WordPress and CDN for local AND remote serving.  Please see

Primer needed: CDN is slower than my slowpoke for galleries – FIX IT NOW

Today, I tackled the final missing tire to get this Ferrari on the road. We’re going to get a custom cacheing system in place.

Let’s recap:

Server: optimized for Sql cache, APC cache, object-cache in WP,  gzip of all acceptable file formats, and full hard-coded redirection to CDN for all images, js, css which work remotely.

Small change to the image upload directory in WordPress, and we used a  tool called Auto Post Thumbnail (WP plugin) to ensure all our old thumbnails were now using the new CDN links.

Good to go from here forwards,… but we were missing a cacheing system.

We quickly built  a system which would use Cron to create a static home page to our visitors.  This simply updates the home page every 10 minutes,  a very simple, elegant and straight forwards “keep it simple” solution.

KISS all the above,  but we’re still missing a full cacheing system.

We needed a system to cache our over 2000 pages today, and get ready for massive growth in near future.  We plan to have over 5000 items online by December

We looked at all the cacehing plugins for WordPress and found all lacking.  For many various reasons either we could not get them to work, or they did not work fully, or simply mucked up our scripting and custom coding on our site.  These , like Hyper-cache, W3TotalCache and WP-Super-Cache Quick Cache, are best used for sites on shared hosting, and they do a great job there.  We have a dedicated host.

So to the cacheing strategy I found all/any of the above somewhat limiting.  I needed finer grained cacheing, and I needed it to WORK FOR ALL files at all times.

I’m sure I spent over 40 hours tackling the various plugins, installing, testing, and re-installing. I did not find a single one (and I tested almost all available, including many not mentioned here) which did the job at 99%.  I insist on 99%.

All the bells and whistles the plugins contained, we already had covered on our own server setups here, and the only piece missing now was the page cache itself.  I tried to install the cache plugins and JUST USE THE PAGE CACHE… but the convoluted .htaccess rules again led me to problems.

So….. I sat back and bit the bullet.  Something I should have done in the beginning.  I just built it myself, as usual.   (I spent less time creating my cacheing system and testing then trying out the plugins, by the way.

What I found was simple.  Use some logic, define your needs, and then just use PHP to cache your files.

Here’s how it works.

Pages to be cached.

1. Home page  – Cron w/ 10 minute intervals, always cached, no live version. ONE of these
2. Singel Posts — PHP cacheing, cached page created on demand, or I can batch create from a list. 2000 of these
3. 404 error page:  nice to keep it cached…  One of these
4. Static Pages:  like about us… Green spot is… consignment info, etc..   no need for these to be dynamic :  about 30 pages.

Timeouts:  I requred that each of these groupings have differing times of renewal… for example

1. Home page:  should be renewed every 10 minutes.
2. Single Posts:  these can be renewed every 12-24 hours.
3. 404 should be cached for longer period,  like 6 days
4. Static Pages,  cache for 24-48 hours.

If one could count on browsers having javascript and browser cacheing enabled — well that would be simpler… you could count on the browser to cache it given specific directions from our server.

But you can’t.

So we take matters into our own hands.  WE tell the browser how long to cache files for.. and then force-feed them cached files just in case they didn’t obey.  A lesson learnt from raising three boys, I guess

The system also needed to remain dynamic, so if a user logged in, they would get live files AND we considered anyone who viewed more than a few pages a potential customer who should be given fresh pages to view.   We did this with cookies.

WE set different values for different parts of the site according to our best guesstimate of utility. (working from memory, I think these are the values,  or roundabout)

If a user is logged in…  this totally overrides the cache. But if not (the lookey-loos and casual surfers), then

1. Home page:  needs 10 views to override cache – nothing to do on homepage. no live buy button.
2. Single Posts:  needs 4 views to ovferride cache — hmm might really be interested., and each page contains a live purchase option. (except the SOLDS).
3. 404 page never overriden
4. Static Pages,  needs 6 views to override.

So there are the rules.  here’s how we solved the issue.

Note: techy stuff below.

Step 1: this goes into the TOP of your file:  best use is for your single.php file, for example.

you should add <?php  to the beginning of these scripts, of course.

The result: If a visitor logs in they get live pages, if they view more than 6  pages they get live pages,. If they leave after viewing 4 pages, the cookie, set for one hour starts the count again.  The cached file remains the same for 24 hours.

Now the occasional visitor, the lookey-loos, and search engines all create no real strain on our server,  they are all fed cached files.  

Step 2: And for the footer you place this in your pages.

This is just to close off the operation… don’t ask me, I’m no PHP genius…..  it just works.

The step at the top determines if you should send a cached file…. if not,  the server spits out the composed page, and now saves it to a cached file.   A simple operation.

CAVEAT:  you don’t want to be using this for multi-page archives, tag archives, etc.

Besides:  why would you wish to cache a multi-page archive or search result?   just bad karma I think, asking for trouble and extra complexity for little advantage.

Using the above system, we’ve combined this with our CDN systems, optimized server, and reduced our page load to 2 seconds from OUR server, and about 2 seconds max from CDN service.  That’s  3  second load for about 1.4  Megs of composed “live” pages.   We have tested the scenario out where we utilize the CDN system to serve out the static cached pages, and it leads to a 2 second download for ALL parts.

Impressive, and definetly a round the lap in Ferrari-like time now.

If you require assistance in optimizing your websites, do give us a shout.  Remember, you’re viewing this page from a home iMac sharing time with our office software…. can it get any better than that?

High-end results : inquire here… because We do Biz websites too

This entry was posted on September 15, 2010 ... filed under Vinces Blog