Tag Archives: varnish

All Magento pages from Varnish

I just realised I haven’t updated the site since our last big development. We’re now serving almost all of our pages from Varnish. Crude research suggests around 90% of our pageviews are now coming from Varnish. In simple terms, we’re doing did the following:

  • Render the cart / account links from a cookie with javascript
  • Ajax all pages, so everything can be cached (with a few exceptions we manually exclude)
  • Cache page variants for currencies and tax rate

We’re also warming / refreshing the cache using a bash script parsing the sitemap and hitting every url with a PURGE then a GET.

The hardest part of the whole performance space has been to measure the impact of our changes. But our TTFB was previously in the 300-500ms range for most pages, and now it’s in the 20-30ms range for pages that come from Varnish. I’m very confident that it’s impacting our bottom line.

All category pages from varnish

It’s a glorious day in the pursuit of ultra high performance on Magento. Today, we serve all our category pages from varnish. Plus, we artificially warm the cache to ensure that all category pages across all sites are already in the cache when a real user hits the sites.

Varnish typically takes our time to first byte from around 300ms – 400ms to 20ms – 30ms. We were previously serving 80% of landing pages from varnish, but this changes should improve overall performance by a noticeable margin. Happy days. 🙂

The implementation is fairly custom. Essentially, we’re adding a header to all pages which tells varnish whether the page can be cached or not. So on category pages that header says yes, on product pages that header says no. We also did some custom coding to dynamically the header links (My Cart, Login, Logout, etc) from a cookie. We set that cookie on login, add to cart, etc.

Traffic profile by page

I’ve been looking at the profile of our traffic. On our biggest site, the top 50 pages account for more than 50% of our traffic. Across all four sites, our top 100 pages account for 50% of total pageviews.

Getting this data out of Google Analytics was easy. To start with I went to the Content > Site Content > Pages report, switched the view to percentage, and added the percentages for our top 10 then top 50 pages with a calculator.

To get data for all our sites, I used this trick to export the data for all our sites. I had to use the old interface to do this, then Content > Top Content. I combined all the data into a single sheet in LibreOffice Calc, added a row for the site domain and one for the total pageviews on that site. That allowed me to graph percentage of total traffic for our most popular pages.

I graphed our traffic with a log scale, and it looks a lot like the graphs from the Long Tail.

Our pages are around 10K. So a quick calculation suggests that (allowing 100% extra for Varnish overhead) we could serve 50% of our total traffic from 2MiB of cached data, or 80% of our traffic from 12.5MiB (640 pages, 20KiB per page). That puts an interesting spin on the idea of using a varnish cache.

I wonder how our traffic profile compares to that of other sites. I’d imagine some sites are more niche, with less traffic in the “head” and more in the “tail”. In those cases, caching might provide less of a performance boost.

This also got me thinking about benchmarking cache performance. I’ll think about it and write more in a later post.

Update: Another couple of perspectives. We have fewer than 500 pages out of a total 12k which get more than a 100 pageviews per month (3 per day). If we can serve 80% of traffic from 12.5MiB of cached data, in economic terms that’s £7/month!