CloudFlare vs. Incapsula - Benchmark Review
As I have noted in my previous blog post, I’ve been testing out few CDN services for use with this website; CloudFlare, Incapsula & Torbit.
Currently, Torbit is in invite only beta. Josh Fraser was kind enough to provide me an access but since their system is not public yet, Torbit is not included in the benchmarks. But I have included a short review based on my recent experience. I’m planing to update this post with benchmark numbers in the future as soon as they are eligible…
How I tested
For the load time benchmarks, I have decided to use WebPageTest.org as it is free, providing actual browser results instead of simulated results and offering enough locations to see global timing. In the advanced settings, I have selected “First View Only” & 5 test runs to make sure content get in to the cache. I have repeated those test for multiple times at the different times of day to make sure server load won’t make a big difference.
First View only>
Test runs: 5
Basic (Free) Account
Medium Security (Default)
Trial period (All Features)
|Dulles, VA – IE8 – DSL||1.624s||1.567s||1.792s|
|New York, NY – IE8 – DSL||1.685s||1.596s||1.675s|
|*Atlanta, GA – IE8 – DSL||1.853s||1.740s||N/A|
|Chicago, IL – IE9 – DSL||3.130s||1.715s||2.032s|
|Kansas City, MO – IE8 – DSL||2.105s||1.730s||2.158s|
|San Jose, CA – IE8 – DSL||3.806s||1.704s||2.510s|
|San Francisco, CA – IE9 – DSL||2.620s||3.090s||2.571s|
|Los Angeles, CA – IE8 – DSL||2.953s||1.987s||2.150s|
|Montreal, Canada – IE8 – DSL||2.153s||2.003s||2.107s|
|São Paulo, Brasil – IE8 – DSL||3.700s||3.292s||3.773s|
|Dublin, Ireland – IE7 – DSL||4.185s||2.673s||2.604s|
|Gloucester, UK – IE7 – DSL||4.761s||3.306s||3.330s|
|London, UK – IE8 – DSL||3.833s||2.265s||2.038s|
|Paris, FR – IE8 – DSL||2.934s||1.921s||2.209s|
|Amsterdam, NL – IE8 – DSL||3.826s||2.967s||3.566s|
|Geneva, Switzerland – IE8 – DSL||3.350s||2.332s||3.239s|
|Frankfurt, DE – IE8 – DSL||4.536s||2.225s||3.591s|
|Stockholm, Sweden – IE8 – DSL||3.019s||1.903s||3.894s|
|**Israel – IE8 – DSL||4.446s||403 Error||2.189s|
|**Nairobi, Kenya – IE8||29.327s||403 Error||35.014s|
|Delhi, India – IE7 – DSL||8.699s||4.494s||9.532s|
|Bangalore, India – IE8 – DSL||10.470s||5.442s||5.825s|
|Hyderabad, India – IE8 – DSL||6.978s||4.787s||4.233s|
|Moscow, Russia – IE8 – DSL||3.447s||2.280s||3.531s|
|Singapore – IE8 – DSL||6.022s||4.712s||6.192s|
|Taipei, Taiwan – IE9 – DSL||5.045s||2.857s||4.009s|
|***Jiangsu, China – IE7 – DSL||3.728s||2.447s||8.083s|
|Seoul, Korea – IE 7 – DSL||7.080s||3.059s||7.240s|
|Tokyo, Japan – IE9 – DSL||4.288s||2.464s||3.803s|
|Sydney, AU – IE7 – DSL||8.211s||5.011s||7.412s|
|Wellington, NZ – IE8 – DSL||5.527s||4.072s||5.520s|
** CloudFlare returns 403 Error for those locations when using default (MEDIUM) security settings. Setting it to LOW solves that problem.
*** YouTube access blocked in China, so our current home page cannot fully load. Because of that I have used “Start Render” times instead of “Page Load”.
Torbit works in a different way than the other two services. First of all, they do not offer any security features but purely focused on optimizing and speed.
They are offering some unique optimizations such as HTML5 “Local Storage” or “DOM Storage”. Check out the blog post by Josh or read more about Local Storage.
They have a similar DNS based setup as other two but their CDN system works in the old fashioned way. Once they have your JS, CSS & images loaded in to their cache, URL’s pointing those files will be re-written to use their domains. When the browser connection limits are low (for example, it was 2 for IE7) that is very useful as it allows more simultaneous downloads which means faster page loads. But that is not the case with newer browsers as they already have higher connection limits and using multiple domains may reduce performance (that’s my opinion) because of the extra DNS lookups.
They also have limited (since not all images and/or browsers are compatible) support for WebP images. If all the conditions are matched, it will kick in and Torbit will serve WebP versions on those images for further speed up.
Their current benchmark numbers are almost on par with others. And if your site can benefit from the unique optimizations such as using Local Storage you should check them out today!
If you are primarily looking for security with the added benefit of some extra speed, you should check out Incapsula.
Their free account offers more protection than CloudFlare’s free account. Also they are offering few extra services such as uptime monitoring.
But Incapsula is a bit slower when it comes to the speed. I think that’s because of using less data centers (3 according to that post) spread across the world.
I must also not that I have signed but for their free (Personal) plan but they also have a 30 days trial. So, I had access to all features and tests were done with full acceleration.
On a side note, their setup is a bit different. Instead of a full DNS change you’ll just need to point your A & CNAME to provided locations.
As you can see from my tests CloudFlare provided the fastest access, period. At this time, they have 12 data centers spread around the globe and that information (along with datacenter status) is publicly accessible.
They were a bit slow (but just a bit) for the UK and AU/NZ region. But I’ve been told a new DC for UK is coming very soon.
Their setup is pretty straight forward but as they take over all of your DNS, you should be careful to not to miss any entries. Their system detected pretty much everything but I had to manually setup the TXT record for Domain Keys.
I should also mention about a problem I had, somehow my IP is locked out and I was receiving “502 Bad Gateway” error following a false “Web Page Not Available” message when I try to access my sites. But at the same time, sites were fully accessible to the rest of the world. That problem lasted for a while but eventually it returned to the normal.
There are also some reports of false errors over the web (Google “cloudflare 502”) but don’t forget CF is currently in beta.
Another thing to consider is the pages that is shown to users in case of a failure or blocking. Those pages contain CloudFlare links & logo and there are some who not fond of this. You may beg to differ but for a FREE service that is OK with me. And apparently PRO account will have some control over this very soon.
For me, CloudFlare’s WordPress plugin is better than the provided by Incapsula. Both resolves actual visitors IP’s so WordPress will show it correctly. But on top of that when you mark a comment as a spam, CloudFlare posts that information back to their system to prevent further access of that offending user.
Edit: CloudFlare also supports HTML LocalStorage and other some other tricks when their new (beta) Rocket Loader feature is enabled. Thanks to Matthew Prince of CloudFlare for letting me know.
All three services are very good. All has their strengths and weaknesses so you should pick one based on your website & your own requirements. But for Pixel Envision, my pick is: