CloudFlare vs. Incapsula - Benchmark Review
As I have noted in my previous blog post, I’ve been testing out few CDN services for use with this website; CloudFlare, Incapsula & Torbit.
Currently, Torbit is in invite only beta. Josh Fraser was kind enough to provide me an access but since their system is not public yet, Torbit is not included in the benchmarks. But I have included a short review based on my recent experience. I’m planing to update this post with benchmark numbers in the future as soon as they are eligible…
How I tested
For the load time benchmarks, I have decided to use WebPageTest.org as it is free, providing actual browser results instead of simulated results and offering enough locations to see global timing. In the advanced settings, I have selected “First View Only” & 5 test runs to make sure content get in to the cache. I have repeated those test for multiple times at the different times of day to make sure server load won’t make a big difference.
| Test Location First View only> Test runs: 5 | InMotion Business Class East Coast | CloudFlare Basic (Free) Account Medium Security (Default) | Incapsula Free Account Trial period (All Features) |
|---|---|---|---|
| Dulles, VA – IE8 – DSL | 1.624s | 1.567s | 1.792s |
| New York, NY – IE8 – DSL | 1.685s | 1.596s | 1.675s |
| *Atlanta, GA – IE8 – DSL | 1.853s | 1.740s | N/A |
| Chicago, IL – IE9 – DSL | 3.130s | 1.715s | 2.032s |
| Kansas City, MO – IE8 – DSL | 2.105s | 1.730s | 2.158s |
| San Jose, CA – IE8 – DSL | 3.806s | 1.704s | 2.510s |
| San Francisco, CA – IE9 – DSL | 2.620s | 3.090s | 2.571s |
| Los Angeles, CA – IE8 – DSL | 2.953s | 1.987s | 2.150s |
| Montreal, Canada – IE8 – DSL | 2.153s | 2.003s | 2.107s |
| São Paulo, Brasil – IE8 – DSL | 3.700s | 3.292s | 3.773s |
| Dublin, Ireland – IE7 – DSL | 4.185s | 2.673s | 2.604s |
| Gloucester, UK – IE7 – DSL | 4.761s | 3.306s | 3.330s |
| London, UK – IE8 – DSL | 3.833s | 2.265s | 2.038s |
| Paris, FR – IE8 – DSL | 2.934s | 1.921s | 2.209s |
| Amsterdam, NL – IE8 – DSL | 3.826s | 2.967s | 3.566s |
| Geneva, Switzerland – IE8 – DSL | 3.350s | 2.332s | 3.239s |
| Frankfurt, DE – IE8 – DSL | 4.536s | 2.225s | 3.591s |
| Stockholm, Sweden – IE8 – DSL | 3.019s | 1.903s | 3.894s |
| **Israel – IE8 – DSL | 4.446s | 403 Error | 2.189s |
| **Nairobi, Kenya – IE8 | 29.327s | 403 Error | 35.014s |
| Delhi, India – IE7 – DSL | 8.699s | 4.494s | 9.532s |
| Bangalore, India – IE8 – DSL | 10.470s | 5.442s | 5.825s |
| Hyderabad, India – IE8 – DSL | 6.978s | 4.787s | 4.233s |
| Moscow, Russia – IE8 – DSL | 3.447s | 2.280s | 3.531s |
| Singapore – IE8 – DSL | 6.022s | 4.712s | 6.192s |
| Taipei, Taiwan – IE9 – DSL | 5.045s | 2.857s | 4.009s |
| ***Jiangsu, China – IE7 – DSL | 3.728s | 2.447s | 8.083s |
| Seoul, Korea – IE 7 – DSL | 7.080s | 3.059s | 7.240s |
| Tokyo, Japan – IE9 – DSL | 4.288s | 2.464s | 3.803s |
| Sydney, AU – IE7 – DSL | 8.211s | 5.011s | 7.412s |
| Wellington, NZ – IE8 – DSL | 5.527s | 4.072s | 5.520s |
Remarks
** CloudFlare returns 403 Error for those locations when using default (MEDIUM) security settings. Setting it to LOW solves that problem.
*** YouTube access blocked in China, so our current home page cannot fully load. Because of that I have used “Start Render” times instead of “Page Load”.
Torbit Review
Torbit works in a different way than the other two services. First of all, they do not offer any security features but purely focused on optimizing and speed.
They are offering some unique optimizations such as HTML5 “Local Storage” or “DOM Storage”. Check out the blog post by Josh or read more about Local Storage.
They have a similar DNS based setup as other two but their CDN system works in the old fashioned way. Once they have your JS, CSS & images loaded in to their cache, URL’s pointing those files will be re-written to use their domains. When the browser connection limits are low (for example, it was 2 for IE7) that is very useful as it allows more simultaneous downloads which means faster page loads. But that is not the case with newer browsers as they already have higher connection limits and using multiple domains may reduce performance (that’s my opinion) because of the extra DNS lookups.
They also have limited (since not all images and/or browsers are compatible) support for WebP images. If all the conditions are matched, it will kick in and Torbit will serve WebP versions on those images for further speed up.
Their current benchmark numbers are almost on par with others. And if your site can benefit from the unique optimizations such as using Local Storage you should check them out today!
Incapsula Review
If you are primarily looking for security with the added benefit of some extra speed, you should check out Incapsula.
Their free account offers more protection than CloudFlare’s free account. Also they are offering few extra services such as uptime monitoring.
But Incapsula is a bit slower when it comes to the speed. I think that’s because of using less data centers (3 according to that post) spread across the world.
I must also not that I have signed but for their free (Personal) plan but they also have a 30 days trial. So, I had access to all features and tests were done with full acceleration.
On a side note, their setup is a bit different. Instead of a full DNS change you’ll just need to point your A & CNAME to provided locations.
CloudFlare Review
As you can see from my tests CloudFlare provided the fastest access, period. At this time, they have 12 data centers spread around the globe and that information (along with datacenter status) is publicly accessible.
They were a bit slow (but just a bit) for the UK and AU/NZ region. But I’ve been told a new DC for UK is coming very soon.
Their setup is pretty straight forward but as they take over all of your DNS, you should be careful to not to miss any entries. Their system detected pretty much everything but I had to manually setup the TXT record for Domain Keys.
I should also mention about a problem I had, somehow my IP is locked out and I was receiving “502 Bad Gateway” error following a false “Web Page Not Available” message when I try to access my sites. But at the same time, sites were fully accessible to the rest of the world. That problem lasted for a while but eventually it returned to the normal.
There are also some reports of false errors over the web (Google “cloudflare 502”) but don’t forget CF is currently in beta.
Another thing to consider is the pages that is shown to users in case of a failure or blocking. Those pages contain CloudFlare links & logo and there are some who not fond of this. You may beg to differ but for a FREE service that is OK with me. And apparently PRO account will have some control over this very soon.
For me, CloudFlare’s WordPress plugin is better than the provided by Incapsula. Both resolves actual visitors IP’s so WordPress will show it correctly. But on top of that when you mark a comment as a spam, CloudFlare posts that information back to their system to prevent further access of that offending user.
Edit: CloudFlare also supports HTML LocalStorage and other some other tricks when their new (beta) Rocket Loader feature is enabled. Thanks to Matthew Prince of CloudFlare for letting me know.
Verdict
All three services are very good. All has their strengths and weaknesses so you should pick one based on your website & your own requirements. But for Pixel Envision, my pick is:
This is a good acceleration focused review but today I’ve read this Cloudflare vs Incapsula security focused comparison and I must admit I was take back by the lack of CF security features.
See here:
http://www.ehackingnews.com/2012/11/incapsula-vs-cloudflare-security-review.html
Bottom line: CF failed 90% (17/18) of security tests while Incapsula had 85% (16/18) success rate.
The person who did the review was a customer of both services and he documented his test in video and a 22 page article…
I`m using Incapsula already and thought about switching but not anymore.
Thanks for posting this. I agree Incapsula seems better for security, but on our case speed is more important, that’s why we’re using CloudFlare. Being 100% free also helps… :)
Btw, do you know if the reviewer used free or paid version of Cloud Flare for the tests? Pro or Business account offers more protection… Also worth to mention that Incapsula’s free and personal plans does not have any firewall protection features. You’ll need $59/month business plan for that…
Thank you for your sharing.
I just try to use Incapsula for my blog and would like to see how it going for the next few days.
Hopefully everything is okay.
I am using Incapula, since it offers better security protection than cloudflare, but when my website exceeds the traffic limits, I will switch to CF, the latter offers no limited traffic.
Yeah, I agree with Jan. Inacpsula provides better security than and is as speedy as CloudFlare.
Just a follow-up on my comment above. I just wanted to let your readers know that Incapsula has just released a major overhaul of their dashboard and user options. This may not impact performance that much, but it certainly is a big step towards better security, with richer reporting details and extremely fine-tunable security options, making it now very easy to pinpoint and select which traffic to allow and which to reject.
Thanks for an excellent review and comparison. As a previous-CloudFlare-now-Incapsula user I would agree that Incapsula offers more security than CloudFlare.
I did also test performance for both, but I used watchmouse.com, sampling from 60 random locations every 15 minutes for three days straight, showing less difference in performance than your test. Essentially, I found no significant difference between CloudFlare and Incapsula as far as speed.
Looking at yottaa.com over a period of a month for each, with 4 sampling points every day, I found the same: no significant difference in page loading times.
For full results and my opinion on either service, see http://www.husdal.com/2011/07/01/incapsula-versus-cloudflare/
To me, as far as performance goes, both are equal, at least on my own site. Not all sites are created equal and it could be different for other sites. The only way to find out is to test, as you did, and then decide which one that suits your needs better.
I picked Incapsula, for security reasons.