Comparing CDN Performance (Part 2)

Last week I wrote an article comparing Amazon S3 and Cloud Front to Rackspace Cloud Files. Since then I have gotten additional requests to compare more content delivery networks. The following is a comparison of Amazon Cloud Front, Rackspace Cloud Files, SimpleCDN, and GoGrid CDN. I tried to cover as many bases as I could. Please let me know if I missed anything.

Response Time

We will start by taking a look at the response time for the three services over a period of 1 week

Not only did Rackspace Cloud Files the lowest average response time, it also had the fastest response time overall in the one week time frame. GoGrid CDN also proved to be a good contestant with an average response time of a mere one millisecond over that of Rackspace Cloud Files. While Amazon Cloud Front had quite a bit higher average response time, it did have the lowest, slowest response time with 522ms as the longest response time. SimpleCDN has slowest response time of over two seconds, which is just awful for a content delivery network.

Pricing

Here’s a quick breakdown of pricing for Amazon and Rackspace CDN services.

*After June 30th inbound data transfer is $0.10 / GB

†Variable Cost; Decreases as more space or bandwidth is used.

††Storage not necessary, more information in the features section.

‡Higher charges apply for other regions such as Japan. Click here to view the full pricing model.

I wouldn’t consider SimpleCDN to be a true pay-as-you-go service, they require you to add funds to your account, with the smallest amount being $50. You can view the cost calculator for the services here. Rackspace Cloud Files, Amazon Web Services, SimpleCDN, (GoGrid CDN currently has no cost estimation calculator). Keep in mind when you setup the Amazon Web Services monthly calculator you will need to add S3 and Cloud Front. The S3 account will hold all of your data, and the outbound bandwidth will vary depending on how many files you have, their sizes, and how often the Cloud Front endpoints refresh their cache of your data. Cloud Front will then be where you allocate your total projected outbound bandwidth and average object size (which is to calculate GET requests I’m guessing) as well as allocate percentages to different regions, since Amazon charges differently per GB depending on the region the server responding to the request is located.

Speed

Next up is a speed test. I’ve uploaded a 100MB file to Amazon S, Rackspace Cloud Files, SimpleCDN, and GoGrid CDN to download from various locations. Listed below is the time taken (in seconds) to download the 100MB file. Each test was done three times with the fastest of the three tests listed below.

Not counting the slow residential ISP download rate we can see that Amazon Cloud Front came out ahead on the speed tests with an average download speed of 39.3MB/s. Coming in second was GoGrid CDN with 36.9MB/s followed by Rackspace Cloud Files at 34.3MB/s and finally SimpleCDN with just 28.3MB/s. SimpleCDN had some good tests, however, with a fastest speed of 54MB/s clocked from Amazon’s networks in Northern California. The 3.09MB/s speed was what really brought it down, which came from ThePlanet’s network in Texas. All these tests were done between 7PM and 9PM EST and are subject to change throughout the day due to network congestion.

Features

What’s a CDN without features? Well, it’s just a CDN. But hey, that’s what a CDN does, deliver content. Anything above and beyond that is an added bonus. Lets go over the various features Amazon and Rackspace have to offer.

GoGrid also offers live streaming, allowing you to push your live content to a rtmp server and having others tune in. GoGrid charges you their standard rate for how much bandwidth you use, however, SimpleCDN charges you a flat monthly fee for Live services starting at $59 a month for 250 concurrent views and going up from there. On the plus side, there is no bandwidth charges for their Live services, and no limit to the video bitrate. You would have to do the math to determine if GoGrid or SimpleCDN would be more cost-effective for you. Chances are if you plan to do a lot of streaming, or stream high-definition content, SimpleCDN might be the better  bet for you.

GoGrid and SimpleCDN offer something else that Rackspace and Amazon don’t, the ability to map a CDN URL to your server. What I mean is, instead of uploading files to GoGrid and SimpleCDN and having to pay monthly storage costs, you could just host the files on your server and use what they call Customer Origin/Mirror Buckets. After you map one of these to a URL where you keep all of your files, requests to the file through the CDN url will cause that end point to fetch the data from your own server and cache it for future requests. You’ll be able to get the same CDN delivery speed with none of the CDN storage costs.

Using Amazon Cloud Front requires an Amazon S3 account. Having an S3 account means you can use its torrent feature. If you add “?torrent” to the end of any S3 URL, the server will return a .torrent file instead of the file requested. This is especially great for large files since you will have other users helping you out with distribution, lowering your bandwidth cost. The torrent uses Amazons’ tracker and their S3 machines are seeding the file. This way your torrent will never die, so long as the file remains on the S3 servers. Once again, while this isn’t a feature of their CDN service, it’s still a feature of S3, which is required to use their CDN service. If it’s streaming video you want, then GoGrid CDN or SimpleCDN is the choice for you. Both offer Windows Media and Flash Streaming services for your content. Using these will allow users to fast forward through your video without having to wait for it to load to that point. GoGrid also offers a way to throttle bandwidth usage. This is especially useful and can save you a ton on bandwidth costs. Limiting the users bandwidth will allow the video to buffer and offer a constant viewing experience, however, it will prevent the entire video from buffering too quickly. This way, if the user decides they do not wish the watch the entire video, you won’t have to pay for the bandwidth of the entire video, only that which has been buffered.

Conclusion

GoGrid CDN is still in beta and is relatively new. However, it’s proven so far to have very low response times like Rackspace Cloud Files. While GoGrid wasn’t as fast as Amazon Cloud Front, it was still in second place and achieved very high speeds. SimpleCDN used to be a great CDN service, giving users streaming capabilities with their media. However, now with GoGrid CDN stepping up to the plate with the same streaming services as SimpleCDN, it looks like it’s taken the lead. The great thing about pay-as-you-go CDN networks is that you can use them all, and not have to pay  large monthly fees. For example, I’m using GoGrid CDN to deliver streaming video content, while using Amazon Cloud Front to deliver my static content. It all comes down to what you need. And if you need to stream live high-definition content to thousands of people at the same time, well then SimpleCDN just might be the best thing for you. Each CDN has their own features, and it’s up to you to decide exactly what you need. With pay-as-you-go services, you really have nothing to lose but your time.

Update (December 17, 2009)

I’m in the process of adding 2 more CDN networks to the charts along with different tests (i.e. rtmp streaming, downloading many small images, and anything else I can think of). A new article will be posted with the new test results along with pretty charts and graphs!