As of a little after 19:00 UTC on 2 July 2014, Drupal.org is now delivering as many sites as possible via our EdgeCast CDN.

Why a CDN?

A content delivery network or content distribution network (CDN) is a large distributed system of servers deployed in multiple data centers across the Internet.[1]

We are primarily concerned with the network level security that a CDN will provide Drupal.org.

The CDN enables us to restrict access to our origin servers and disallow directly connecting to origin web nodes (which is currently possible). The two big advantages are:

  1. Accelerate cacheable content (static assets, static pages, etc).
  2. Allow us to easily manage network access and have a very large network in front of ours to absorb some levels of attacks.

Here are some examples of how the CDN helps Drupal.org:

  • We were having issues with a .js file on Drupal.org. The network was having routing issues to Europe and people were complaining about Drupal.org stalling on page loads. There was basically nothing we could do but wait for the route to get better. This should never be a problem again with EdgeCast's global network.
  • We constantly have reports of updates.drupal.org being blacklisted because it serves a ton of traffic coming in and out of a small number of IP addresses. This should also not happen again because the traffic is distributed through EdgeCast's network.
  • A few months ago we were under consistent attack from a group of IPs that was sub-HTTP and was saturating the origin network's bandwidth. We now have EdgeCast's large network in front of us that can 'take the beating'.

updates.drupal.org

By enabling EdgeCast's raw logs, rsync, and caching features, we were able to offload roughly 25 Mbps of traffic from our origin servers to EdgeCast. This change resulted in a drastic drop in origin network traffic, which freed up resources for Drupal.org. The use of rsync and the raw log features of EdgeCast enabled us to continue using our current project usage statistics tools. We do this by syncing the access logs from EdgeCast to Drupal.org’s utility server that processes project usage statistics.

CDN caching results screenshot

Drupal.org

Minutes after switching www.drupal.org to use the CDN, there were multiple reports of faster page load times from Europe and North America.

A quick check from France / webpagetest.org:
Pre-CDN results: first page load=4.387s. repeat view=2.155s
Post-CDN results: first page load=3.779s, repeat view=1.285s

Why was the www.drupal.org rename required?

Our CDN uses a combination of Anycast IP addresses and DNS trickery. Each region (Asia, North America, Europe, etc.) has an Anycast IP address associated with it. For example cs73.wac.edgecastcdn.net might resolve to 72.21.91.99 in North America, and 117.18.237.99 in Japan.

Since 72.21.91.99, 117.18.237.99, etc. are Anycast IPs, generally their routes are as short as possible, and the IP will route to whatever POP is closest. This improves network performance globally.

Why can't drupal.org be a CNAME?

The DNS trickery above works by using a CNAME DNS record. Drupal.org must be an A record because the root domain cannot be a CNAME. MX records and any other records are not allowed by the RFC on CNAME records. To work around this DNS limitation, Drupal.org URLs are now redirected to www.drupal.org.

 

 

Related issues
https://www.drupal.org/node/2087411
https://www.drupal.org/node/2238131

Comments

Anonymous’s picture

I personally never liked the redirection to drupal.org from www.drupal.org, so yey!

giorgio79’s picture

Ongoing attacks will inflate the CDN costs no? Why not use a solution like fail2ban to weed out malicious traffic http://www.fail2ban.org/wiki/index.php/Main_Page 

Some Drupal specific rules for Fail2Ban https://www.drupal.org/project/fail2ban

It's funny how Edgecast website does not quote any pricing info. Love these hush hush sales pages :)

Steven Jones’s picture

The issue with using something like Fail2Ban is that the traffic still needs to get to your firewall before you stop it. If you're under a DOS attack then your network and servers just won't be able to handle saying 'no' to all those packets fast enough. If you move that 'no' response much further out toward the origin of the requests, and have multiple servers filtering the requests, then you can support a much higher level of traffic.

markconroy’s picture

Great... but would be so much better if we could find an open-source solution for this.

gnulux’s picture

That would be great, markconroy, but the problem is a really free as in freedom solution would be a CDN built with free software and with Drupal's control.

Even if Edgecast ran its machines or virtual machines on Debian GNU/Linux, Drupal would have no control on them.

I may say something silly but I think that Drupal might have set up servers with some free software like CosyCloud and start running virtual machines. It's something you can do with CosyCloud, I watched a long presentation. You can run or stop virtual machines all over the world as needs be.

The problem with CDNs is that all our data, we the community, — pseudo, emails, posts, etc. — is stored on machines we or Drupal have no control over. We don't even know what is done with our data.

I understand Drupal's reasons but I don't like the decision.

yoroy’s picture

but what is a CDN? I have a vague notion, but defining the thing would help :)

SidneyGijzen’s picture

A CDN is a Content Delivery Network. More info.

basic’s picture

Thanks for the feedback @yoroy, I've added a line from Wikipedia as well as a link to the article on CDNs

budda’s picture

Another great example for Edgecast usage.

did you use any Drupal integration to purge edgecast at all?

apramakr’s picture

In my case, I use a CDN in front of the origin webserver.

On our homepage is a form with an AJAX post button. After a sucessfuk post submit, I end up getting 'Invalid form post data' messages due to expired form_ids.

I do not know how to make the CDN by pass caching just on this form.

Would the only solution be to NOT cache the html for this page?

Thanks!!

Castonguay118’s picture

Great at many websites and also if located you will be XHubs out all the video clips from the digital collection of Fine.

Oleary77’s picture

good see regarding Snapchat Online Login Internet.The middle section of https://snapchatonlinelogine.com the job Manymo emulator This emulator is a variation of great.