Quartz 4

Content Distribution Networks (CDN)

3 min read

Content distribution networks (CDNs) provide scalability, load balancing, and low latency

  • Host content for their customers in web caches spread around the world in edge networks and data centres
  • Reduces load on the main servers – rather than keep a local copy of the files, they link to the CDN-hosted copy
  • Reduces latency to respond to requests
  • Reduces chances of successful denial-of-service attack

This is like a global cache/load balancing network

  • Highly effective for delivering and caching HTTP content
  • Global deployment – agreements with most large ISPs to host caching proxy servers
  • Only works with HTTP-based content – does not support RTP-based streaming

Ran by companies like:

  • Cloudflare
  • Fastly
  • Akamai

Many large companies run their own:

  • Google, Facebook, Netflix

Load Distribution

CDNs distribute load by locating web caches around the world and answering most requests from a local cache Needs extensive investment, large-scale cooperation with ISPs, Internet Exchange Points, etc. Mutual benefit for an ISP to host the servers for a CDN

  • Reduces the load on the ISP’s network
    • One copy of the content sent to cache; the cache then distributes many times
    • Avoids overload of link from ISP to outside world
    • e.g., Netflix distribute “tens of terabits per second” of video – not possible from a single data centre; needs local fanout
  • Increases the reach and robustness of the CDN

Latency

Key CDN benefit is reducing latency → content is delivered from nearby caches

  • e.g., requests to a US-based service are answered from a CDN cache in Europe, rather than having to cross long-distance path to the US Requires global distribution of CDN proxy caches
  • Are CDNs effectively serving the entire world?
  • Providing effective Internet access to developing regions requires more than just connectivity; also needs data centers and infrastructure to host the CDN nodes Effective for cacheable static content – video, software updates, images – may need edge compute infrastructure to support other applications

Finding your Nearest CDN

DNS

Locate nearest CDN node using the DNS Each resource hosted by the CDN has a unique DNS name – each resource is given a different domain name

  • e.g., the CDN hosts: https://example.com/images/kitten.jpg as https://9BC1C10B7947A890B9.cdn-example.com/kitten.jpg DNS server for the CDN returns different A or AAAA records for a name, depending on where it’s requested from, what CDN caches have the data Directs to local cache based on IP address of resolver making the query DNS client subnet extension for remote resolvers (if the client isnt in the same place as the subnet it can tell where it is) Doesn’t need to be particularly accurate – trying to figure out if you’re in UK rather than US, to direct you to data centre in London rather than New York Allows very fine-grained control, but puts high load on DNS

Anycast Routing

Each resource hosted by the CDN has a unique filename.

The DNS name always maps to the same IP address - that IP address is a load balancer at the entrance to a data centre

The CDN uses multiple Data Centers All using the same IP addresses

  • Each data centre advertises its addresses via Border Gateway Protocol (BGP)
  • Inter-domain Internet routing will ensure traffic goes to the closest data centre to source

Graph View

Backlinks