FedEx still faster than the internet

FedexNet When you need to transfer very large amounts of data over the internet, sooner or later you will hit a limit where it will actually be faster to send that data on disks over regular mail (often called sneakernet). Internet transfer rates are simply not enough for large data sets.

Imagine a company with two offices in different cities, perhaps even in different countries. Each office has a 100 megabit internet connection. If the company needs to send a large amount of data from one office to the other, theoretically a 100 megabit connection can muster about 45 gigabyte in one hour if there are no bottlenecks on the way. This ends up being just over one terabyte of data in 24 hours.

In other words, for anything larger than one terabyte, it would be faster for this company to just send the data on disks for over-night delivery.

What does Google do?

Google’s initiative to transfer all the Hubble space telescope data is a good case study. The Hubble data takes up 120 terabyte (120,000,000,000,000 byte). How does Google transfer it? Not over the internet. Instead they send actual physical disk arrays via regular mail, something they have dubbed, for fun, FedExNet. This allows them to get the data within 24 hours.

To transfer the same amount over the internet in 24 hours, Google would have to be able to achieve transfer rates of more than 11 gigabit/s running constantly maxed out. On a regular 100 megabit connection, transferring 120 terabyte of data would take almost four months (111 days).

Is Internet 2 the answer?

The basic point here is that the internet still needs a lot more capacity. How soon can we reach such massive transfer rates? There is hope at the horizon, but FedEx probably hopes the answer is “never.” 🙂


    1. If you know you want to transfer the data the time to take a copy doesn’t really apply because you can just write it to twice as many discs at the same time. Also with transferring it over the internet you have to write it down eventually anyways, so while you do lose some parallelism you are removing the massive internet bottleneck.

      1. Still there’s some overhead in detaching the disks from one infrastructure and attaching them to the other one. On this front AWS has their import/export Snowball service with an 80TB case which is very delivery and import/export export friendly.

  1. Does this take into account the amount of time required to transfer the data from the drive that was shipped onto the drive where it will be used? It seems to me it would take quite a while to move it from ‘external drive X:’ to your internal hard drive….

Leave a Reply

Comments are moderated and not published in real time. All comments that are not related to the post will be removed.required