IEEE Future Directions
Related Articles
It is almost 35 years since the introduction of the MacIntosh Hard Drive 20 (September 17, 1985) of which I became a proud owner, bragging with my friends and colleague of its huge capacity, 20MB.
Now I could buy a 20 TB hard drive, at about the same price I had to pay for that one (1499$ suggested retail price), but if we take the inflation into account that hard drive was actually 2.5 times more expensive than today’s 20 TB hard drive.
Between a 20 MB and a 20TB there is a 1 million factor, that is a 2 **20, almost a doubling of capacity every 20 months, 19 if you factor in the decrease in price, pretty close to the Moore’s prediction of a doubling every 18 months.
If that 20MB hard disk seemed huge at that time, this 20TB hard disk seems huge today but as I bought that one 35 years ago I am most likely to consider buying this one in the near future.
On the other hand, are 20TB of storage really huge?
Every single day YouTube experience an upload of 432,000 hours of video clips. Now, let’s do some math:
- One hour of HD video on YouTube requires a 1.65 GB
- One hour of 4K video on YouTube requires 2.7 GB
- One hour of lower resolution video on YouTube requires some 0.26 GB
Note that you can code video in many different ways resulting in different storage requirement (that is why I specified “on YouTube”).
There are no detailed statistics on the percentage of uploads on YouTube in terms of video quality (at least I was not able to find anything looking accurate) but as a back of napkin calculation I would think that at least 50% are in HD quality (since smartphones today are filming at that quality and most uploads are from smartphones) with a very low number of 4K uploads (although it will grow in this decade) and the rest in low quality clips. Based on this, every day YouTube would have to add to its storage capacity 425TB (350TB of HD, 55TB of low quality video and some 20TB of 4k video).
As shown in the picture on the side this means that every day YouTube needs to add (not taking into account redundancy consideration) 4 racks of storage, every week an additional closet (a closet contains 28 racks for a capacity of 2.8 PB based on a 4TB single unit HD capacity).
When you deal with this kind of volumes it is clear that increase in storage capacity (density) is most welcome. In general, the big data centres are a bit slow in acquiring the latest generation technology because of cost, but give it a year or two and the cost will go down to the point of making the new technology competitive vs the cost of space to accommodate the storage.
At enterprise level there are niches where the extra cost of the newest technology is justified and these are the early adopters. I clearly come much later in terms of perceived cost benefit but I have no doubt that in 3 years time a 20TB hard disk will find a spot on my desk. I have seen my storage needs doubling every two years over the last twenty years since I shifted from film photography to digital photography, and I am not shooting any movie…
