Home / Blog / Give me more storage, ’cause I need more storage

Give me more storage, ’cause I need more storage

The 20TB internal hard drive was announced in September 2019 and is now ready to hit the market. Image credit: Western Digital

It is almost 35 years since the introduction of the MacIntosh Hard Drive 20 (September 17, 1985) of which I became a proud owner, bragging with my friends and colleague of its huge capacity, 20MB.

Now I could buy a 20 TB hard drive, at about the same price I had to pay for that one (1499$ suggested retail price), but if we take the inflation into account that hard drive was actually 2.5 times more expensive than today’s 20 TB hard drive.

Between a 20 MB and a 20TB there is a 1 million factor, that is a 2 **20, almost a doubling of capacity every 20 months, 19 if you factor in the decrease in price, pretty close to the Moore’s prediction of a doubling every 18 months.

If that 20MB hard disk seemed huge at that time, this 20TB hard disk seems huge today but as I bought that one 35 years ago I am most likely to consider buying this one in the near future.
On the other hand, are 20TB of storage really huge?

Every single day YouTube experience an upload of 432,000 hours of video clips. Now, let’s do some math:

  1. One hour of HD video on YouTube requires a 1.65 GB
  2. One hour of 4K video on YouTube requires 2.7 GB
  3. One hour of lower resolution video on YouTube requires some 0.26 GB
A storage rack containing 28 hard drives. Using lower cost HD with a 4TB capacity, this means some 100TB per rack. Image Credit HP Enterprise

Note that you can code video in many different ways resulting in different storage requirement (that is why I specified “on YouTube”).

There are no detailed statistics on the percentage of uploads on YouTube in terms of video quality (at least I was not able to find anything looking accurate) but as a back of napkin calculation I would think that at least 50% are in HD quality (since smartphones today are filming at that quality and most uploads are from smartphones) with a very low number of 4K uploads (although it will grow in this decade) and the rest in low quality clips. Based on this, every day YouTube would have to add to its storage capacity  425TB  (350TB of HD, 55TB of low quality video and some 20TB of 4k video).

As shown in the picture on the side this means that every day YouTube needs to add (not taking into account redundancy consideration) 4 racks of storage, every week an additional closet (a closet contains 28 racks for a capacity of 2.8 PB based on a 4TB single unit HD capacity).
When you deal with this kind of volumes it is clear that increase in storage capacity (density) is most welcome. In general, the big data centres are a bit slow in acquiring the latest generation technology because of cost, but give it a year or two and the cost will go down to the point of making the new technology competitive vs the cost of space to accommodate the storage.

At enterprise level there are niches where the extra cost of the newest technology is justified and these are the early adopters. I clearly come much later in terms of perceived cost benefit but I have no doubt that in 3 years time a 20TB hard disk will find a spot on my desk. I have seen my storage needs doubling every two years over the last twenty years since I shifted from film photography to digital photography, and I am not shooting any movie…

About Roberto Saracco

Roberto Saracco fell in love with technology and its implications long time ago. His background is in math and computer science. Until April 2017 he led the EIT Digital Italian Node and then was head of the Industrial Doctoral School of EIT Digital up to September 2018. Previously, up to December 2011 he was the Director of the Telecom Italia Future Centre in Venice, looking at the interplay of technology evolution, economics and society. At the turn of the century he led a World Bank-Infodev project to stimulate entrepreneurship in Latin America. He is a senior member of IEEE where he leads the New Initiative Committee and co-chairs the Digital Reality Initiative. He is a member of the IEEE in 2050 Ad Hoc Committee. He teaches a Master course on Technology Forecasting and Market impact at the University of Trento. He has published over 100 papers in journals and magazines and 14 books.