Tape had always been the preferred medium for datacenter archive, even though it could be difficult to deal with and was a medium that (because it literally touched the recording head) degraded every time you recorded or read the tape. Plenty of enterprise information is archived on tape, but that is about to change. We all know that information is growing exponentially and now archiving is struggling to keep up. With the expectation of 44 zettabytes of data being created by 2020 and archive data being the fastest part of this growth, tape vendors should be in a great position. But, tape leaders like Oracle and IBM might have more to fear from this huge increase in data than they have to gain. And it may come from the most unlikely place of all: optical. We recently looked at Sony Optical Archive Inc. and how their entrance into enterprise archiving could give tape–and hard drives for that matter–a real run for the money. You can download our paper here.
(Photo source: Sony)
There are different types of storage in a datacenter. Primary storage is for the data that servers use most, and it gets the fastest drives (often the lightning-fast SSDs) and the most attention. This is often the company databases, the Exchange mailboxes and other applications that run the business. The rest of the storage (which you can think of as all of those working files that are sitting on the network file shares) is typically running on slower, but still enterprise-class hard drives for reliability.
Archive data is the fastest growing segment of the enterprise data market, but also the least sexy. Previously it was comprised of years of old records, backups or snapshots of system state; information that was no longer deemed important enough for expensive storage systems. It had normally been relegated to the lowest cost medium because there was so much of it, and slower access times were tolerable. But that is about to change.
Archive data is becoming much more critical because in some areas there can be huge penalties for not properly archiving and retrieving data. Financial organizations, for instance, have to record every transaction and be able to play it back, in sequence, if the government ever decides that something in the market looked fishy. Some medical regulations require maintaining archives of data indefinitely.
Another new and large source of archive data is the Internet of Things (IoT) where millions of sensors on products are streaming real-time telemetry that must be captured, stored and then analyzed. This IoT information can be mined over and over again to help discern trends, improve products or even create new business models. Because of the repeated access, you really don’t want that on a media like tape that degrades with each access.
High performance computing (HPC) labs are creating huge multi-petabyte data sets that need to be loaded and analyzed. Tape had been used in the past, but tape can be slow, which is not a good situation when highly paid researchers are sitting around waiting for data to load.
Because the stakes are high and tape is being pushed to its boundaries, Sony Optical Archive has developed an optical library that can hold up to 181PB in a single system (4 of which can be tied together for 724PB of total storage). By storing data on optical, customers can literally store everything, virtually forever, and retrieve it in no time at all. How much is 181PB? If you consider that all of the data for Google Maps and Google Earth combined is roughly 20PB, you could get 9 copies of all of that data on a single Sony Everspan Library, and those four libraries tied together could probably hold a whole Google Solar System (when they get around to mapping that).
Some had already seen the shortcomings in tape and had been looking to hard drives for archiving, but this is not without its faults. Backblaze recently reported their status on hard drive failures and found a 5.84% annual failure rate for hard drives, meaning they might not be any better for archiving in the long run because over time, they are still magnetic media and still fail. When you try to stack tape or hard drives up against Sony’s 100-year warranty on media, you see those can’t even come close to competing. These optical drives are not the same as the consumer-grade drives that had been sold in the past to address PC archiving. The industrial-strength Archive Drives (AD) are designed for enterprise use.
With huge investments in tape technology already in the market today, it is doubtful that we’ll see customers ripping out their tape libraries and replacing them with optical tomorrow. But the economics, performance, durability and reliability of optical will make it an interesting addition to many datacenters, especially those dealing with big data/analytics, HPC, or IoT in markets like financial, medical, government or telecommunications. Tape won’t disappear soon, but for those that are dealing with the crush of information or the heavy hand of regulations, archiving on optical may be a much better solution. Especially when the media can be read for 100 years. Sony even makes it easy to transition as there is an LTO tape emulation that can make the Everspan Library connect to existing environments just like a tape library (as an Amazon S3 interface for cloud customers or even a direct interface if needed.) To find out more about optical is changing archiving, download our paper here.