There isn’t a whole lot in the way of activity when it comes to new designs and advances in spinning magnetic hard drives these days. The capacity wars have plateau’d around 4TB or so. The next big threshold to cross is either Shingled recording or HAMR (which uses a laser to heat the surface just prior to a write being committed to the disk). Due to the technical advances required and the adoption by a slightly smaller field of manufacturers (there’s not as many here as there was a while ago) the speed at which higher density devices hit the market has slowed. We saw 1TB and 2TB quickly show up one after the other, but slowly eventually the 3TB and 4TB drives followed. And usually they were priced at the high end premium part of the market. Now Seagate has stitched together a 5TB drive and LaCie is rushing it into a number of its own desktop and pro-sumer level products.
The assumption for now is Seagate has adopted the shingled recording method (which folds writing of blocks of data in an overlapping pattern to increase the density). We’ll see how well that design decision performs over the coming months as the early adopters and fanbois needing each and every last terabyte of storage they can get for their game roms, warez and film/music collections.
Seagate is selling the drive today for $250. Cables to add new interfaces or support vary from $20 to $50. Internal drives are expected in the future but may wait until more systems can properly boot; using a larger than 2.1TB disk as a boot drive requires EFI firmware that most Windows PCs don’t have.
No doubt the internal version known as Constellation is still to be released. And take note EFI or Extensible Firmware Interface is the one thing differentiating Mac desktops from the large mass of Wintel desktops now on the market. Dell, HP, IBM, Acer, Asus, etc. are all wedded still to the old Intel BIOS based motherboard architecture. Mac along adopted EFI and has used it consistently since it first adopted Intel chips for its computer products. Now the necessity of EFI is becoming embarrassingly clear. Especially for the gamer fanboys out there who must have the largest hard drives on the market. Considering the size of these drives it’s amazing to think you could pack 4 of these into a Mac Pro desktop, and get 12TB of storage all internally connected.
Regarding the internals of the drive itself. Some speculation in this article included a suggestion that this hard drive used 4 platters total to reach 3GB of storage. Computing how many GBytes per platter this would require puts the density at 750 Gbytes/platter. This would mark a significant increase over the more common 640Gbytes/platter in currently shipping. In fact in a follow-up to this original announcement yesterday Seagate has announced it is using a total of 5 platters in this external hard drive. Which computes to 600 Gbytes/platter which is more inline with currently shipping single platter drives and even slightly less dense the the 640 GByte drives that are at the top of the storage density scale.
Some people may remember the poorly marketed and badly implemented Microsoft ReadyBoost technology hyped prior to the launch of Windows Vista. Microsoft’s intention was to speed throughput on machines without sufficient RAM memory to cache large parts of the Windows OS and shared libraries. By using a small Flash memory module on
the motherboard (Intel’s Turbo Memory) or by using a USB connected Flash memory stick one could create a Flash memory cache that would offset the effect of having 512MB or less RAM installed. In early testing done by folks like Anandtech and Tom’s Hardware system performance suffered terribly on computers with more than the 512MB of RAM targeted by Microsoft. By trying to use these techniques to offset the lack of RAM on computers with more than 512MB of RAM the computers ran slower using Vista. I had great hopes ReadyBoost at the time the flash cache method of speeding throughput on a desktop PC was heralding a new early of desktop PC performance. In the end it was all a myth created by the Microsoft marketing department.
Some time has passed since then Vista was released. RAM prices have slowly gone down. Even low end machines have more than adequate RAM installed to run Vista or now Windows 7 (no more machines with 512MB of RAM). The necessity of working around those limits of RAM is unnecessary. However total system level I/O has seen some gains through using somewhat expensive Flash based SSD (solid state disks). Really this is what we have all been waiting for all along. It’s flash memory modules like the ones Intel tried using for it’s ReadyDrive capable Turbo Memory technology. However these were wired into a PCIe controller and optimized for fast I/O, faster than a real spinning hard disk. The advantage over the ReadyBoost was the speed of the PCIe interface connected to the Flash memory chips. Enterprise data centers have begun using some Flash SSDs as caches with some very high end product using all Flash SSDs in their storage arrays. The entry level price though can be daunting to say the least. 500GB SSD disks are the top of the line, premium priced products and not likely to be sold in large quantity until the prices come down.
Seagate is now offering a product that has a hybrid Flash cache and spinning disk all tied into one SATA disk controller.
The beauty of this design is the OS doesn’t enter into the fray. So it’s OS agnostic. Similarly the OS doesn’t try to be a disk controller. Seagate manages all the details on its side of the SATA controller and OS just sees what it thinks is a hard disk that it sends read/write commands. In theory this sounds like a step up from simple spinning disks and maybe a step below a full flash based SSD. What is the performance of a hybrid drive like this?
As it turns out The Register did publish a follow-up with a quick benchmark (performed by Seagate) of the Seagate Moments XT compared to middle and top of the line spinning hard drives. The Seagate hybrid drive performs almost as well as an the Western Digital SSD included in the benchmark. That flash memory caches the stuff that needs quick access, and is able to refine what it stores over time based on what it is accessed most often by the OS. Your boot times speed up, file read/write times speed up all as a result of the internal controller on the hybrid drive. The availability if you check Amazon’s website is 1-2months which means you and I cannot yet purchase this item. But it’s encourage and I would like to see some more innovation in this product category. No doubt lots of optimization and algorithms can be tried out to balance the Flash memory and spinning hard disks. I say this because of the static ram cache that’s built into the Momentus XT which is 32MBytes in size. Decide when data goes in and out, which cache it uses (RAM or Flash) and when it finally gets written to disk is one of those difficult Computer Science type optimization problems. And there are likely as many answers as there are Computer Scientists to compute the problem. There will be lots of room to innovate if this product segment takes hold.
First let’s just take a quick look backwards to see what was considered state of the art a year ago. A company called STEC was making Flash-based hard drives and selling them to big players in the enterprise storage market like IBM and NetApp. I depends solely on The Register for this information as you can read here: STEC becalmed as Fusion-io streaks ahead
STEC flooded the market according to The Register and subsequently the people using their product were suddenly left with a glut of product using these Fibre Channel based Flash Drives (Solid State Disk Drives – SSD). And the gains in storage array performance followed. However the supply exceeded the demand and EMC is stuck with a raft of last year’s product that it hasn’t marked up and re-sold to its current customers. Which created an opening for a similar but sexier product Fusion-io and it’s PCIe based Flash hard drive. Why sexy?
The necessity of a Fibre Channel interface for the Enterprise Storage market has long been an accepted performance standard. You need at minimum the theoretical 6GB/sec of FC interfaces to compete. But for those in the middle levels of the Enterprise who don’t own the heavy iron of giant multi-terabyte storage arrays, there was/is now an entry point through the magic of the PCIe 2.0 interface. Any given PC whether a server or not will have open PCIe slots in which a
Fusion-io SSD card could be installed. That lower threshold (though not a lower price necessarily) has made Fusion-io the new darling for anyone wanting to add SSD throughput to their servers and storage systems. And now everyone wants Fusion-io not the re-branded STEC Fibre Channel SSDs everyone was buying a year ago.
Anyone who has studied history knows in the chain of human relations there’s always another competitor out there that wants to sit on your head. Enter LSI and Seagate with a new product for the wealthy, well-heeled purchasing agent at your local data center: LSI and Seagate take on Fusion-io with flash
Rather than create a better/smarter Fibre Channel SSD, LSI and Seagate are assembling a card that plugs into PCIe slot of a storage array or server to act as a high speed cache to the slower spinning disks. The Register refers to three form factors in the market now RamSan, STEC and Fusion-io. Because Fusion-io seems to have moved into the market at the right time and is selling like hot cakes, LSI/Seagate are targeting that particular form factor with it’s SSS6200.
STEC is also going to create a product with a PCIe interface and Micron is going to design a product too. LSI’s product will not be available to ship until the end of the year. In terms of performance the speeds being target are comparable between the Fusion-io Duo and the LSI SSS6200 (both using single level cell memory). So let the price war begin! Once we finally get some competition in the market I would hope the entry level price of Fusion-io (~$35,000) finally erodes a bit. It is a premium product right now intended to help some folks do some heavy lifting.
My hope for the future is we could see something comparable (though much less expensive and scaled down) available on desktop machines. I don’t care if it’s built-in to a spinning SATA hard drive (say as a high speed but very large cache) or some kind of card plugging into a bus on the motherboard (like the failed Intel Speed Boost cache). If a high speed flash cache could become part of the standard desktop PC architecture to sit in front of monstrous single hard drives (2TB or higher nowadays) we might get faster response from our OS of choice, and possible better optimization of reads/writes to fairly fast but incredibly dense and possibly more error prone HDDs. I say this after reading about the big charge by Western Digital to move from smaller blocks of data to the 4K block.
Much wailing and gnashing of teeth has accompanied the move recently by WD to address the issue of error correcting Cycle Redundancy Check (CRC) algorithms on the hard drives. Because 2Terabyte drives have so many 512bit blocks more and more time and space is taken up doing the CRC check as data is read and written to the drive. A larger block made up of 4096 bits instead of 512 makes the whole thing 4x less wasteful and possibly more reliable even if some space is wasted to small text files or web pages. I understand completely the implication and even more so, old-timers like Steve Gibson at GRC.com understand the danger of ever larger single hard drives. The potential for catastrophic loss of data as more data blocks need to be audited can numerically become overwhelming to even the fastest CPU and SATA bus. I think I remember Steve Gibson expressing doubts as to how large hard drives could theoretically become.
As the creator of the SpinRite data recovery utility he knows fundamentally the limits to the design of the Parallel ATA interface. Despite advances in speeds, error-correcting hasn’t changed and neither has the quality of the magnetic medium used on the spinning disks. One thing that has changed is the physical size of the blocks of data. They have gotten infinitesimally smaller with each larger size of disk storage. The smaller the block of data the more error correcting must be done. The more error-correcting the more space to write the error-correcting information. Gibson himself observers something as random as cosmic rays can flip bits within a block of data at those incredibly small scales of the block of data on a 2TByte disk.
So my hope for the future is a new look at the current state of the art motherboard, chipset, I/O bus architecture. Let’s find a middle level, safe area to store the data we’re working on, one that doesn’t spontaneously degrade or is too susceptible to random errors (ie cosmic rays). Let the Flash Cache’s flow, let’s get better throughput and let’s put disks into the class of reliable but slower backing stores for our SSDs.