Fitting More In
N
N
0Overall Score

Less is more in the crammed world of data storage, and yet the aim is to give everyone access to all human knowledge.

As if the World Wide Web wasn’t big enough already, the Internet Archive’s Wayback machine collects and stores snapshots of web pages over time to create a permanent record of the ever changing and evolving cyberspace. The Internet Archive’s Wayback project is only one of a number of initiatives. The non-profit organisation also collects audio, video and software. It has some 20,000 live concert recordings stored as well as a significant number of movies, books, and so on. The goal is to provide universal access to all human knowledge, no less.
Since 1996 the Wayback machine has accumulated over 40 billion web pages and grows at the rate of 20TB a month. The storage demand is significant. The machine consists of row upon row of HP servers in racks. According to the Archive, the computers cost about US$1620 each, assuming eight 120GB drives per machine. With 20 per rack, plus a switch, the cost per rack is around US$33,000 not including electrical build out, HVAC, cabling, physical racks and shelves.
Much of the archive is stored on these hundreds of slightly modified x86 Linux servers. But over the past year the organisation has been migrating to a new storage regime and has commissioned a full Petabyte of storage from Capricorn Technologies.
Deployed progressively over the last nine months, Capricorn reached the Petabyte milestone in late June. The high-density device consists of an array of over 600 clustered storage computers with 2500 spinning disks. The scalable solution is made of 19-inch racks able to support up to 64TB of raw disk space. These are in turn made up of 1U storage nodes with a capacity of 1.6TB.
This high density is achieved through an economy of design that consumes as little as 50 watts per TB. Such modest power consumption provides a host of benefits – low operating temperatures improve reliability while operating and cooling costs are kept to a minimum.
Not everybody needs this kind of storage, though plenty of large organisations are quickly heading this direction. Industry research specialist IDC recently announced some startling figures on the growth in storage demand, predicting that virtualisation technologies and the impost of compliance regulation will drive the storage market over the next four years. IDC says that over 50 per cent of all storage TB (38 per cent of the worldwide storage revenue) shipped in 2009 will be based on capacity-oriented disk drives. These external disk systems are expected to grow at a compound annual growth rate (CAGR) of nearly 60 per cent.
Not that internal storage is at a standstill. Internal TBs will increase at 37 per cent CAGR over the same period. By 2009 this will price the HDD market at US$26.3 billion, up from US$22.6 billion in 2004, indicating just how much the cost per TB will decrease. Cramming more data into less space is a priority for storage vendors, whether they are putting together subsystems or building the drives, there is a relentless need to increase data density.
With form factors pretty much cast in stone, there are only really two options. Adding more platters is one solution, but it is limited because the thickness of the drive is as fixed as the diameter. More spinning platters also means more weight, more power and more heat.
The other option is to increase the density of data on each platter. That’s the technique that has got us to where we are today. Reducing the granularity of the recording sectors requires ever more accurate engineering in the read/write heads. That’s fine, but we are reaching the limits and to gain significant capacity increases manufacturers are looking at another approach.
The latest strategy is perpendicular recording, a technology that has been around for 20 years but is still not technically feasible in mass HDD manufacture. The idea of perpendicular recording is a simple one. Instead of making the magnetised particles north/south poles running from left to right, you make them run up and down, as it were. One of the big advantages of this technique is that it also reduces the repulsion between adjoining bits so the space saving is even higher.
Researchers anticipate the switch to perpendicular recording will ensure they can continue to increase disk capacities through till about 2010. By then they anticipate new storage technologies, such as thermally assisted magnetic recording (which involves using a laser to heat particles so they can store data) will take over from traditional magnetic read/write heads.