Enterprises’ need for storage is increasing exponentially asbusinesses use more and more rich media such as audio and video bothwithin their networks and on customer-facing websites.
That demand is going to continue growing, driven both by customerdemand and demand from new hires who grew up with the Internet andwant the same ease of use and features they already have at home.
However, storage is expensive, and a decent subsystem runs into themultiple tens of thousands of dollars. This is driven manycorporations to consider cloud computing and cloud storage solutions,and the Storage Networking Industry Association (SNIA) earlier thisyear put out a paper on managing private and hybrid clouds for data storage.
Open storage systems are generally less expensive, but despite this, manyenterprises elect to remain with their proprietary storage solutions.
“Customers may be on the bleeding edge when choosing applications, butwhen it comes to fundamental infrastructure like storage, they’re riskaverse,” Sakthi Chandra, senior director of marketing at Nexenta, toldLinuxInsider.
Money Isn’t Everything
Why wouldn’t enterprises leap onto the open storage bandwagon,especially in these tough times?
The answer is complex. Open storage is useful in certaincircumstances, but may not be suitable at other times, and someenterprises are working judiciously with a mix of both.
Even calculating savings due to using open storage isn’t a simple matter.
“It isn’t simply a matter of cost so much as it is cost for relativeperformance,” Charles King, principal analyst at Pund-IT, toldLinuxInsider. “If a company’s storage requirements are relativelysimple, open source products could be seen as viable alternatives toproprietary solutions.”
Some open source solutions are being optimized or integrated forspecific solutions, King said. For example, the ZFS file system,developed at Sun Microsystems, is being used in Oracle’s Exadatasolution. Oracle owns Sun, by the way, so perhaps that factored intothe equation.
However, “I’m not certain what the future of ZFS or the other open sourceprojects at Oracle will be,” King remarked. “Mike Shapiro, who ledthe ZFS development team, was one of the three inventors of DTracediagnostics and was vice president of open storage at Oracle, resignedrecently. All three of the DTrace engineers have left Oracle, as haveseveral MySQL engineers.”
ZFS is a combined file system and logical volume manager designed bySun Microsystems that offers support for high storage capacities,integration of the concepts of file system and volume management andoffers snapshots and copy-on-write clones. Other features arecontinuous integrity checking, automatic repair and Raid-Z. It isimplemented as open source software.
DTrace is a dynamic tracing framework created by Sun fortroubleshooting problems in kernels and applications in productionsystems in real time. It has been released under the CommonDevelopment and Distribution License.
The Old Order Changeth
The drive toward virtualization and its complement, cloud computing,are forcing new approaches to storage, Nexenta’s Chandra suggested.
“In the past five years, there’s been a shift in how IT approachesinfrastructure that has been driven by the reality of virtualization,”Chandra explained. “Further, the idea of cloud computing gives yougreat economics — if you look at Amazon’s S3, the pricing of storageis down to cents per terabyte depending on when you look at itspricing, and IT is questioning why its internal IT structure can’tmatch that cost.”
Virtualization and the cloud are forcing IT shops to reevaluate theirbroad storage strategy, Chandra said.
Take the issue of desktop virtualization, for example. “When youvirtualize your desktops and move compute resources back into the data,it impacts the network,” Chandra pointed out. “Also, you’re taking thedata on commodity storage from your laptop and pushing it back intodata center storage, which is provided by SAN and NAS vendors. Thecosts don’t add up.”
The problem isn’t one of adding more storage subsystems but of keepingstorage capabilities in lockstep with virtualization, Chandra said.ZFS lets users bring on “thousands of machines in minutes instead ofhours” because it allows unlimited snapshotting, he added.
Virtualization issues like this are forcing IT to fundamentallyrethink storage and consider how to accommodate virtualization,bringing in the economics of cloud computing, Chandra said.
For example, traditional storage systems offer hardware redundancy inRAID (Redundant Arrays of Inexpensive Disks) configurations, but opensource solutions go beyond that problem altogether.
“The fundamental assumption ZFS makes is that your hardware systemsare bound to fail,” Chandra remarked. “So we take the approach thatit’s the software’s responsibility to provide consistency throughoutthe lifespan of the data. Using copy on write, every time before we doa write of the data, we copy it and make sure it’s available in an openstorage disk somewhere else.”
Hopping Onto the Open Source Train
However, proprietary storage systems vendors haven’t been napping at the wheel.
“One of the way that vendors of proprietary storage systems havemaintained or increased the innovation of their products is byincorporating increasingly sophisticated software functionality that’sdifficult for open source vendors to keep up with,” Pund-IT’s Kingpointed out.
Add that to the trust built up over years of business dealings, andproprietary storage product vendors appear to be in a good position.
However, “One of the issues with the big guys is that their architectures arerooted in legacy design and the applications that are becoming moreand more popular today are very different from those written 10 to 15years ago,” Jack O’Brien, vice president of marketing at Gluster,told LinuxInsider.
“Everything’s more dynamic now, everything’s designed to be stored inthe virtual server environment,” O’Brien explained. “Today theunstructured file base data is exploding, and people are looking tomove off the legacy systems because these aren’t meeting theirrequirements.”
The Open Window of Opportunity
Storage virtualization technology lags behind server technology, andthat provides a window of opportunity for open source storage solutionvendors, O’Brien pointed out. “Traditionally, people went with SANsfor virtual storage but that meant they had to provision one SAN forevery virtual machine. The situation’s better now but you still haveto provision a lot of storage and manage a lot of LAN connections, andhave to deal with input/output (I/O) bottlenecks.”
Open source storage solutions are able to run huge files, and expandingthem is relatively easy. “We can scale to petabytes, so you can createas many virtual machines as you want and take as many snapshots as youwant,” O’Brien said. “We put everything under one large single globalname space that any number of virtual machines can access at one mountpoint. If you need more I/O you just scale out your hardware, unlikeproprietary systems where you have to go buy a new system or do aforklift upgrade.”
Further, open source storage solutions are becoming more sophisticatedas third-party add-ons that offer high-end features evolve. Forexample, OpendedUp, an open source deduplication file system forLinux that’s also known as “SDFS,” emerged in March. This is designedfor enterprises with virtual environments looking for ahigh-performance, scalable deduplication system that’s not too costly.
Other open source add-ins that provide storage management are alsoemerging, signaling a move among open source storage solutions towardsmeeting the needs of the enterprise.
“Open source serves the needs of smaller companies with acost-efficient alternative, and forces innovation and service fromproprietary vendors,” Jake Sorofman, chief marketing officer at rPath,told LinuxInsider. “Whether or not open source is a true alternativein a given category, it typically creates advantages for end userenterprises and forces commercial providers to deliver more innovationand value.”
It’s not likely that open source storage solutions will supplanttraditional ones. It’s more probable that, like other new technologiesin computing, they’ll alter the face of their industry but thenthey’ll coexist with existing solutions.