Hitachi NAS Platform adds primary deduplication

Hitachi Data Systems is offering primary deduplication in its Hitachi NAS Platform, a feature that was more than two years in the making.

That's about par for the course for primary dedupe. There was a lot of noise around the technology in 2010, followed by a lot of silence.

Hitachi NAS (HNAS) is the network-attached storage (NAS) platform Hitachi Data Systems (HDS) gained when it acquired BlueArc in 2011. BlueArc first revealed a license deal with primary dedupe software vendor Permabit in August 2010. HDS at that time sold BlueArc systems under an OEM deal, but BlueArc remained an independent company.

BlueArc still hadn't implemented dedupe at the time of the HDS acquisition, and it took HDS engineers more than a year to make it ready for commercial use. HDS made sure the primary dedupe didn't impact performance by offloading much of the performance to hardware.

Michael Hay, vice president of product planning for HDS, said Hitachi NAS uses Permabit's hash database, but "all the other heavy lifting is done by Hitachi." That includes using hardware acceleration and the ability to read deduped data without having to rehydrate it.

HNAS uses an object-based file system offload engine powered by field-programmable gate arrays (FPGAs) to accelerate the hashing and chunking involved with primary dedupe. A basic hashing/chunking engine is free with HNAS, and additional engines for parallel hashing/chunking are available for an additional license.

"We have a unique architecture with HNAS," Hay said. "Our file system is overlaid on two types of processors, and that gives us the ability to dedupe without compromise."

The summer of 2010 was a busy time for primary dedupe news. Dell acquired primary dedupe vendor Ocarina Networks and IBM bought primary compression startup Storwize two weeks apart in July 2010. Permabit revealed its deal with BlueArc a few weeks later. It also had an OEM deal with LSI for its Engenio storage, but that dissolved when NetApp acquired the Engenio product line in 2011.

IBM sells the old Storwize products as its Real-Time Compression appliances, but Dell has yet to add Ocarina dedupe to its EqualLogic or Compellent storage arrays.

"Primary dedupe is in no man's land right now," according to Taneja Group consulting analyst Arun Taneja. "I don't think it's made big strides yet. I think eventually it will happen, but most vendors haven't found a way to do it without affecting performance."

Permabit is pushing on, looking for OEM deals in other places. The vendor is talking to SMB storage vendors about using its software for Linux-based NAS systems, and Vice President of Marketing Mike Ivanov said he expects one announcement around mid-2013. He said several flash array vendors are also evaluating Permabit's dedupe software in hopes of bringing down the per-gigabyte cost of flash by optimizing capacity. However, there is no guarantee that any of those who use the software will identify it as Permabit's.

"A lot of people we're talking to have attempted to do their own dedupe and failed," Ivanov said. "They will never let the world know we're the OEM."

Primary dedupe is showing up in non-traditional storage systems. Startup GreenBytes uses what it calls "zero latency" inline dedupe in its IO Offload Engine built to improve performance of virtual desktop infrastructure storage. GreenBytes CEO Steve O'Donnell said his company actually began with designs on dedupe for backup. The dedupe algorithm it developed worked with little latency, which doesn't help much for backup, but is valuable primary data.

"For backup, zero latency is irrelevant. Nobody cares if it's 20 minutes of latency," O'Donnell said. "But for primary storage, our dedupe technology makes a lot of sense. It's a unique differentiator and perfect for highly virtualized workloads."

View the original article here

Read More

Denver Broncos score with NetApp filers for primary storage

When the Denver Broncos play at home, more than 76,000 fans can watch Peyton Manning and his teammates from their seats in Sports Authority Field at Mile High. What they might not realize is that their National Football League experience is more virtual, digitized and cutting-edge than ever.

Entertainment video for the scoreboard, playbooks

accessed field-side by iPads and video for post-game film study are among the IT requests that need to be available at a moment's notice.

"We're a small shop and almost everything we do is being digitized -- whether it's a server becoming virtualized, entertainment video for the scoreboard, player video or pictures. We have a lot of pictures from the history of the franchise, [and] they're all getting digitized," said Russ Trainor, vice president of IT for the Broncos. "So from a storage standpoint, it's really exploded for us."

The Broncos' IT department is responsible for two data centers: one at the stadium and the other at the team's practice site in Dove Valley, Colo. The department has 320 people responsible for handling 100 TB of data across the two sites. Two entry-level NetApp Inc. FAS2240 filers, one for each site, were purchased in July 2012. Chris Newman, an IT architect for the team, explained that having identical storage was essential to making everything easier to manage. The two arrays back up data to each other.

The Broncos began using NetApp filers in 2004 when they bought four FAS270s to store 1.4 TB of data. But because of budget constraints, the two environments evolved to have different NetApp filers: a FAS2040 at Sports Authority Field at Mile High, and a midrange FAS3140 at Dove Valley.

"We used to have a 3140 at the valley site, which actually was kind of an overkill for us," Newman said. "[The] 2040 we had here at the stadium was working fine, but we really wanted to get into that 64-bit platform and have it at both sites so we could eventually take advantage of clustering."

The Broncos' servers are 95% virtualized with VMware vSphere, and connect the two FAS2240s via Cisco Fibre Channel switches. They run Microsoft SQL Server 2008 and one Exchange 2010 server (with plans to add another in a couple months), and handle 46 virtual machines. The environment is managed through NetApp's OnCommand Unified Manager.

Trainor said the team sought a new system that could be clustered and support 10 Gigabit Ethernet to keep up with expanding virtualization. He said the Broncos bought the NetApp filers without testing any other products. "It's been steady, reliable and the 2240 gives us that performance bump as well," he said. "The 10 Gigabit backbone provides us with a nice growth path."

The Broncos also use the NetApp filers for disaster recovery. Backups are performed nightly using NetApp's Snapshot and SnapRestore technology, so there are full data copies of everything at both sites. Additionally, the team is saving as much as 25% of its capacity using NetApp's primary data deduplication feature.

Trainor and Newman both said they are looking forward to getting another FAS2240 for the 2013 season, which will be provided by a recent independent deal between the NFL and NetApp. It will give them an additional 120 TB of capacity that Trainor said will be used at the Dove Valley site primarily to store coaching video.

View the original article here

Read More

Permabit Technology extends primary data deduplication software

Permabit Technology Corp. this week added compression and replication applications that can be built into its Albireo Virtual Data Optimizer primary data deduplication software in hopes of making it more appealing to OEM vendors.

Permabit Technology developed Albireo dedupe for storage array vendors in 2010. It has only one announced OEM partner -- Hitachi Data Systems, which uses Albireo in its Hitachi network-attached storage (NAS) platform. Permabit added Albireo Virtual Data Optimizer (VDO) in 2011 to provide inline deduplication and thin provisioning for Linux-based storage. VDO is aimed at low-end NAS systems and flash arrays.

Permabit CEO Tom Cook mentioned NetGear, Synology, QNAP, Buffalo Technology and similar systems as potential small- to medium-sized VDO customers. He said vendors selling all-flash arrays are also candidates.

"VDO was designed in part to address the flash market," Cook said. "All-flash arrays commonly use Linux. We have licenses in that space, and we will work to implement compression and replication [into VDO implementations for flash]."

Albireo Compress and Replica are plug-ins for OEMs that use VDO. Compress performs inline block-level compression. Permabit claims it has a sequential write performance of 750 MB per second. It is a variation of the Lempel-Ziv compression algorithm, but optimized for random access block storage. Dedupe and compression are always on for systems running VDO Compress. VDO deduplicates redundant blocks and compresses other unique blocks.

Albireo Replica provides bandwidth optimization for data in transit, improving replication performance between storage systems running VDO deduplication. Replica only sends new blocks across the wire and it supports many-to-one replication. Cook said the lightweight application can be easily integrated into a VDO client.

Dave Simpson, a senior analyst at 451 Research, said dedupe and compression are valuable features for flash vendors looking to shrink the footprint of data on solid-state drives that have smaller capacities than most hard drives. He said flash startups in particular would benefit from getting data reduction through an OEM partner.

"This would not be easy for smaller flash guys to develop," he said. "I'm not ruling out big vendors as OEM partners, but they can build it. For the smaller vendors, this is not where they're going to put their R&D [research and development] dollars. It's complex stuff."

Permabit Technology's compression and replication applications will be available to OEM partners May 31.

View the original article here

Read More