UPDATE 11/28/2017

Over the last 4 years we’ve seen strong adoption of SDS with a growth rate over 100% YoY at OSNEXUS.   Today we see our storage grid technology in QuantaStor being a key differentiator for customers with hyper-scale deployments, rack-scale deployments and multi-site deployments.

Things have changed alot on the hardware side too with many manufactures like HPE, Cisco and SuperMicro intoducing specific servers and disk enclosures designed specifically for the Software Defined Storage market.   At HPE they’ve introduced the Apollo 4000 series which includes space for 68 drives and is ideal for Ceph deployments. Similarly Cisco has launched their UCS S3260 which does one better as it has two server sleds allowing one to use it for both Ceph or ZFS based clustered storage. And last SuperMicro has several options including a 45/60/90 bay and others in the 4U form factor that are ideal for Ceph.

Storage density has improved a lot too where 4 years ago a 1PB deployment using 4TB drives used up an entire rack. Today using 12TB drives it’s just one shelf.  Four years ago the average QuantaStor SDS deployment was <= 128TB today today our average deployment is petabyte scale.

IT administrators are also better informed today and many are SDS savvy.  What was once tech for early adopters is now tech for the mainstream and you see the reverse happening with traditional SAN/NAS products across all the major vendors (Dell EMC, HPE, IBM, etc).  Hardware flexibility, agility, and automation via REST APIs are the table-stakes today as IT admins need to do more and more each year with flat IT budgets and growing storage demands.  In short, that makes for a bright future for SDS and a great efficiency boost for IT organizations everywhere.

ORIGINAL ARTICLE

Software Defined Storage (SDS) has become something of an overloaded term to mean many things these days.  This article is about how I see it being defined, what it is and isn’t.

Fundamentally, SDS is about a sea change that’s happening in the storage industry, it’s about providing companies with a way to plan for the future and to manage the reality of > 40% year-over-year data growth rates.  The key feature of SDS that makes managing this growth possible is the use of commodity-off-the-shelf (COTS) server hardware.  By moving beyond proprietary storage hardware, companies can take control over their largest component of IT spend, namely storage. Much like the migration from mainframes to open-systems took place over a decade ago, today a migration is happening from mainframe-like proprietary hardware storage systems to SDS appliances built using COTS hardware.  In my view this is the only way you can explain the flat growth of the leading traditional storage companies in this exponential data growth environment.

Don’t Believe the Hype

There are a lot of storage solutions out there under the banner of SDS but I’m going to make the argument that these are not all true SDS systems.  Here are a few signs the storage appliance you’re looking at may be traditional proprietary hardware under an SDS umbrella:

• You’re being sold a system in a proprietary chassis or a white-box chassis with a custom bezel
  • Custom bezels look cool and there’s a lot of value to having the storage vendor constrain the hardware compatibility list to a narrow subset so that problems are easier to triage and solve. That’s great, but be careful that the fancy bezel doesn’t eliminate your ability negotiate the price of the hardware vs the price of the software or limit your ability to upgrade the system.
• The price you’re paying on a per TB basis is equal to or higher than traditional Tier 1 systems.
  • If you’re cost per TB is not considerably lower you need to ask whether or not it is sustainable at this price and your data growth rate for the next 4-5 years let alone on into the future.
• You have little to no ability to customize the hardware or software in the appliance such as adding more RAM, NICs, reporting software, etc
  • At OS NEXUS we get asked frequently can I add additional software or hardware to the system and because the platform is modular and Linux based usually the answer is often yes.  Just as you can add more RAM and NICs to your VMware servers you should have the flexibility to expand your storage appliances using approved commodity hardware when you need it.
• The appliance APIs are not publicly available and/or are not REST APIs
  • If you can’t automate the appliance or if all they provide you with is antiquated SNMP, SMIS, or C/Java API interface that’s a red flag that indicates the vendor is selling rehashed legacy kit under an SDS banner.  Your appliances should be easily script-able with REST APIs and a solid CLI with XML output so you can fully automate common provisioning and configuration tasks.
• The appliance has a completely proprietary IO stack / filesystem / volume manager
  • There’s no doubt plenty of room for great innovations in filesystems but they take a long time to mature (typically 10 years), and if the underlying filesystem is proprietary, you’re effectively locked in.  Great for the storage vendor, not so good for you.
• The web interface looks like something from a circa ’99 DSL modem
  • You’ve got to maintain the box for the life of the data and that can be a long time.  When you have new engineers join your IT team you need to make sure the interface is intuitive and designed with good access controls.  Without that, your new IT staff can and inevitably will shoot themselves (and you) in the foot.

Taking a brief look at the state of IT today it’s clear that open-source-software (OSS) is king.  OpenStack, Hadoop, MongoDB, Gluster, Ceph, ZFS and many other open storage technologies are the tip of the spear of the cloud revolution.  It’s a market that demands a near zero-cost entry point with commercial support and management software for commercial deployments.  When you choose an SDS solution that’s built on open storage I/O stack you not only have a community behind it, you have an insurance policy that the technology will be there long into the future.

A Focus on OSS

This philosophy is central to what we do at OS NEXUS and is why the majority of our innovations are focused on our scale-out storage management layer and in making storage easy to manage.  For us it’s about making the best and most reliable SDS platform on the market by leveraging and integrating with the best enterprise OSS and commercial commodity hardware available.  We do that, package the SDS platform up as downloadable ISO image, and provide fanatical customer support so you have the peace-of-mind you need when you deploy and realize the benefits of SDS for your business.

Categories: Filesystems, Storage Appliance Hardware