High-Performance Computing (HPC) is a fast-growing field and with the popularization of AI and deep learning technologies, it’s only growing faster. HPC spans many industries and involves large datasets and computationally intensive applications such as Monte Carlo simulations, CAE, Seismic Data Processing, computational fluid dynamics, optical design and 3D simulations. To get results quickly the storage must be fast, highly scalable and be able to ingest and deliver storage to large compute clusters with ultra-low latency.

Storage Grids for HPC

QuantaStor’s storage grid technology helps organizations meet the HPC scalability challenges by combining appliances together into a grid that can deliver over 3GB per second per node with scalability to over 64 nodes per grid using hybrid HDD+SSD storage. QuantaStor storage grids are scalable to over 100PB which enables HPC organizations to expand by adding disk and appliances to meet almost any HPC performance and scale requirement. To ensure low latency QuantaStor offers a number of network interconnects including Ethernet and Infiniband with RDMA acceleration. This combination of software and hardware is available on all QuantaStor hardware Reference Configurations including Intel, Supermicro, Dell, HPE, Cisco, and Lenovo.

Versatility

HPC applications vary widely in their protocol use from file (NFS), block (FC/iSCSI), to object (S3/SWIFT). QuantaStor supports all these protocols and can be dynamically re-configured via the web interface, REST API, and/or CLI to meet the needs of specific HPC workloads. This gives organizations the tools to run various different HPC workloads with different requirements without having to purchase separate solutions with single protocol support.

Tear it down, build it up – Automated HPC

QuantaStor includes many features catered towards the HPC field, one of which is the unique grid technology built into the software. QuantaStor’s grid technology allows IT Administrators to link together a mix of local and remote appliances into a grid and fully automate operations via the QuantaStor REST API and CLI. API call routing to the correct storage appliance is automatically handled by the storage grid so administrators can use the REST API, CLI and web management interface through any appliance in a storage grid to manage and monitor all appliances in a given storage grid. The need for collaboration between hardware and software is one of the biggest trends in HPC, and QuantaStor’s grid technology can help solve this.

Open Storage Integrated

Another trend in HPC is the need to leverage open source software due to its high level of innovation and adoption of HPC technologies. QuantaStor is Linux based and integrates with popular enterprise open source technologies including Ceph, ZFS, and Gluster. OSNEXUS prides itself on its open source community participation and sponsorship of events like the OpenZFS Developer Summit.

Operational Simplicity

The unique hyperscale grid technology built into QuantaStor enables organizations to deploy massive HPC clusters with just a few lines of scripting or via the QuantaStor web API in just a few clicks. Where teams often take weeks deploying Ceph using low-level tools and scripts QuantaStor makes it easy to do in just three dialogs and in just a few minutes. QuantaStor’s hardware integration and call-home system make it easy for IT Administrators to identify and replace bad components like failed disk drives, instead of having to use low-level tools and scripting to gather the necessary information to triage, identify, and fix hardware and software configuration issues. QuantaStor is Linux based and is derivative of Ubuntu Server LTS releases. This helps OSNEXUS keep current with security updates and makes it possible for HPC organizations to customize the systems with additional software and monitoring tools.

Ceph + RDMA

An additional benefit of Ceph for HPC is combining it with the new Remote Data Memory Access (RDMA) technology that Mellanox has been behind. RDMA allows applications to pass information over the network with ultra-low latency and with much lower CPU load. This is a critical advantage for HPC organizations because it increases storage performance and lowers cost and latency.

Summary

For more information on designing High-Performance Computing storage solutions using QuantaStor storage grid technology, visit osnexus.com or contact us at sdr@osnexus.com.

Categories: Storage Appliance Hardware