Hybrid cloud is go-anywhere storage

Businesses can, and do, use object storage in their data centers and as local storage, to handle larger volumes of larger files.

Organizations in the media, research and development, scientific research, medicine and government use object storage, especially for media and imaging data. Here, the benefits are mostly around availability and resilience, and of course scalability.

Openstack Swift, for example, is used to store genetic research data, as well as data files for mass-market online games.

And object storage is also being pushed forward by its use in the cloud.

Amazon’s cloud storage architectures, including S3 and Glacier, are object-based storage systems.

AWS’s S3 architecture is so widely used that it has become a standard for large-scale, cloud based data stores. Cloud-based application suppliers increasingly write their software so it works with S3, and S3 is being used to build services such as cloud-based backup and recovery. Nor is S3, and object storage, limited to public clouds: Private clouds use it too.

But to see object storage as either an architecture for the data center, or a building block for cloud applications, is to miss much of its potential.

As object storage is hardware- and therefore location-independent as well as scalable, it can cover both use cases. Vitally, it allows data to move between on-premise and cloud-based storage, and makes it easier to share computing workloads between the two, so is potentially ideal for hybrid cloud.

Hybrid cloud dreams

One of the challenges for developers of early cloud computing was data storage.

The conventional, hierarchical systems used for local storage become increasingly inefficient at scale. File system paths also force developers to work with fixed links between applications and storage.

One of the benefits of the cloud is being able to aggregate resources, and scale capacity up or down on demand. This works well enough with compute resources, but until developers adopted object-based storage, it was a challenge for data.

With object storage, it’s possible to design architectures that are hardware- and location-agnostic. In turn, this allows businesses to run the same workloads locally, in private and in public clouds.

With hybrid cloud object storage, there is a greater chance of fine-tuning the architecture to meet performance and budget requirements, as well as other constraints such as data location and security.

Businesses can, for example, use local resources for processing-intensive tasks but tap into the cloud for lower-cost longer-term storage, or for additional compute capacity.

Movie studios, for example, use cloud computing for rendering, especially overnight when cloud charges for compute instances are generally lowest.

But for that to work in practice, the cloud-based compute instance needs to be able to read and write to local, or private cloud, storage. Object storage can do this, because it uses a single namespace for data.

Furthermore, hybrid object storage isn’t restricted to scenarios where organizations want data to span on-premises, private and public clouds. Multi-cloud systems are increasingly common.

Object storage supplier Scality calculates that most enterprises already use four or more clouds. Object storage allows application designers to put data and workloads on whichever cloud system offers the best features, performance and cost – and to move them again, if the need arises.

To do this, the object storage system needs to keep the data in the application’s native format, but that is possible with today’s technology. Bloomberg Media, for example, uses hybrid and multi-cloud resources to store its vast quantities of video and other files.

As Enterprise Strategy Group senior analyst Scott Sinclair puts it: “With hybrid cloud object storage you can extend on-premise object storage into the public cloud, as essentially a single massive storage pool.”

Ready to deploy?

Organizations looking at hybrid cloud object storage are likely to be those that already store vast amounts of information, or that expect their data storage needs to grow rapidly.

The industries at the forefront of the technology’s adoption have largely been those in social media, gaming, video and film production, and scientific research because they handle large volumes of large files. Usually, they already have substantial experience of object-based storage, in-house, in the cloud, or both.

But for any organization looking to move to hybrid cloud object storage, the starting point should be the data. Suppliers increasingly offer powerful services and tools that can move data across all physical storage categories, and orchestrate them in a way to maximize efficiency and performance. Only the business, though, can decide if a data store suits their own resilience, security and privacy criteria.

Hybrid cloud object storage suppliers

Caringo

Texas-based Caringo offers Swarm ‘scale out hybrid storage’. Swarm operates as software, hybrid cloud (on Microsoft’s Azure platform) and as integrated hardware as Swarm servers.

DataDirect Networks (DDN)

Data-direct Networks provides WOS, an object storage architecture that supports S3, Swift and REST application programming interfaces. The company provides hardware and software-only systems for use with industry-standard equipment. The company is 15 years old and has a strong presence in financial services and manufacturing, as well as cloud services.

DellEMC

Dell EMC is the enterprise and storage arm of Dell Technologies. Dell’s takeover of EMC in 2015 was the industry’s largest at the time. Dell EMC offers software-defined object storage under its Elastic Cloud Storage brand, but also offers appliance and software-only object storage systems. It claims a significantly lower total cost of ownership (TCO) than standard public cloud.

Hitachi GST

HGST, now part of Western Digital, brands its object storage system as Active-scale. This is provided as a turnkey solution, and is focused on large capacity and low TCO operations. HGST systems come with an S3 connector, so they can be combined with cloud-based storage for a hybrid system.

HPE

HPE has a number of object storage systems under its Scaleable Object Storage label. This includes a partnership with Scality to offer its RING hybrid cloud storage architecture, as a software-based system. HPE supports connectors to AWS and Microsoft Azure.

IBM

IBM offers a cloud object storage architecture via RESTful APIs. This allows applications to connect directly to object storage systems, and IBM cloud services, as well as operating across disparate datacentres. There are also S3 connections, as well as specialist services for collaboration, file sharing and archiving.

NetApp

NetApp’s StorageGRID is its ‘web scale’ object storage offering, available as an appliance-based system or as software. It can also deploy on VMware, OpenStack and Docker.

Scality

Scality RING is a software-defined object storage system that works on industry-standard hardware, and includes S3 compatible APIs. The technology supports S3, for cloud, and NFS, for local volumes, and a connector for Azure Blob Storage. Scality has also developed Zenko, an open source multi-cloud controller, for businesses that want to span storage across cloud providers.