Object Stores edit   discuss  

Our list of and information on commercial, open source and cloud based object stores, including Amazon S3, Azure Blob Storage, Google Cloud Storage, OpenStack Swift, Scality, Caringo and alternatives to these.

Category Definition

Storage solutions whereby data is stored without any concept of folders or organisational structure, instead being referenced by a unique identifier, allowing for massively parallel and scalable solutions. Generally access via a REST API, with Amazon S3 the defacto standard, although many also support a range of file based interfaces as well, simulating a file system using the underlying object storage. Common features include support for multiple storage tiers, storage of custom metadata against data, replication of data for redundancy, and object versioning.

Further Information

Good explanation of object stores in comparison to file and block stores: http://searchcloudstorage.techtarget.com/feature/How-an-object-store-differs-from-file-and-block-storage

Excellent list of object stores including their history: http://www.theregister.co.uk/2016/07/15/the_history_boys_cas_and_object_storage_map/

Object Store Services

The three major cloud vendors all have object store services as follows:

Amazon S3Eventually consistent object store service
Azure Blob StorageStrongly consistent object store service, with support for multiple object types (block, page and append)
Google Cloud StorageStrongly consistent object store service

All other “Enterprise” cloud vendors will also have a comparable offering, generally including Amazon S3 compatibility. Many are based on OpenStack Swift, including IBM Bluemix Object Storage and Rackspace Cloud Files.

Other object storage services include:

WasabiUltra low cost high performance object storage service - https://wasabi.com/
BackBlaze B2Claims to have the lowest cost high performance object storage service - https://www.backblaze.com/b2/cloud-storage.html

There are also a wide range of other smaller cloud vendors that provide object storage solutions, and many of the object store technologies listed below are also available as a managed service.

The following analyst material covers a number of technologies in this category:

Open Source Object Store Technologies

The following are all Open Source object store technologies:

OpenStack SwiftSupports multiple configurable storage tiers and backing storage; part of the OpenStack suite but can be installed stand-alone
Zenko CloudServerOpen source object storage server based on the S3 compatible API from Scality RING, previously known as Scality S3 Server
CephDistributed object store that also supports block and file storage, with development led by RedHat - http://ceph.com/
Basho Riak S2 (formally CS)http://basho.com/products/riak-s2/
Joyent (Samsung) Triton Object Storagehttps://www.joyent.com/triton/object-storage
LinkedIn Ambryhttps://github.com/linkedin/ambry
Open IOhttp://openio.io/

Commercial Object Store Technologies

Options for on premise object stores from enterprise vendors, many of which are also available as a service or as a hardware and software appliance, include:

SwiftStackCommercial on premise software solution based on OpenStack Swift with a number of added management features and synchronisation to the cloud, sold by the largest contributor to Swift - https://www.swiftstack.com/
Scality RINGNative object store with POSIX filesystem support, and a range of object, file and OpenStack compatible APIs
Caringo Swarm (formally CAStore)Software and appliance solution - https://www.caringo.com/
Cloudian HyperStoreSoftware and appliance solution - https://cloudian.com/products/
Dell EMC Elastic Cloud Storage (ECS)https://www.emc.com/en-us/storage/ecs/
DDN WOShttp://www.ddn.com/products/object-storage-web-object-scaler-wos/
RedHat Cephhttps://www.redhat.com/en/technologies/storage/ceph
HGST (Western Digial) ActiveScalehttps://www.hgst.com/products/systems
Hitachi (HDS) Content Platform (HCP)https://www.hds.com/en-us/products-solutions/storage/content-platform.html
IBM Cloud Object Storage (previously Cleversafe)https://www.ibm.com/cloud-computing/products/storage/object-storage/cloud/
NetApp StorageGRID Webscalehttp://www.netapp.com/us/products/data-management-software/object-storage-grid-sds.aspx

The following are other object store products:

Quantum Lattus Object Storagehttp://www.quantum.com/products/bigdatamanagement/lattus/index.aspx

The following analyst material covers a number of technologies in this category:

Object Store Caches

RubixOpen Source object store cache for Hadoop compatible tools from Qubole - https://www.qubole.com/rubix/

Parallel Distributed Filesystems

Many parallel distributed filesystems (such as Gluster) also support object store interfaces, simulating an object store using the underlying filesystem.

Historical Technologies

The following technologies are no longer sold or maintained:

CoreOS TorusDevelopment stopped Feb 2017 - https://github.com/coreos/torus
SkylableWebsite no longer live - http://www.skylable.com

Blog Posts