Throughout this article, I mentioned and referred to various storage types available on AWS. If you need a quick refresher, please refer to the previous article titled "Storage Types with Associated Characteristics in AWS."
Hybrid storage is one of the four storage types available on AWS, the remaining three being object, block, and file storage. Actually, hybrid storage differs from the rest in that it can integrate any of these three types of storage with on-premises systems, hence the name hybrid.
The dedicated hybrid solution on AWS is AWS Storage Gateway, which comes in different flavors: File Gateway, Volume Gateway, and Tape Gateway.
Apart from AWS Storage Gateway, various storage services and tools offer to some extent hybrid features, like AWS DataSync, AWS Snow Family, Amazon Elastic File System (EFS).
AWS Storage Gateway
As mentid in introduction a truelly hybrid storage that can integrate on-premise and AWS cloud storage solutions covering various use cases hence three different types of this service which are cover below. It's worth to notice that it can be used as a VM or hardware appliance.
File Gateway
Related to file-based storage, the concept of File Gateway is to extend your local (on-premise) storage with cloud storage, which can be either Amazon S3 or Amazon FSx for Windows File Server, depending on the protocol your environment utilizes.
File Gateway helps seamlessly integrate with the aforementioned AWS cloud storage services, providing all the benefits of their services such as scalability, durability, availability, and cost-effectiveness.
Typically, File Gateway is used for data archiving, backup, and making on-premises data available for cloud-based processing and analytics.
A notable feature is local caching for frequently accessed files, reducing costs and enhancing performance.
It utilizes NFS (for Linux) and SMB (for Windows) protocols which are mainly designed for sharing files over network.
Volume Gateway
Volume Gateway differs from File Gateway in that it uses the iSCSI protocol, which is primarily utilized in Storage Area Networks (SANs) with block storage instead of file storage.
Similar to File Gateway, Volume Gateway extends local storage into the AWS cloud, enabling the storage and retrieval of data over the network. It can operate in two different modes depending on requirements.
Stored Volumes
In this mode, you store data locally while asynchronously sending data to AWS in the form of EBS snapshots. All data is accessed locally, and the data sent to AWS serves as a backup or for disaster recovery purposes.
Cached Volumes
This mode is the opposite of Stored Volumes. Most data is accessed over the network from Amazon S3, while the most frequently accessed data is cached locally to enhance performance and reduce costs.
Tape Gateway
Tape Gateway aims at environments that utilize physical tape-based backup systems and want to continue using the same tape-based software but with the benefits of cloud storage. To mimic physical tapes, it utilizes a Virtual Tape Library (VTL) and stores data in Amazon S3 Glacier Flexible Retrieval or Amazon S3 Glacier Deep Archive.
Similar to traditional tape-based solutions, Tape Gateway primarily serves as an archiving and backup solution. However, you can't access data directly after it's stored on S3 Glacier due to the characteristics of S3 Glacier. Accessing or retrieving the stored data requires using the AWS Storage Gateway console or the AWS Storage Gateway API.
AWS DataSync
AWS DataSync is not a hybrid storage service, but it plays a crucial role in hybrid storage architectures. The service facilitates data transfer between on-premises storage systems and AWS services, as well as between AWS services themselves.
It's commonly used for data migration because it can greatly accelerate data transfers through parallel transfers, supports integration with a plethora of storage systems, can analyze local storage to better plan migration, which can be fully automated and scheduled.
Utilizing this service is a good idea when first migrating data to AWS.
AWS Snow Family
The AWS Snow Family can be considered as a hybrid storage solution due to its computing capabilities, such as those offered by Snowball Edge. However, it is primarily designed for the physical transfer of data into and out of AWS. It is available in three main configurations (with variations within the Snowball category):
Snowcone
Supports up to 14 TB of data and is suitable for edge computing and data transfer in environments with limited connectivity.
Snowball
Available in different versions to serve various needs, including storage-optimized and compute-optimized options, with capacities of up to 80 TB for storage-optimized and 42 TB for compute-optimized Snowball Edge devices.
Snowmobile
A truck-mounted solution capable of migrating up to 100 PB of data, ideal for extremely large-scale data center migrations to AWS.
Common use cases include transferring large amounts of data securely, especially where the network is insecure or weak. The entire Snow Family is rugged, designed to securely transport data under various environmental conditions.
References
Hybrid Cloud Storage | Amazon Web Services
AWS Storage Gateway Documentation
AWS Storage Gateway Hardware Appliance
What is Amazon S3 File Gateway
What is Amazon FSx File Gateway?
What is Volume Gateway?
What is AWS DataSync? - AWS DataSync
AWS DataSync FAQs | Amazon Web Services
Simplify multicloud data movement wherever data is stored with AWS DataSync
AWS Snow Family FAQs | Amazon Web Services
Using Amazon S3 compatible storage on Snow Family devices - AWS Snowball Edge Developer Guide