Many enterprises are looking for easy ways to migrate large amounts of unstructured file and object data to the cloud, but understanding the available options and their trade-offs can be confusing.
In this post, we will cover what Cloud Storage Gateways are, the use cases they address, and the pros and cons of using cloud storage gateways for cloud data migrations.
What is a Cloud Storage Gateway?
A Cloud Storage Gateway is infrastructure that is designed to front the cloud. As the name suggests, they are an on-premises “gateway” to data in the cloud and they fulfill this function by moving all data to the cloud and then caching a subset of the cloud data locally.
How a Cloud Storage Gateway works:
- Local Infrastructure: Cloud Storage Gateways are typically hardware-based since they have to serve hot data from the cache. Many vendors also offer virtual appliance options for smaller deployments.
- 100% Data in the Cloud + Cache Overhead: Cloud Storage Gateways typically put all the data in the cloud and then cache some data locally. So, if you are using a Cloud Storage Gateway for 100TB, then all 100TB of data is in the cloud and a subset of it (maybe 20TB or 30TB) is also cached locally. This means you may need 130TB of infrastructure to house 100TB of data. Depending on the size of the local cache, this may be larger.
- New Storage Silo: A Cloud Storage Gateway is a new storage infrastructure that caches some data locally and keeps all of the data in the cloud. It replaces your existing Network Attached Storage (NAS), it does not work with it.
Use Cases for Cloud Storage Gateways
Cloud Storage Gateways are used when you need low latency access to data that is in the cloud and you do not want to use your existing NAS to deliver that low-latency access.
Some common Cloud Storage Gateway use cases are:
- Backups to the Cloud: Use a Cloud Storage Gateway to backup data to the cloud and then locally restore backups.
- File Server for Cloud Data at Branches: If you need a local cache at a branch site for data in the cloud and you do not want to use a NAS, a Cloud Storage Gateway could be a local file server.
Why Cloud Storage Gateways are Not a Good Choice for Cloud Data Migrations
Data migrations to the cloud require fast, reliable data movement to the cloud. They do not require a local “gateway” to the moved data. Migration software is all that is required. Cloud data migrations are not a core use case for Cloud Storage Gateways. In fact, using a Cloud Storage Gateway to handle data migrations results in a mismatch and unnecessary overhead in costs and time.
Local caching of moved data by Cloud Storage Gateways creates unnecessary overhead that is not acceptable for the cloud data migration use case. Cloud data migration requires fast data movement and a way to elastically spin up a lot of capacity initially to move the bulk of the data and then spin down this capacity to do incremental updates. A Cloud Storage Gateway is not designed for such elastic scaling and will either result in slow initial transfer or over-provisioning of resources. Also, insufficient planning is a primary reason why cloud data migrations fail. To plan data migrations, you need analytics of on-premises NAS and object data to understand what data you have and how it’s being used, so you can prioritize the migrations. Cloud Storage Gateways typically do not provide such analytics.
Komprise Elastic Data Migration is designed to deliver fast, efficient, reliable cloud data migrations for both file and object data without any unnecessary “gateway” overhead or expensive infrastructure. Komprise scales to migrate petabytes of data and spin down once the data migration is complete. Komprise Elastic Data Migration optimizes migrations to the cloud and is shown to perform 27 times faster – download the white paper to learn more.
For customers who do not want to migrate entirely off their NAS, Komprise offers a transparent way to archive cold data to the cloud while maintaining all data access from the original NAS – without creating another gateway silo. Read the TMT white paper to learn more.
Here is a table that summarizes the common cloud data migration requirements and the differences between Komprise Elastic Data Migration and Cloud Storage Gateways:
| Core Feature for Cloud Data Migrations | Cloud Storage Gateways | Komprise Elastic Data Migration |
| Migration Planning | No. Lacks any data migration planning since this is not a core use case. | Yes. Provides analytics of any NFS, SMB, and object stores to plan your data migration strategy. |
| Fast Data Migrations | No. Fixed infrastructure for caching means you cannot elastically add and remove capacity as needed for fast migrations. | Yes. Elastically increase VMs to migrate data 27 times faster and spin down once the initial data migration is completed. |
| Manage Migration Iterations | No. Since Cloud Storage Gateways are not designed for data migration, they do not have any capabilities to manage data migration iterations. | Yes. Intuitive dashboards and API to manage data migration iterations and monitor progress. |
| Minimize Migration Infrastructure | No. Cloud Storage Gateways require hardware appliances or dedicated virtual appliances since they act as primary storage. There is no easy way to increase and shrink capacity dynamically to support data migrations. | Yes. Elastic architecture designed to dynamically grow and shrink virtual capacity as needed to handle data migration loads. |
| Outside the Data Path | No. Cloud Storage Gateways front all the data they move and this can result in a high amount of IO to them. | Yes. Komprise migrates data outside the data path — you continue using the original NAS until you are ready to cutover. |
| MD5 Checksum Reporting | No. Since data migration is not a core use case, no reporting on data migrations are available. | Yes. Komprise reports on data migrations, and reports results of MD5 checksums on every file. This is useful for organizations that require an audit log for compliance reasons. |
| Minimize Cutovers | No. Since Cloud Storage Gateways are not designed for data migration, they do not provide mechanisms to cutover to the cloud or minimize disruption unless you use the Cloud Storage Gateway as the new NAS. | Yes. |
In summary, cloud data migrations require an elastic data management solution that can provide data migration planning, fast data movement, and efficient cutovers with minimal downtime. Cloud Storage Gateways are not designed to enable cloud data migrations and are not tailored to handle this use case. Using Cloud Storage Gateways for data migrations results in unnecessary headaches, costs, time, and complexity.
Be sure to also read the white paper: Cloud File Tiering: Storage-Based vs. Gateways vs. File Based
