Data Management Glossary
Elastic Data Migration
What is Elastic Data Migration?
Data migration is the process of moving data (eg files, objects) from one storage environment to another, but Elastic Data Migration is a high-performance migration solution from Komprise using a parallelized, multi-processing, multi-threaded approach that speeds NAS-to-NAS and NAS-to-cloud migrations in a fraction of the traditional time and cost.
Standard Data Migration
- NAS Data Migration – move files from a Network Attached Storage (NAS) to another NAS. The NAS environments may be on-premises or in the cloud (Cloud NAS)
- S3 Data Migration – move objects from an object storage or cloud to another object storage or cloud
Data migrations can occur over a local network (LAN) or when going to the cloud over the internet (WAN). As a result, migrations can be impacted by network latencies and network outages.
Data migration software needs to address these issues to make data migrations efficient, reliable, and simple, especially when dealing with NAS and S3 data since these data sizes can be in petabytes and involve billions of files.
Elastic Data Migration
Elastic Data Migration makes its orders of magnitude faster than normal data migrations. It leverages parallelism at multiple levels to deliver 27 times faster performance than NFS alternatives and 25 times faster for SMB protocol performance.
- Parallelism of the Komprise scale-out architecture – Komprise distributes the data migration work across multiple Komprise Observer VMs so they run in parallel.
- Parallelism of sources – When migrating multiple shares, Komprise breaks them up across multiple Observers to leverage the inherent parallelism of the sources
- Parallelism of data set – Komprise optimizes for all the inherent parallelism available in the data set across multiple directories, folders, etc to speed up data migrations
- Big files vs small files – Komprise analyzes the data set before migrating it so it learns from the nature of the data – if the data set has a lot of small files, Komprise adjusts its migration approach to reduce the overhead of moving small files. This AI driven approach delivers greater speeds without human intervention.
- Protocol level optimizations – Komprise optimizes data at the protocol level (eg NFS, SMB) so the chattiness of the protocol can be minimized
All of these improvements deliver substantially higher performance than standard data migration. When an enterprise is looking to migrate large production data sets quickly, without errors, and without disruption to user productivity, Komprise Elastic Data Migration delivers a fast, reliable, and cost-efficient migration solution.
Komprise Elastic Data Migration Architecture
What Elastic Data Migration for NAS and Cloud provides
Komprise Elastic Data Migration provides high-performance data migration at scale, solving critical issues that IT professionals face with these migrations. Komprise makes it possible to easily run, monitor, and manage hundreds of migrations simultaneously. Unlike most other migration utilities, Komprise also provides analytics along with migration to provide insight into the data being migrated, which allows for better migration planning.
Fast, painless file and object migrations with parallelized, optimized data migration:
- Parallelism at every level:
- Leverages parallelism of storage, data hierarchy and files
- High performance multi-threading and automatic division of a migration task across machines
- Network efficient: Adjusts for high-latency networks by reducing round trips
- Protocol efficient: optimized NFS handling to eliminate unnecessary protocol chatter
- High Fidelity: Does MD5 checksums of each file to ensure full integrity of data transfer
- Intuitive Dashboards and API: Manage hundreds of migrations seamlessly with intuitive UI and API
- Greater speed and reliability
- Analytics with migration for data insights
- Ongoing value
