How to Accelerate NAS and Cloud
Data Migrations

Using Komprise Elastic Data Migration

Businesses today are looking at modernizing storage and moving to a multi-cloud strategy. As they evolve to faster, flash-based Network Attached Storage (NAS) and the cloud, migrating data into these environments can be challenging. The goal is to migrate large production data sets quickly, without errors, and without disruption to user productivity. Komprise Elastic Data Migration delivers a fast, reliable, and cost-efficient data migration solution.

Why data migrations are hard

Migrating unstructured file data from one NAS to another and to the cloud is no simple feat. It involves file data that consists of the file content and metadata, including permissions and attributes that aren’t always represented identically by all the file servers, which creates a complex process. And since the amount of data to be migrated can be huge, it can mean long migration windows and significant effort. For companies that require minimal downtime, the disruption of long windows poses a major problem making migration speed critical for success.

Migrating files fast is easy when the migrated files are large, because the migration overhead is small compared to the time required to transfer the file over the wire. It’s a different story when the files are small as the migration overhead can be a multiple of the transfer time.

To address migration to a cloud NAS or to a NAS in another datacenter, modern migration solutions also need to support migrations over WAN connections with acceptable performance. Migrations over a WAN will be much slower than over a LAN due to the higher latency. For these cases, it’s important to be able to reduce the back- and-forth protocol “chatter” and automatically partition the task into multiple smaller ones that run in parallel to overcome the performance degradation from the WAN.

A strong migration product not only needs to work across different vendors’ NAS solutions quickly, but it also needs an intuitive UI that lets you run, monitor, and manage hundreds of migrations simultaneously. And, of course, it has to be fast to reduce the downtime when IT cuts over from old file servers to the new ones.

Komprise Elastic Data Migration: greater speed and reliability

To address the critical migration issues IT faces today, Komprise has developed Elastic Data Migration. This super-fast migration solution is a highly parallelized, multi-processing, multi-threaded approach that works at two levels:

  • Multi-level Parallelism:  Maximizes the use of available resources by exploiting parallelism at multiple levels: shares and volumes, directories, files, and threads to maximize performance. Komprise Elastic Data Migration breaks up each migration task into smaller ones that execute across the Komprise Observers. Komprise Observers are a grid of one or more virtual appliances that run the Komprise Intelligent Data Management solution. All of this parallelism occurs automatically across the grid of Observers. The user simply creates a migration task and can configure the level of parallelism. Komprise does the rest.
  • Protocol-level Optimizations: Reduces the number of round-trips over the protocol during a migration to eliminate unnecessary chatter. Rather than relying on generic NFS clients provided by the underlying operating system, Komprise has fine-tuned the NFS client to minimize overhead and unnecessary back-and-forth messaging. This is especially beneficial when moving data over high-latency networks such as WANs.

Analytics with migration for data insights

Unlike most migration solutions, Komprise Elastic Data Migration provides data analytics. This insight into your data makes it possible to properly plan and manage your migrations. The analytics show how much data you have on each volume (share), the age of that data (whether by last modified time or last accessed time), and for each volume charts show what types of files are present, a histogram of file sizes, and space consumed by files of different sizes.

Insights from Data Analytics

  • How much data is on each share
  • The age of the data
  • The types of files
  • How much space it takes
  • And more

Ongoing value

Komprise eliminates the sunk costs of traditional migration approaches. After the migration, you can use the full Komprise data management platform to maximize your use of the new NAS file servers. Komprise analyzes data growth and usage across your storage to find cold, inactive data, and projects the ROI of moving cold data to secondary storage such as cloud/object. Komprise also moves cold data transparently based on customer-defined policies, so users continue to access the moved data the same as before. Komprise helps organizations reduce over 70% of storage costs while managing data growth.

Figure 1 shows the architecture of a NAS migration using Komprise. The easy-to-deploy, fault-tolerant, distributed
architecture consists of one or more Komprise Observer virtual appliances running at the customer site. The virtual appliances are connected to a Komprise Director, which provides the administrative console UI and can run either as a cloud service or on-premises. Komprise works across on-premises NFS and SMB/CIFS storage and object/cloud storage seamlessly.

cloud data migration

Figure 1. Komprise Elastic Data Migration Architecture

When migrating data, Komprise performs automatic retries if the network or storage is unavailable. It ensures that all permissions and file data are fully migrated and verifies it using MD5 checksums. Performance can be enhanced as needed by increasing the number of threads in an Observer or by adding more Observer virtual machines.

Migration console

Figure 2 is a sample screenshot of the UI of the migration tab. This figure shows a summary of all the ongoing migration tasks and their status. It also shows details of one particular migration task. Note that for this task, it shows details of the multiple migration iterations and details of any errors or issues found during these iterations. 

A CSV export providing details of a single migration task or all the tasks can be downloaded. This UI makes it easy to manage and report on hundreds of simultaneous migrations.

All migration functions, including creating a migration task, obtaining its status and error reports, and stopping or pausing it, can be done via an API. This makes it possible to integrate Komprise Elastic Data migration into an existing management platform.

cloud data migration console

Figure 2: A sample screenshot showing Komprise Migration tab UI

Summary

Komprise Elastic Data Migration provides high-performance data migration at scale, solving critical issues that IT professionals face with these migrations. Komprise makes it possible to easily run, monitor, and manage hundreds of migrations simultaneously. Unlike most other migration utilities, Komprise also provides analytics along with migration to provide insight into the data being migrated, which allows for better migration planning.

While Elastic Data Migration is available as a superior, stand-alone migration solution, it is also a part of the full Komprise Intelligent Data Management platform. With this platform, the sunk cost of migration is eliminated. It can be used to identify and transparently archive cold data with no change in how users and applications access the cold data. 

This dramatically reduces storage costs without disrupting users or existing data protection workflows. Komprise Intelligent Data Management also provides Deep Analytics, which allows you to create a metadata lake of all your files, enabling company-wide search and tagging of your data.

Learn more about how Komprise can help you cut costs, free up primary storage capacity, and strengthen data protection. Call 1.888.995.0290 or start your free 14-day trial

Komprise - 1901 S. Bascom Ave, Tower 1, 5th Floor, Campbell, California 95008. Tel: 1.888.995.0290