Unstructured Data Migration

Data-Migration-Icon-603x603What is Unstructured Data Migration?

Unstructured Data Migration is the process of selecting and moving data from one location to another – this may involve moving data across different storage vendors, and across different formats.

Data migrations are often done in the context of retiring a system and moving to a new system, or in the context of a cloud migration, or in the context of a modernization or upgrade strategy.

When it comes to unstructured data migrations and migrating enterprise file data workloads to the cloud, data migrations can be laborious, error prone, manual, and time consuming. Migrating data may involve finding and moving billions of files (large and small), which can succumb to storage and network slowdowns or outages. Also, different file systems do not often preserve metadata in exactly the same way, so migrating data without loss of fidelity and integrity can be a challenge.

NAS Data Migration

Network Attached Storage (NAS) migration is the process of migrating from one NAS storage environment to another. This may involve migrations within a vendor’s ecosystem such as NetApp data migration to NetApp or across vendors such as NetApp data migration to Isilon or EMC to NetApp or EMC to Pure FlashBlade. A high-fidelity NAS migration solution should preserve not only the file itself but all of its associated metadata and access controls.

Network Attached Storage (NAS) to Cloud data migration is the process of moving data from an on-premises data center to a cloud. It requires data to be moved from a file format (NFS or SMB) to an Object/Cloud format such as S3. A high-fidelity NAS-to-Cloud migration solution preserves all the file metadata including access control and privileges in the cloud. This enables data to be used either as objects or as files in the cloud.

Storage migration is a general-purpose term that applies to moving data across storage arrays.

Unstructured Data Migration Phases

Data migrations typically involve four phases:

  • Planning – Deciding what data should be migrated. Planning may often involve analyzing various sources to find the right data sets. For example, several customers today are interested in upgrading some data to Flash – finding hot, active data to migrate to Flash can be a useful planning exercise.
  • Initial Migration – Do a first migration of all the data. This should involve migrating the files, the directories and the shares.
  • Iterative Migrations – Look for any changes that may have occurred during the initial migration and copy those over.
  • Final Cutoff – A final cutoff involves deleting data at the original storage and managing the mounts, etc., so data can be accessed from the new location going forward.

Resilient data migration refers to an approach that automatically adjusts for failures and slowdowns and retries as needed. It also checks the integrity of the data at the destination to ensure full fidelity.

Types of Unstructured Data Migrations

When it comes to file data, there are NAS Migrations and Cloud Migrations. There are also NAS migrations to the cloud. Data migrations are often seen as a dreaded and laborious part of the storage management lifecycle. Free tools are often considered first but they can introduce risk, time and cost overruns and they are typically labor intensive and error-prone. On the other hand, traditional migration tools have complex legacy architectures and are expensive point products that do not provide ongoing value – resulting in sunk costs.

Look for easy-to-use, fast, reliable data migration tools are not one-and-done point tools. The right data migration solution should be able to handle other unstructured data management use cases, including cloud data tiering and data replication.

How to Plan a Smart NAS or Cloud Unstructured Data Migration?

The typical steps for any unstructured data migration project are:

Analytics: Before you start an unstructured data migration project, it’s important to have visibility into:

  •  How fast is your data growing?
  •  How much data is hot vs. cold data?
  •  Who is using your data?

Savings: Estimate how much you’ll save by moving to the new NAS or cloud infrastructure. This information will guide which NAS or cloud storage mix is best for your data.

Offload heavy lifting: Your data migration solution should be able to manage multiple iterations of the migration and handle problems by automatically retrying in a
slowdown or a network or storage failure.

Preserve data integrity: Your data migration solution should provide MD5 checksum on every file and assure all metadata and access controls migrate to the new environment.

Avoid sunk costs: File data migrations are a lot of heaving lifting. Your data migration solution should include automatic parallelization at every level for elastic
scaling and the ability to migrate petabytes of data seamlessly and reliably.

Reduce downtime: It is recommended that your data migration solution runs multiple iterations for more efficient cutovers.

Planning Your Cloud File Migration

Komprise and Unstructured Data Migration

Komprise Elastic Data Migration is included in the Komprise Intelligent Data Management platform or is available standalone. Designed for cloud migrations and NAS migrations, with Komprise Elastic Data Migration you can run, monitor, and manage hundreds of data migrations faster than ever at a fraction of the cost. Learn more about Komprise Smart Data Migrations.


Unstructured Data Migration and the Cloud

As unstructured data continues to grow exponentially, organizations struggle to control costs for file data storage. Many are turning to the cloud to scale and manage spend. However, choosing the right files to move can be challenging as there can easily be billions of files. Many enterprises have over 1 PB of data, which represents roughly 3 billion files. This unstructured data is growing exponentially and resides in multi-vendor storage silos for access by various applications and departments.
For these reasons, organizations often lack visibility into file data and are making decisions in the dark. To be agile and competitive, IT teams must evolve storage management to become a holistic data management strategy. The right approach to data migration and the cloud for file and object data is to use analytics in cloud data management:

  1. Understand your data patterns
  2. Plan using a cost model
  3. Use data to drive stakeholder buy-in
  4. Eliminate user disruption
  5. Create a systematic plan for ongoing data management

Read the eBook: 5 Ways to Use Analytics for Cloud Data Migrations

Top Unstructured Data Migration Challenges

Businesses today are looking at modernizing storage and moving to a multi-cloud strategy. As they evolve to faster, flash-based Network Attached Storage (NAS) and the cloud, migrating data into these environments can be challenging. The goal is to migrate large production data sets quickly, without errors, and without disruption to user productivity.

The top cloud data migration challenges are:

  1. How do you manage cloud data migrations without downtime?
  2. How can you automate cloud data migrations to eliminate manual effort?
  3. How can you ensure all the permissions, ACLs, metadata are copied correctly during a cloud data migration so you can access the data in the cloud as files?

You can overcome these challenges with some planning and automation that preserves file-based access both from on-premises and the cloud.

Unstructured Data Migration Tools

  • Free Tools: Require a lot of babysitting and are not reliable for migrating large volumes of data.
  • Point Data Migration Solutions: Have complex legacy architectures and create sunk costs.
  • Komprise Elastic Data Migration: Makes cloud data migrations simple, fast, reliable and eliminates sunk costs since you continue to use Komprise after the migration. Komprise is the only solution that gives you the option to cut 70%+ cloud storage costs by placing cold data in Object classes while maintaining file metadata so it can be promoted in the cloud as files when needed. Learn more >

Learn more about Smart Data Migrations for unstructured file and object data:

Part 1

Part 2


What is Data Migration?

Data Migration is the process of selecting and moving data from one location to another and can involve moving data across different storage vendors, and across different formats.

How is data migration done?

Data migrations are often done in the context of retiring a system and moving to a new system, or in the context of a cloud migration, or in the context of a modernization or upgrade strategy.

What tools to use for data migration?

There are a variety of free tools but these require the most babysitting. Point Data Migration solutions have complex legacy architectures and can create sunk costs. Komprise Elastic Data Migration makes cloud data migrations simple, fast and reliable and eliminates sunk costs.

Want To Learn More?

Related Terms

Getting Started with Komprise:

Contact | Data Assessment