In 2022, Komprise unveiled a new program for customer onboarding called ACE (Assessment of Customer Environment) to analyze the expected performance of a customer migration or any movement of data–whether file to file migration, object to object migration, file to object migration, on premises or in the cloud. The ACE tool proactively identifies potential bottlenecks and other issues independent of Komprise Elastic Data Migration running in the customer’s environment and takes an hour or less of the customer’s time.
There can be several hurdles living within the customer’s data center that can delay or break a data migration process. The larger the data set, the more stress to the network and that’s when things go wrong which normally don’t surface under a normal transactional load. ACE helps customers understand their topology and their environment to ward off preventable issues that can disrupt migrations. It’s like an insurance policy for your data.
Common problems include:
Network bandwidth, topology and configurations: For instance, a customer inadvertently did not choose the fastest, dedicated route for the migration traffic. WAN-based migrations can also run much slower than the negotiated speed of the circuit based on distance or everyday business traffic.
Security systems and configurations: Firewall and other security tool settings can disrupt data traffic and may require whitelisting.
ACE complements the analysis that Komprise Elastic Data Migration has always delivered. That includes seeing how many small and large files are in storage. Since file size affects migration performance, you can get an early sense of whether the migration will go quickly or not.
Also, Komprise Analysis tells you if there are many directories containing just a few files, or if there are directories with more than 100,000 files. Those conditions will also slow down the migration. Komprise warns the customer of these things early before the migration begins. Even in cloud-to-cloud migrations, it’s important to plan ahead. If the migration takes place over the public internet, transfer times may be unpredictable and impose unplanned delays.
Real-world ACE in action
Here’s a recent example with one of our customers that was migrating 500 terabytes from the on-prem NAS in their corporate data center to a cloud-hosted solution. The customer was seeing transfer speeds of 3MB/s (24Mbps), which they expected to be closer to 63MB/s (500Mbps). Using ACE to identify potential bottlenecks, we first looked to see if the correct routes were used. Next we tested data movement in both directions by testing the topology such as firewalls, gateways and routers in the data path.
The issue wound up being related to firewalls. The company had two firewalls: one on prem and one in the cloud, but the migration team had forgotten about the latter. We used trace routes to identify the second firewall and then added exceptions for that one as we had done for the first firewall. Voila: the stuck data started moving and the transfer speed improved drastically.
In another case, a customer was experiencing delays despite having a a 10Gb WAN link. ACE discovered that the router was failing, because it only had a 1Gb line card. After the customer replaced the card with the proper size, the bottleneck went away.
Here’s how ACE works:
- ACE creates a configurable number of files of various sizes based on the specified number of threads. ACE uses these files to test source and target read, write, archive, recall, checksum and attribute copy operations, using any customer provided Linux or Windows Server.
- ACE starts with a round of pings and trace routes to determine basic connectivity and latency.
- Next, ACE tests how long it takes to read and write the various sized files on both source and destination along with checksums and attribute copy operations. Based on the number of threads and repetitions configured these tests can take as little as a few seconds.
The resulting dataset helps identify and isolate issues with the following:
- Firewalls, routers, anti-virus systems, security appliances, WAN accelerators;
- External connectivity from the customer environment to the target storage system (e.g., slow link, excessive hops, asymmetric routes);
- Source and target storage system bottlenecks.
ACE has proven its ability to prevent hours of tedious troubleshooting when things go awry during the data movement process. Walking through the various checkpoints is also valuable because it requires different functions within IT to communicate about data management—not a bad thing at all.
As with any major IT project, there is measurable value in mapping out the challenges and limitations up front, so your team knows what to expect later. Ultimately, using an assessment process that includes ACE means customers will complete unstructured data migrations faster and encounter fewer issues while reducing the cutover process.
Komprise customers can contact their account team for more information.
Learn more about the Komprise Elastic Data Migration technology, including performance benchmarks, in this paper.
