This blog was adapted from the original version on the AWS Storage Blog.
What you will learn: How a real enterprise cut file storage costs by 50% and gained faster access to petabytes of data by combining Amazon FSx for NetApp ONTAP with Komprise.
A global manufacturer fixes a massive file storage and data access problem
A large manufacturing company with more than 100 sites faced an overwhelming data challenge as it undertook an Industry 4.0 transformation to create smart factories. The initiative connects their production systems, automates operations, and implements new data analytics for real-time decision making. Yet there were barriers with unstructured data management:
- Their production systems were generating more than 1 PB of new data every year and most of it was sitting on expensive on-premises storage. File system storage and access costs were hitting $1 per GB annually.
- Engineers were dealing with slow data access, growing bottlenecks and rising costs.
- Stale production data that had not been touched in years took up space meant for active files and the company had no easy way to separate the two.
- To compound matters, unmanaged “shadow storage” solutions at manufacturing sites made it impossible to track total storage costs, understand data usage patterns, or enforce enterprise-wide data retention and protection policies.
- Manufacturing teams wanted to use enterprise-wide data analytics and machine learning, but their fragmented storage infrastructure thwarted those plans. Without a unified, performant storage solution, the company’s smart factory initiatives were stalled.
The IT team decided to move 3PB of data into Amazon FSx for NetApp ONTAP with Komprise. The migration happened with no downtime and the impact was immediate. They reduced storage costs by more than 50% and consolidated more than 2PB of unmanaged shop floor data into a central system for easier access and insights.
Why this approach works
Today’s enterprise storage environments often treat all data the same. Files that are active and frequently accessed sit beside files that have not been opened in months or years. When everything lives on the same high-performance storage, costs rise quickly and the system gets slower as it grows.
Amazon FSx for ONTAP delivers high performance file storage that can scale to petabytes in the cloud while still supporting familiar NAS protocols like NFS and SMB. Komprise adds intelligence on top of that foundation. It analyzes how your data is used and identifies what is active and what is not. Then it moves cold data to lower cost capacity pools or to Amazon S3, while keeping the files fully accessible. Users and applications still see the same paths and can open files at any time.
The combination means you can keep active workloads fast while placing everything else on more cost-effective tiers. You also gain visibility into storage consumption across locations and departments, which helps with planning and governance.
Organizations can use the global infrastructure of Amazon Web Services (AWS), the high-performance hybrid-cloud file system of NetApp ONTAP, and the data movement specialization of Komprise to reduce storage costs by over 50%, eliminate manual data management tasks, prevent production delays due to slow file access, and gain insights from their data through integration with analytics and AI systems.
Solution Architecture
- Cloud-hosted managed file system: FSx for ONTAP is a high-performance, AWS-hosted file system. It provides automatic storage tiering, multiprotocol (NFS, SMB) file access, data de-duplication, snapshots and replication.
- Edge cache volumes: The FSx for ONTAP FlexCache volumes are optionally deployed at locations that need low-latency access to their frequently used files. They provide writable, persistent copies of requested data from the main file system.
- Data lifecycle: Komprise Intelligent Data Management indexes unstructured data across storage for full visibility into data growth and its usage. It identifies cold data, moves it to cheaper storage such as Amazon S3 or S3 Glacier and delivers transparent access after tiering.
- Archival storage: Archived data is stored in cost-effective S3 Glacier object storage with its metadata preserved by Komprise. Different S3 Glacier tiers are available for an optimal storage cost and retrieval experience.
- Dedicated connectivity: AWS Direct Connect provides dedicated network connections from user locations to AWS, providing consistent, low-latency access to the FSx for ONTAP file system. This is necessary for FlexCache performance and improves the performance of file activities against the cloud-hosted file system.
Customer Outcomes
- Komprise migrated 3PB of data from on-premises NAS into FSx for ONTAP at a 25x faster rate than popular migration tools and with chain-of custody reporting for the manufacturer’s regulated business segments.
- Nearly 70% of old data was tiered by Komprise into Amazon S3 Glacier storage. The remaining 30% stayed in FSx for ONTAP and 60% of that was moved to the capacity pool tier within 30 days, resulting in the optimization of more than 90% of the initial data.
- The manufacturing customer achieved more than 50% in storage cost reductions, consolidated 2 PB of shop-floor data, and reduced file retrieval time for production issues from days to minutes. It also added production data insights for enhanced anomaly detection, historical trend analysis, and quality control.
- The business used the improved data and usage visibility offered by Komprise and FSx for ONTAP to track company-wide data growth for proactive capacity planning.
Three Takeaways for IT Leaders
1. Cost control starts with data visibility
Most enterprises underestimate how much cold data sits on expensive storage. Gaining a clear picture of usage and age can unlock immediate savings without disrupting users.
2. Tiering works when it’s automated and transparent
Moving inactive data to lower-cost tiers delivers maximum value if applications and users still see files in the same place. Look for solutions that offer flexible, automated tiering policies so you can set and forget. Read more about cloud tiering.
3. A scalable storage foundation accelerates AI and analytics
By placing active data on high-performance storage and archiving the rest intelligently, IT teams free up budget and performance headroom for future AI, analytics and digital initiatives.
