In an Increasingly Data Driven World, Your Cold Data is Here to Stay

 cold data ai

After attending the recent IBM TechU event in Orlando, one of the themes I came away with was a renewed importance on the need to intelligently manage large amounts of cold data driven by trends in artificial intelligence (AI).

With more and more organizations looking to gain a competitive advantage, old data that was once thought to be of little or no value has become important to keep around.  Companies are looking to their data to power AI insights and give them an edge against rivals and smaller startups.  This has increased the demand for cost-effective and capacity-optimized storage to be stronger than ever—with no letting up in sight.

Organizations are now looking to object storage solutions such as IBM Cloud Object Storage, whether on-premises or in the cloud, as a way to address this challenge.  And interestingly, a technology that was written off by many, tape, is also making its own resurgence.  Both are addressing the major business challenge of how to store cold/inactive data without breaking the IT budget, but they also introduce new challenges.  Object and File are inherently different protocols with different formats.

How can organizations take advantage of lower cost object storage for file data without changing the way the data is accessed?  Companies want to identify what data is cold and move/archive that data in a way that doesn’t disrupt existing users, applications, and workflows.

This is where an unstructured data management solution like Komprise delivers a tremendous amount of value.  Komprise can analyze multiple petabytes of source data (SMB, NFS, or Lustre protocols are supported today) and automatically classify which files are cold based on access patterns (much deeper reporting and analytics are also provided).  Users can then decide what they want to do with this cold data, and Komprise automatically archives the data to object storage targets, whether on-premises or in public or private clouds.

Komprise also works with storage offerings that can tier data off to tape, like IBM Spectrum Scale or Spectra Logic BlackPearl. Additional features allow Komprise to bring back any data that starts to become active again or allow for manual recalls of certain data sets or directories.

Komprise does all this without disrupting existing user access, standardized workflows, or requiring applications to be re-written to interface with object storage.  Instead of relying on a proprietary stubbing technology, Komprise leverages symbolic links which are native to the SMB and NFS protocols and is able to dynamically convert file data to an object format and back again.

Whether it’s a research university managing large-scale genomics data, a broadcasting company managing petabytes of media files, or a healthcare organization who needs to retain radiology imaging for a lifetime, vast amounts of “low velocity” data need to be stored in a cost-efficient manner. This data can be used to find the genetic markers that indicate risk for a certain disease, understand the composition of movie trailers that get the most interest, or help radiologists identify which images may have an anomaly. But keeping these massive amounts of cold data on expensive primary storage systems just doesn’t make financial sense.

To learn more about the ROI of Intelligent Data Management, check out our Quantifying the Value of Intelligent Data Management Report.