The Rise of Data Services for Unstructured Data Management

This blog was adapted from the original article on ITProToday.

IT infrastructure leaders are now responsible for not only managing data, but also delivering data services. These data services include protecting data, managing compliance requirements, archiving data, managing data lifecycle and costs, and yes even deleting data when it’s no longer needed.

Data services responsibility also means making the right data easily available to end users and tools (such as cloud AI) and with cost optimization in mind.

Here are a few proof points to consider:

  • Gartner predicts that by 2026, large enterprises will triple their unstructured data capacity across their on-premises, edge, and public cloud locations, compared to 2023.
  • Hybrid cloud and edge computing are now the predominant models for IT workloads, pushing data storage outside of the corporate data center.
  • Shadow IT brought on by the cloud has complicated matters: How much data does the organization own and where does it reside?
  • Finally, IT executives are seeing the imperative of efficiently curating the right data sets across their petabytes of increasingly hybrid, multicloud storage to feed new AI tools.

Defining Data Services in Data Management and Storage

Data services is a broad term that describes a range of activities typically provided by enterprise IT, such as: data processing, data integration, data security, data reduction, data protection, data storage, and unstructured data management.

As relates to data storage and unstructured data management, data services involve the management of data throughout its lifecycle. Beyond primary storage, it covers analysis and reporting on data storage growth and costs including departmental showback, data usage, self-service file search and tagging, along with data mobility use cases such as data migration, data tiering, replication, and deletion. This new approach requires the ability to understand data usage and manage data independently of storage.

Benefits of Data Services in Unstructured Data Management

A data services approach helps IT and business teams in many ways, enabling:

  • Holistic visibility and granular search across multiple storage systems and clouds;
  • Analytics and insights on data types and usage for more accurate storage decisions;
  • Automated, policy-driven actions based on that analysis;
  • Reduced security and compliance risks;
  • Full use of data wherever it is stored, especially in the cloud;
  • User self-service access to support departmental and research needs for data storage, management, and AI workflows;
  • Greater flexibility to adopt new storage, backup, and DR technologies because data is managed independently of any vendor technology.

Data Services in Action

To better understand the potential of data services, here are examples across higher education, biotechnology, energy, media & entertainment, and retail.

Midsize university: A storage administrator can look across all the shares in the university to search for anomalies that pose risk: files belonging to people no longer with the university, sensitive files that aren’t being stored in the right location, and old video files from the website team that are taking up a lot of capacity and are no longer needed.

Medical device maker: A company with regulated products must regularly answer questions from auditors about its data: What is it, who owns it, and how is it used? Ensuring that data is being stored and protected according to various regulations such as HIPAA is imperative to avoid large fines and penalties. The company is also working to bring in “shadow” data from remote sites so that IT can ensure permissions and other protections are up to date. Using a solution that indexes all unstructured data is the key to accomplishing these critical compliance tasks. Read more about healthcare and unstructured data management.

Life sciences: The central IT team at a global pharmaceutical firm has been tiering cold data to the cloud but now wants to give its research teams the ability to identify and tag their project files for later use. This approach flips the dynamic from IT having to police data storage to providing an analytics-based data service to the business. Read more about life sciences and unstructured data management.

Energy: A global oil and gas services provider with data centers around the world is modernizing and optimizing its infrastructure. The company uses the cloud to deliver digital services to its customers such as managing data generated from equipment at the bottom of the ocean. By adopting a data services approach and toolset, the energy company tiered 85% of its data to far cheaper, archival storage in the cloud. A centralized data services strategy and tools help them easily move data from one platform to another, rather than using various point migration tools. Data services is now central to how IT operates — for cost savings, risk management and, flexibility. Read an energy sector case study.

Retail: A national retail conglomerate went through a divestiture, shuttering a few of its brands and standardizing its IT infrastructure. The process resulted in a large quantity of zombie files from technologies no longer in use, such as Microsoft .PST files. A data services toolset allowed the organization to quickly find the unwanted files across all storage and delete them. With a simpler, more streamlined data environment, the company is in a great position to grow in its next stage of evolution.

Entertainment: The storage director at a large Hollywood studio was investigating why certain shares were taking an excessively long time to back up. By running analysis on the data, the director discovered many old files belong to a handful of users. Once the users were made aware of the situation, they deleted the files, which improved backup cycles. The IT organization plans to give departmental data managers access to the data management solution. That way, they can view their own data and tag groups of folders or shares for data management actions such as data tiering to cold data storage or deletion altogether. This puts data owners in control of their data, while also helping IT meet its objectives.

A Roadmap to Data Services

Komprise-Analysis-blog-SOCIAL-1-1There’s no right way to transition to data services, but analysis is at the heart of the matter.

  • Using data analytics and data management to understand data usage, data growth, and data storage costs across storage and cloud environments is a good start;
  • Other core requirements include allowing data teams to search and tag data based on share-based access permissions. These tags can then inform central IT to execute automated policies, such as deleting project files that are more than three years old;
  • Above all, data management and storage infrastructure experts will need to shift their thinking and practices from managing storage technologies to understanding and managing data for a variety of purposes.

A data storage and data management infrastructure that supports flexibility and agility to shift with organizational data needs will allow IT to make the shift faster and with better outcomes for all.

Getting Started with Komprise:

Contact | Data Assessment