Best Practices for Data Management and Storage in 2023

With each new year comes new threats, new technologies and changing business mandates. For managers and directors overseeing data storage and unstructured data management, there is always much to consider – from availability and security, cloud cost optimization, performance, and cloud migration strategies. These days, IT leaders must also factor in supporting AI initiatives, a top priority and challenge identified in our latest industry survey.

In our Fall news blog, we review 2023 unstructured data management best practices to address these objectives along with recent news from Komprise.

Storage Security

“Data storage security is a 360-degree discipline that covers a lot of things, including protecting IT assets, setting appropriate security policies, vetting vendors and partners, and educating employees in secure practices,” writes Mary Shacklett in Enterprise Storage Forum. The tips include: enact company-wide data storage security policies; role-based access controls; data encryption and data loss prevention, maintain strong network security and endpoint security systems, copy data to redundant storage, and more.

Storage Refresh

This article in eWeek by Komprise COO Krishna Subramanian provides guidance on this evergreen topic by highlighting what not to do when performing a data storage refresh. “When IT managers discover that they need more storage, it’s easy to simply buy more than they need,” Subramanian writes. She advises getting metrics to understand data volumes, data growth rates, storage costs and how quickly data ages and becomes suitable for archives or a data lake. “These basic metrics can help guide more accurate decisions, especially when combined with a FinOps tool for cost modeling different options.” She also suggests matching distinct workload needs to vendor offerings: “You can likely build more cost-effective storage infrastructure if you select from the offerings of multiple vendors.”

Data Backups

Stephen Bigelow’s comprehensive guide for TechTarget on data backups delivers solid advice: “Because of ransomware, data centers must increase the frequency of backups — once a night is no longer enough.” Among other tips, Bigelow addresses cloud backups: “Although the cloud backup provides an attractive upfront price point, long-term cloud costs can add up. Repeatedly paying for the same 100 TBs of data eventually becomes more expensive than owning 100 TB of storage.” At Komprise, we advise an intelligent tiering strategy to our customers. This entails analyzing data to understand data owners and usage patterns: delete zombie and orphan data and then tier cold data (often 80% of all data sitting on top-tier storage) to low-cost object storage where backups are not usually needed. This can save tremendously on backups, which typically comprise around 75% of the overall data storage bill.

Data Governance

Data governance is about setting and enforcing internal standards related to methods for gathering, storing, processing and deleting data when it reaches the end of its lifecycle, writes Drew Robb in ESF. Object storage is important, he adds, because it simplifies governance by providing a centralized and unified solution for data management activities and includes strong metadata capabilities. Understanding unstructured data: its volume, location, owners, growth rates, costs to manage and usage patterns is also key, Krishna Subramanian said in a quote. Look at what data can be deleted, tiered, archived and governed more and add automation to manage tasks efficiently and deliver an audit trail of what corporate data has been ingested by applications, she adds.

Unstructured Data Management

Optimizing unstructured data is a growing priority for enterprise IT organizations that need to both be cost efficient and deliver data to AI for new value and productivity benefits. Getting visibility into unstructured data across silos is the first step, per this TechBeacon article by our COO, Krishna. This is required for IT to make informed decisions about how best to manage it. Treat cloud data migration as an ongoing process and support it with policy-based automation wherever possible. Look for opportunities to deliver new value from data by indexing it so that data becomes easier to find, search, and use. Read the article for more tactics!

What’s New at Komprise

September was a busy month at Komprise, as we unveiled new research and a significant product update, after a long, hot summer.

Komprise 2023 State of Unstructured Data Management

Our third-annual survey covered the latest trends and priorities for data management and storage, across IT leaders in the US and UK. You can download the full report here.

Top findings include:

• Preparing for AI is the leading data storage priority in 2023, followed by cloud cost optimization;
• Most organizations (90%) allow employee use of generative AI yet 66% of organizations cited top data governance concerns of preventing security and privacy violations, lack of data source transparency leading to unethical, biased or inaccurate outputs and corporate data leakage into the vendor’s AI model;
• The majority (40%) will pursue a multi-pronged approach to manage AI risk, encompassing storage, data management and security tools;
• Organizations managing more than 10PB of data grew from 27% to 32% this year, a 19% increase.

James Maguire, Editor in Chief of eWeek, interviewed our COO Krishna Subramanian on the survey findings and what these trends signify for enterprise IT.

Product News

Komprise Intelligent Data Management 5.0 was released in September and the highlight is the new Storage Insights console which combines data and storage metrics in one customizable view. “As unstructured data becomes a central concern for IT administrators, the market needs tools to manage the sprawl while also addressing top-level governance and data protection directives,” said Steve McDowell, industry analyst with NAND Research, in an article for Forbes.

“This is where Komprise is strong. This company continues to evolve, keeping a step ahead of the needs of the market it’s serving. This is what makes it a leader.”

Watch the demo:

New White Paper: Unstructured Data Management In the Age of Generative AI

This paper delivers guidance for using enterprise data in AI platforms, considering needs for security, privacy, lineage, ownership and governance. Download it here to learn how to protect, segment, track and audit data in generative AI and the role of unstructured data management.

For more unstructured data management best practices, subscribe to the blog!