Unstructured Data Management Challenges

Komprise_Architecture_WhitePaper_PaperThumbnail_darkglowIn our Komprise Intelligent Data Management Architecture Overview we review the principles of Komprise technology, the 6 components of the Komprise architecture, and how it works. In this post we’ll cover some of the common unstructured data management challenges we talk to our customers and partners about every day and the benefits of analytics-driven data management-as-a-service from Komprise.

Massive Growth Within Flat Budgets

Data is growing fast—nearly 90% of the world’s data was created in the last two years and enterprise data is doubling every two years. The challenge is how to retain all this data (as much of it is valuable) while keeping within flat budgets. Enterprise IT organizations need to do more with less by using the right mix of cloud and on-premises file and object storage options.

Right Data, Right Place, Right Time

Over 60 to 90% of enterprise data is cold and infrequently accessed within months of creation, but is often stored, backed-up, replicated, and managed in the same way as active data. This is because there have been no easy approaches to systematically identify, move, and access data without tedious, manual processes and disrupting users. Cold data storage costs are even higher with the cloud given the ongoing OpEx costs of storing and retrieving data from the wrong cloud tier. A systematic way to continuously understand data usage and dynamically move the right data to the right place at the right time, without impeding user access, is required.

Legacy Data Management Challenges

Legacy data management solutions are too costly and complex, and cloud tiering solutions provided by storage vendors are too limited, proprietary, and create unexpected cost overruns.

  • Costly – Require expensive enterprise licenses and upfront infrastructure investments. •
  • Complex – Multiple moving parts such as storage agents, hardware, software, and databases to manage.
  • Brittle – Static stubs can be corrupted or orphaned, agents need to be kept up to date as the storage evolves, and detailed rules need to be specified and managed.
  • Disruptive – Performance slowdowns due to the management overhead they generate and user disruption by not maintaining transparent access to moved data.

Cloud Tiering and Storage Pool/Tiering Challenges

Cloud vendor tiering solutions don’t provide enough flexibility to meet your unique needs and only work on a subset of your enterprise storage. Managing multiple cloud data archive and tiering solutions creates unnecessary costs and complexity.

  • Limited – Intelligent tiering and cloud tiering solutions are usually limited to a few simplistic policies (e.g. any data not used in 60 days) and limited to a few storage choices. Enterprises need a robust data management solution with flexible policies to address different groups and their unique needs.
  • Proprietary – Tiering solutions move data at the block level, so the data cannot be directly used from a lower-cost tier such as the cloud. The cloud is used as a cheap under-the-hood storage tier, meaning that cold data cannot be directly accessed in the cloud nor can it be used with native tools in the cloud or 3rd party applications for AI, ML, or compliance use cases.
  • Cost Overruns – Cloud tiering and storage tiering or “Storage Pool” solutions have unexpected costs since they are proprietary and access typically causes rehydration of the cold data and results in expensive egress costs from the cloud.
  • Difficult to Manage – Although storage vendor-based tiering may sound like a simple choice, they are not suited for tiering to the cloud and will lead to performance degradations of the file system. These solutions are limited in the storage they support, create proprietary silos, do not provide visibility into the data that is being tiered, and they lock you in. Switching to another vendor in the future is not simple.

Review your cloud tiering choices.

Why Komprise?

komprise_laptopKomprise puts you in control of your unstructured data—not your storage, cloud, or backup vendor, so you can know, move, manage, and harness data anywhere. With analytics-driven data management-as-a-service from Komprise you get:

  • Data Insight: Know before you buy more storage, backup, or cloud infrastructure.
  • Data Mobility: Migrate, archive, tier, and move the right data to the right place at the right time.
  • Open Standards: Work across any storage using standard protocols and no proprietary interfaces.
  • Cloud Native Access: Use data as files or objects anywhere without lock-in, without expensive rehydration.

In the next post, we’ll review the principles of Komprise technology. Be sure to download the Komprise Intelligent Data Management architecture overview white paper to learn more.


Getting Started with Komprise:

Contact | Data Assessment