Komprise Technical Overview

Your organization’s data is valuable. But the value of data changes as it ages. What is critical, active data today may or may not be critical down the road.

At the same time, unstructured data is growing exponentially and analysts project data to reach 163 Zettabytes by 2025.

Existing unstructured data management approaches were not built to handle this monumental explosion within extremely tight budgets. The time to start planning for an effective Intelligent Data Management solution is now.

Manage Data With Insight

Komprise is an intelligent, analytics-driven data management solution that analyzes data growth and usage across an organization’s current storage to project the ROI of moving inactive / cold data to secondary storage such as cloud data storage / object storage. Komprise then moves data based on customer defined policies transparently so users continue to access the moved data as before.

Komprise helps organizations reduce over 70% of data storage, backup and cloud costs while managing data growth and delivering greater value with Deep Analytics and a Global File Index.

Simple To Operate & Scales On-demand

Komprise uses an easy-to-deploy, easy-to-scale distributed architecture that consists of one or more Komprise Observer virtual machines running at the customer site connected to a Komprise Director that can run either as a cloud service or on premises. Komprise works across any on-premises NFS, SMB/CIFS, and object/cloud storage seamlessly. Learn More about the Komprise Elastic Grid.

Komprise partners include IBM, NetApp, Dell/EMC, Amazon, Microsoft, Google, Wasabi, Cloudian, Qumolo, Caringo, Spectra Logic, Scality, and others as the data-centric, storage-agnostic data management solution.

We see immense benefits of using Komprise as it provides a ubiquitous layer across storage to analyze, manage and migrate data.
Jay Smestad
Senior Director Infrastructure Architecture
Pacific Biosciences

Massive Data Growth Within Flat Budgets

Data is growing fast – nearly 90% of the world’s data was created in the last two years and enterprise data is doubling every two years. The challenge is how to retain all this data (as much of it is valuable) while keeping within flat budgets. Businesses need to do more with less and are looking to affordable scale-out storage options, such as cloud and object storage.

Managing All Data As Mission Critical

Over 60 to 90% of the data is infrequently accessed within months of creation, but is often stored and managed in the same way as active data. This is because there have been no easy approaches to identifying inactive data and moving it without disrupting users.

Legacy Solutions Too Costly, Complex & Disruptive

Legacy solutions are:

• Costly – requiring expensive enterprise licenses and upfront infrastructure investments
• Complex – with multiple moving parts such as storage agents, hardware, software, and databases to manage
• Brittle – with static stubs that can be corrupted or orphaned, agents that need to be kept up-to- date as the storage evolves and detailed rules that need to be specified and managed
• Disruptive – creating performance slowdown due to the management overhead they generate, and user disruption by not maintaining transparent access to moved data

Data management has been hard to adopt and scale to meet today’s massive data growth. Why Komprise? Smarter, Faster Unstructured Data Management.

Komprise addresses this need with a modern architecture that has been built from the ground-up to handle today’s massive scale of data with intelligent automation.

Komprise stands apart from other solutions because it gives visibility across our storage and manages data at scale at a fraction of the cost
Carl Siva
Senior IT Group Director,
Architecture and R&D Solutions
Cadence Design Systems

Komprise Intelligent Data Management is built upon the below design principles:

• Simple: Komprise is simple to deploy and operate — it requires no proprietary interfaces or complex infrastructure setup.
• Open: Komprise works using open standards — NFS, SMB/CIFS and REST/S3 — without the use of proprietary stub files of agents.
• Vendor Agnostic: Komprise is built on open standards and works with any storage supporting those standards — allowing you to keep your preferred vendors.
• Analytics Driven: Komprise uses the analytics it provides into data usage and growth to provide an ROI driven approach and optimize how it manages data based on your unique data needs.
• Transparent: With our patented Transparent Move Technology (TMT), data migrated, tiered, replicated and moved in any way by Komprise is fully accessible from the sources as files exactly as before, and as files or objects on the target.
• Scale-out: Komprise scales out on-demand — there are no central bottlenecks, databases or servers to limit scalability.
• No Lock-in: Data is always accessible from your source storage and your target storage, even if Komprise is taken offline.
• Software-based: Komprise does not require dedicated hardware or upfront infrastructure investments.
• Non-intrusive: Komprise analyzes and manages data in the background, with no impact to storage and network performance, and outside the hot data and meta-data paths.
• Adaptive: Komprise throttles back as needed when your storage or network are in active use, so you never have to monitor or schedule when Komprise runs.

One of the best things Komprise does is what it doesn’t do, which is get in the way of users and applications access to data.
Bill Moore
Co-Author, ZFS

Distributed, Scale-Out Architecture

A distributed, scale-out architecture allows Komprise to scale with today’s massive data growth. Start with one Komprise virtual machine and simply add more to scale, balance load and create fault-tolerance. There are no central databases or bottlenecks to limit scalability. Resiliency and high-availability are built-in without requiring any dedicated infrastructure.

Storage Agnostic & Non-Intrusive

Komprise is built to be storage agnostic and non-intrusive in order to work with any environment and adapt as those environments change. Komprise is built on open standards and moves data without putting agents on your storage or by using proprietary, error-prone static stubs. Instead, Komprise uses dynamic links, which are highly resilient and native to filers. Komprise Transparent Move Technology (TMT) also adapts to your environment, so it runs invisibly in the background — without interfering with active usage of your primary storage.

Whitepaper: Komprise Transparent Move Technology

Redundant, Transparent Hierarchical Filesystem

Komprise leverages the sources and targets themselves to create a redundant, overarching namespace. By leveraging open standards and creating a distributed design, Komprise is able to provide unified visibility and management. All permissions and attributes are fully preserved, and no agents or changes to the source NAS are needed. Moved data is accessible as files from the source, and as files/objects on the target – so there is zero lock-in.

Watch the Komprise TMT Chalk Talk

Distributed Search & Analytics

Komprise provides deep insight through a distributed search & analytical framework on all meta-data across storage. This not only enables IT to gain a deeper insights and greater control over its data, but also is used to optimize and drive how Komprise moves, archives, replicates and manages data to suit your particular environment as well as unique data needs.

Komprise Closer Look: Deep Analytics Powered by Elasticsearch

Up & Running In Under 15 Minutes

Deploying Komprise is simple — download the Komprise Observer and point it at existing storage via NFS or SMB/CIFS. Within fifteen minutes, Komprise provides analytics on how much data you have across your storage, how it’s being used, who is using it, and how fast it is growing. Even on petabytes of data, Komprise provides nearly instantaneous results using a patent-pending dynamic sampling technique.

Data Analysis & Interactive ROI Visualization

You set various data management objectives and play with “whatif” scenarios to understand the projected impact on footprint and costs.

Komprise interactively projects:

• How the data footprint at the source will change based on the chosen objectives
• The cost impact of the policy
• The projected Return-on-Investment (ROI) of the policy

This provides a no-risk way to plan capacity, and determine the most effective data management approach before moving data.

The addition of Komprise in our file sharing infrastructure has significantly improved our understanding of the data. Being able to sit down with clients and show them the data allows a level of understanding that was never before available.
Steve DeGroat
Manager Enterprise Storage,
Top Research University

Move Data With No Changes To User & Application Access

Once satisfied, simply activate the plan and Komprise moves the data transparently — without using any proprietary agents or static links on the storage system (mechanisms that have caused problems traditionally). There is no change for users and applications — they continue to see and access the data as they did before.

Handle Data Migrations With Ease

Komprise Intelligent Data Management now includes NAS migrations as a data management capability, so you can eliminate the guesswork and sunk costs associated with migration. Simply pick your source, your destination, and Komprise handles the rest.

Komprise uses a fault-tolerant, highly available architecture. It resiliently moves your data and handles failures automatically, so your data migrations run reliably. It also preserves file hierarchy, NTFS permissions, and all of the meta-data ensuring full integrity of the migration, so you can handle migrations with ease.

The Komprise scale-out, shared-nothing architecture grows on demand enabling you to handle today’s massive scale of data through intelligent automation. Komprise is built from the ground up without centralized bottlenecks and does not use agents, static stubs, or central servers that limit scalability and present single points of failure. With high-availability built-in Komprise handles scale-out by simply allowing you to add more virtual appliances as the amount of data managed grows.

Scales On-Demand To Handle Data Growth

Komprise is designed to deploy in minutes and scale out to handle massive data growth. The architecture is a distributed fault-tolerant scale-out design. As the load on the Komprise Observer grows, additional Observers can be added on-demand. Komprise manages the observers as a single fault-tolerant grid — making management simple while delivering a robust, efficient, scalable, high-performance solution.

Despite tight budgets, data is growing fast. Organizations need a simpler way to efficiently manage data sprawl while curbing growing costs. Komprise is analytics-driven data management software that enables organizations to identify inactive data, assess the ROI of moving old data, and then transparently move it to on-premises or cloud targets of your choice. Unlike legacy data management solutions that are costly, complex, invasive, and hard to scale, Komprise is simple to deploy and operate, seamless to scale, and cost-efficient.

Komprise is so fast, easy and intuitive – in 15 minutes, we had everything up and running!
Wendy Caesar
Director of Information Technology,
Houston Housing Authority