Unstructured data is tricky to manage because of its volume, velocity and variety. Komprise solves the problem of rapidly finding small data sets in petabyte-volume environments and moving it to new environments for analytics and compliance needs.
Data continues to pile up at the edge, in data centers and in clouds. In a large enterprise, this can easily be billions of files. Data stewards often need to search through all this data and find what they need for regulatory compliance or to support analytics for customer intelligence or new product research. For instance, pharmaceutical companies routinely need to provide the raw data files during an inspection—even if some of that data might be in the cloud and some might be at different datacenters. A company going through a merger may need to split some data to a new entity or identify files for deletion.
How can they easily find this subset of data and move it securely with a full audit trail? This can be a highly manual process which doesn’t deliver what stakeholders need or in a timely manner. To that end, the Komprise Intelligent Data Management Fall 2021 release introduces Deep Analytics Actions, a systematic way to find specific data across hybrid cloud storage silos and move just the right subset of data for new uses such as cloud analytics. This gives IT and storage departments the ability to drive closer connections with end users by liberating the nuggets of useful data from petabytes of files, so that new value and customer-facing benefits can be discovered.
Key Features of Deep Analytics Actions
Unstructured data is tricky to manage because of its volume, velocity and variety. Traditional database-driven indexing does not work on unstructured data because of its sheer volume: it can routinely be several billions of files. Any analysis of unstructured data needs to run in the background and not interfere with active users and applications. Unstructured data piles up in multiple silos, so an index needs to be global across different datacenters, storage architectures and clouds. And analytics alone is not enough—data needs to be actionable. Finally, how do you make this easy to spin up, easy to operate, and easy to manage without burdening the customer?
These are the challenges that Deep Analytics Actions solves:
- Global Metadatabase Service across datacenters, clouds, file and object architectures: Once you connect Komprise to your file and object storage, Komprise indexes the data and creates a Global Metadatabase Service, which is an index of all your data. You do not have to move the data anywhere; but you now have a single way to query and search across all file and object stores. For instance, say you have some NetApp, some Isilon, some Windows servers, some Pure Storage at different sites and you have some cloud file storage on Amazon, Azure, and Google. You get a single index via Komprise of all the data across all these environments. You and your users can search and find exactly the data you need across all these environments with a single console and API.
- No central database or bottlenecks: Komprise is designed to handle unstructured data at scale and uses no central databases or central servers. The solution delivers a completely distributed Elastic Grid architecture both for indexing and data movement.
- Elastic scaling in the cloud: You can start by pointing Komprise at a few storage servers and add more on the fly. Komprise elastically scales the index in the cloud so you don’t have to manage the infrastructure.
- Policy-driven data movement on Deep Analytics queries: Once you find the data you want to operate on, you can systematically move it using Komprise. For example, if you want to tier files generated by certain instruments to the cloud, you can create a policy in Komprise so that as new files are generated, they are continuously and automatically moved by Komprise. This makes it easy to systematically leverage analytics to move and operate on data.
- Extensible with tagging, APIs: You can set tags to augment the data that Komprise indexes. Tags can be set outside of Komprise via API or within Komprise. For instance, an organization that is acquiring another entity tags some data from the acquiree as going to one department and other data as going to another department. Deep Analytics queries can then be used to find data based on the tags as well as standard metadata. Komprise also makes all Deep Analytics Actions capabilities available via API so you can incorporate it easily into your business workflows.
- No infrastructure to manage: The best part is that setting this up is easy. Deep Analytics Actions runs as a managed hybrid cloud service; the index is maintained in the cloud by default and you don’t have to manage any infrastructure to run it or scale it.
Use Cases for Deep Analytics Actions
Expanding the popular Komprise Deep Analytics capabilities, Deep Analytics Actions allows customers to:
Find and ingest the right file data into cloud analytics, data warehouse and data lakes.
For example, a manufacturer wants to analyze customer maintenance data for a new line of equipment from a two-day period and then compare it to similar data sets from other periods. The target application could be Snowflake, Databricks, or other popular cloud data management and cloud analytics technologies.
Find and delete specific obsolete data.
A common example is to purge ex-employee-generated emails that have not been accessed in over three years but handle exceptions such as for legal hold.
Comply with regulations.
Identify just the data that needs to be retained and move it to an object-locked cloud bucket. For instance, pharmaceutical companies are often required to produce the raw files during a drug inspection to avoid regulatory violations called “quality of concern.” Komprise can find and move the raw files using Deep Analytics Actions.
Enable “user-driven data management”.
Users can partner with IT on data management by identifying specific data sets they are interested in through Deep Analytics queries, which are then fed into the data movement and data management plans run by IT.
Benefits of Komprise Deep Analytics Actions:Expanding the popular Komprise Deep Analytics capabilities, Deep Analytics Actions allows customers to:
|
_______________________
Other Komprise Fall 2021 Updates
With each release Komprise adds features to support a broader set of customer requirement and conditions:
- Enhanced security for data in flight to EFS with Encryption in Transit;
- Ability to migrate hard links and Unix special files;
- Option to migrate over NFSv4 using ctime;
- Improved capacity management for migrations:
- Add new warning when destination is running out of space
- Improved support for migrations from arrays that have reached full capacity.
More Resources on Komprise Fall Update:
Learn more about our latest releases: www.komprise.com/whatsnew.
