Smart Data Workflows
Automate unstructured data discovery,
detect and take action on sensitive data and prepare data for AI.
Curate Unstructured Data for AI
Unstructured data lacks context and is spread across hybrid IT infrastructure, making it difficult to leverage for AI. Komprise Smart Data Workflows allow you to execute automated processes to discover, tag, segment and move the right data efficiently to AI pipelines. The Smart Data Workflow Manager offers a simple point-and-click UI wizard to search across on-premises, edge and cloud data storage silos to find the data you need, execute Komprise or external AI functions on a subset of data and tag the data with additional metadata. Prepare unstructured data for AI: Search and move only what you need, build custom AI data workflows, protect sensitive data and manage the lifecycle of unstructured data intelligently. Watch the chalk talk > >
Metadata Tagging for AI
Metadata enrichment through tagging transforms unstructured data into AI-ready assets. Apply tags manually or programmatically via APIs by adding key value chains that classify, describe and contextualize data, which allows precise data discovery and processing. Configure tagging workflows that leverage AI to inspect file contents and enrich metadata and then feed that data to other AI workflows. Unlike other tools, Komprise queries on tagged data perform just as quickly and efficiently as standard metadata searches. The ability to tag and search at this scale is essential when dealing with billions of files spread across hybrid and multi-cloud environments.
Smart Data Workflow Use Cases for IT Teams
Image Recognition
- Automate the curation of unstructured data by feeding Amazon Rekognition with both standard and custom LLMs for image analysis.
- Komprise tags extracted insights (e.g., faces, objects, scenes) in its Global File Index for easy search and retrieval.
- Example: A media team preparing training data for an AI model uses Komprise to quickly identify and tag images with specific visual elements across petabytes.
Sensitive Data Management
- Users can identify and tag files containing sensitive data, such as PII or financial information, across disparate storage systems.
- IT teams can automate data management actions like tiering, deletion, or legal hold based on sensitivity and retention policies, reducing risk and manual oversight.
- Example: A compliance officer at a healthcare organization uses Komprise to locate all files containing patient identifiers and move them to secure, access-controlled storage tiers.
Augmented Retrieval with Chatbots
- Augment chatbot responses by feeding relevant corporate data into retrieval-augmented generation (RAG) workflows.
- Komprise finds and filters the right unstructured files to improve chatbot accuracy, without moving all data to a central index.
- Example: A product team uses Komprise to surface only the latest technical documentation for an internal chatbot, keeping responses current and secure.
Watch the Data on the Move discussion: Agentic AI and Unstructured Data Prep
Showback
- Authorized departmental users can monitor and understand their data usage (how many and what type of files, where stored, biggest consumers, etc.)
- Create queries to get real-time visibility and manage costs, rather than ask IT to create static reports.
- Example: A new research initiative at a life-sciences organization requires 100TB of data storage and the project lead needs to work with IT to move older file shares to cloud storage and free up space.
Read the Post: How Storage Teams Use Komprise Deep Analytics > >
User-Driven Tagging
- Users can enrich data with additional metadata tags to facilitate easier search and data management actions such as archival storage or compliance and legal hold.
- Example: HR director needs to ensure that all files with employee PII data are tagged so that information is properly and securely stored for privacy requirements and searchable for deletion when the employee leaves.
Using Amazon Macie with Komprise for Detecting Sensitive Content in On-Premises Data > >
User-Driven Data Mobility
- Authorized users can identify data sets with certain characteristics (such as project or age) to move to cloud storage or other secondary storage for more cost-effective data management or research initiatives.
- Example: All files related to a project need to be copied to the cloud for data analytics. The entire workflow between users and IT is automated via Komprise.
Why Komprise? The fast, easy, no lock-in path to the cloud. > >
Smart Data Workflow Industry Use Cases
Genomics Sequencing Use Case
- Search & Curate: Define and execute a custom query to find all data for Project X with Komprise Deep Analytics and the Komprise Global File Index.
- Execute & Enrich: Execute an external function on Project X data to look for a specific DNA sequence for a mutation and tag such data as “Mutation XYZ”.
- Cull & Extract: Move only Project X data tagged with “Mutation XYZ” to the cloud using Komprise Deep Analytics Actions for central processing.
- Manage Lifecycle: Move the data to a lower storage tier for cost savings once the analysis is complete.
Legal Divestiture Use Case
- Search & Curate: Define and execute a custom query to find all data related to a divestiture project with Komprise Deep Analytics and the Komprise Global File Index.
- Execute & Enrich: Execute an external function to identify PII data and tag it.
- Cull & Extract: Move sensitive data to an object-locked cloud storage bucket and move the rest to a writable cloud bucket using Komprise Deep Analytics Actions.
- Manage Lifecycle: Move the data to a lower storage tier for cost savings once the analysis is complete.
Autonomous Vehicles Use Case
- Search & Curate: Find crash test data related to abrupt stopping of a specific vehicle model with Komprise Deep Analytics and the Komprise Global File Index.
- Execute & Enrich: Execute an external function to identify and tag data with “Reason = Abrupt Stop”.
- Cull & Extract: Move only the related data to the cloud data lakehouse to reduce time and cost associated with moving & analyzing unrelated data using Komprise Deep Analytics Actions.
- Manage Lifecycle: Move the unrelated data to a lower storage tier for cost savings (or delete it) once the analysis is complete.
Webinar: Building a Modern Data Strategy for the Automotive Industry > >
Unstructured Data Classification
Search, classify and index data across file, object and cloud data storage to build a comprehensive Global File Index.
Unlock Unstructured Data Value for AI
Systematically and continuously prepare and move data from multiple unstructured data storage silos to the right AI target location.
Fuel Enterprise AI
Efficiently find, enrich, move only the data you need for AI projects and optimize the AI data pipeline and information lifecycle across hybrid environments.