Episode highlights: Krishna Subramanian, COO and co-founder of Komprise, on IT Ops Query from Informa TechTarget Eye on Tech
Enterprise AI succeeds or fails on the data it can reach. On IT Ops Query, host Beth Pariseau asked Krishna Subramanian why unstructured data is the hardest part of AI context engineering, and what IT leaders should do about it. Below are the key questions from the conversation and a summary of each answer.
Why is unstructured data so hard for AI context engineering?
AI models are getting better fast, so what sets an enterprise apart is how well it connects AI to its own data. Subramanian points out that most of that data is unstructured. It isn’t sitting in a database, a spreadsheet or a lakehouse. It lives in files and objects: MRI scans, video recordings, documents, Slack conversations. Unstructured doesn’t mean the data has no structure. It means the data comes in many different structures, and applications can’t see them. Making sense of that data for AI is the core challenge.
How does Komprise compare with distributed data platforms from Google and IBM?
Subramanian calls these approaches complementary. Google federates agentic AI across lakehouses, and IBM connects to streaming sources such as Kafka. Lakehouses work with data that can be put into tables, but almost none of an enterprise’s unstructured data ever reaches a lakehouse or an AI system. She describes that data as dark.
Komprise connects to file and object storage on premises and in the cloud. It extracts metadata, builds a Global Metadatabase and normalizes that metadata into tables. Those tables can then go into a lakehouse such as Google BigQuery, where agentic frameworks can use them.
Isn’t chunking and vectorizing enough to prepare data for AI?
No. Subramanian notes that much enterprise unstructured data is more than five years old. Chunking, embedding and vectorizing stale data is “garbage in, garbage out.” AI can return answers that are out of date, and you won’t know it. The data has to be prepared first:
- Classify the data.
- Remove outdated and conflicting information.
- Extract the right metadata.
- Apply security and governance so that each user only gets answers from data they are allowed to see.
RAG, embeddings and context graphs work well after that. The hard part is finding the right data across petabytes built up over decades.
How much can technology solve, and how much is data hygiene?
Technology can’t solve everything, but it can do most of the work. Komprise brings out information that is already in files but hard to find: age, usage, content, sensitive data, and domain details such as which body part a medical image shows or which instrument created it. A person may still need to decide which of two conflicting sources is authoritative. Komprise remembers that decision and applies it the next time. The human stays in the loop, but systematic extraction at scale cuts their work sharply.
How does Komprise approach data classification?
Subramanian names two problems with traditional classification:
- Context matters. Applying one context to everything makes classification go stale. Simple key-value tags are limited. The Komprise Global Metadatabase holds richer metadata, including numerical values, so classification can be specific and still work at scale.
- No vendor is an expert in everything. KAPPA data services let customers add their own extraction logic, such as a few lines of Python for a specific image type. Komprise runs the function and writes the results to the Global Metadatabase. This open, extensible design matters most in highly regulated industries.
Does data preparation have to happen before AI agents go into production?
In practice, it happens at the same time. Teams pick narrow use cases and prepare data by hand for pilots, then expect the same approach to work at scale. Subramanian links that gap to the high failure rate of enterprise AI projects. Classification, preparation, governance and curation become essential at scale.
She describes a financial services customer that was unhappy with Copilot results across all of its data. Komprise removed outdated, duplicate and redundant files, tagged sensitive data and added contextual metadata to show the AI where to look. “More data is not good for AI,” she says, because context windows are limited. Measured on 100 queries before and after, accuracy improved by 135%.
Where are most enterprises today?
According to Subramanian, the market is still in a hype cycle. Pressure to adopt AI comes from the top, and the groundwork of preparing data often hasn’t happened. Most organizations are unprepared for AI, and many don’t realize it.
What should IT leaders know about context engineering?
Chunking and vectorizing are the last mile. About two-thirds of the problem comes before that. Subramanian says every enterprise should ask whether it has a systematic way to expose only the right data to each AI use case. Without one, it takes on liability and security risk and gets answers it can’t verify. Komprise calls this the first mile problem.
Do enterprises save money by adding this layer?
Yes. A leading U.S. cancer hospital wanted to use cloud AI on pathology images that can run to terabytes each and add up to petabytes a year. Copying all of it to the cloud was expensive and created HIPAA risk. Komprise extracts DICOM metadata, copies only the images relevant to a case for AI processing and deletes them afterward. The hospital cut its cloud costs by 97% and stayed HIPAA compliant.
What about storage vendors adding data intelligence?
Subramanian sees storage vendors entering the market as proof the problem is real. But AI is supposed to break down silos, and storage-based classification deepens them. Each vendor classifies only the data on its own storage. That leaves customers reconciling several proprietary systems and locked in when they switch vendors, because data lives for 20 years or more while storage lasts three to five. Komprise keeps data and metadata in open standards such as Apache Iceberg, so customers can move freely, including away from Komprise. “Lock-in at the metadata layer can be brutally expensive,” she says.
How will context engineering evolve?
Subramanian describes three dimensions of context:
- Content: what’s inside the file.
- Relationships: who uses the data, where and when.
- Time: how the data changes and is used over time.
Because Komprise runs analytics across time, it can bring all three together into multidimensional context.
Can AI agents build their own context layers?
Yes, if it’s done carefully. Komprise has used AI to extract metadata and feed it back to AI. Generative AI can hallucinate, though, and false context that gets built on can spread errors. That is why every Komprise workflow includes a human in the loop. In one project, AI identified people in images with more than 99% accuracy, and human review still caught false positives before the tags were saved.
What’s the difference between a metadatabase and a knowledge graph?
Metadata describes the data itself. Relationships connect separate entities, such as linking a garment to the person who would wear it or the country where it was made. Relationships can be stored as metadata, but they are a narrower concept. Komprise combines relationships with other metadata.
Where should IT teams start?
Start by knowing your data. Teams make storage, backup and AI decisions without visibility into unstructured data, and doing it by hand doesn’t scale because storage teams and users often aren’t the data owners. Komprise starts with analytics across the full data estate, covering discovery, indexing and insight, so teams can set priorities based on facts.
Listen to the full episode of IT Ops Query on the Eye on Tech YouTube channel.