Data Management Glossary
Data Annotation
What Is Data Annotation?
Data annotation is the process of labeling raw data, text, images, video, audio, or sensor data, with tags that identify what the data represents so a machine learning model can learn from it. These labels become ground truth: the correct answer a supervised learning model is trained to predict. A bounding box drawn around a pedestrian in a street photo, a sentiment tag on a customer review, and a named entity tag on a clause in a contract are all examples of data annotation. Without labeled examples, a model has no reference point for what a correct output looks like.
Data annotation is closely related to but distinct from data tagging. Tagging generally applies metadata for search, retrieval, and management (finding files by project, sensitivity, or owner). Annotation applies labels specifically to teach a model, and the label itself becomes part of the training signal, not just a way to find the file later. See Data Tagging for the broader metadata practice.
Data Annotation for Unstructured Data
Most data that needs annotating is unstructured. Structured records already have fields and columns, so labeling a database row is mostly a matter of adding a column. Unstructured data (files, images, video, audio, etc.) has no inherent schema, which means every object needs its own labeling logic. A radiology image needs modality, body part, and finding labels. A legal contract needs clause type and party labels. A product photo needs category and attribute labels. There is no single label set that applies across all of it.
Scale compounds the problem. Enterprise unstructured data is growing 40-60% annually, spread across NAS, cloud, and object storage silos with no consistent naming or organization.
Source: Gartner, cited in the Komprise 2026 State of Unstructured Data Management Report
That growth is landing at the worst possible moment for storage economics. Gartner forecasts NAND flash prices will rise 234% in 2026 alone as part of the broader memory price inflation it calls memflation, with no meaningful relief expected until late 2027. Learn more about Flash Stretch.
Every inefficient annotation approach gets more expensive to run against that backdrop: staging full copies of files to label them, re-scanning data that is already tagged, or storing annotation output as a duplicate dataset instead of queryable metadata all add storage footprint at a moment when that footprint costs far more than it did a year ago. Manually reviewing and labeling that volume of file data does not scale, and treating every file the same way, applying one fixed label schema regardless of file type, produces annotation that is either too generic to be useful or too narrow to cover the data that does not fit the mold, all while paying a growing storage bill for holding it.
Why Annotating Everything Is the Wrong Approach
The instinct at enterprise scale is to annotate broadly: run every file through a labeling pipeline so nothing gets missed. This is expensive and often counterproductive. Cloudera research found that nearly 80% of AI project time goes toward gathering, organizing, and labeling data rather than building or training models, and a large share of that time is spent annotating data nobody ends up using.
Most enterprise file stores also contain a large share of ROT data, redundant, obsolete, and trivial content that has no business being annotated or fed to a model at all. Annotating ROT data wastes annotation budget and AI compute on files that will never improve model accuracy. See ROT Data.
Model performance is also more sensitive to data quality than to volume. Technavio research on the data labeling and annotation tools market found that over 70% of model performance improvements are attributed to data quality rather than architecture changes.
Annotating a smaller, well-curated, correctly labeled dataset beats annotating everything with less rigor. The right question is not “how do we label all of it,” it is “which files actually need labels, and what labels do each of them need.”
How Komprise Enables Progressive Data Annotation
Komprise takes a progressive approach to annotation: identify what needs labels, apply the right schema for each object type, and add more metadata over time as new use cases emerge, instead of annotating the entire data estate upfront with one fixed schema.
This starts with the Global Metadatabase, a continuously updated index of standard and custom metadata across every storage silo, built on a distributed architecture that indexes file and object metadata in place without moving or copying data. Because the index already knows file type, age, owner, and existing tags across the entire estate, Deep Analytics can query it to identify exactly which files are candidates for annotation before any labeling work happens. This is the opposite of scanning everything: it narrows the field first.
From there, Smart Data Workflows and KAPPA data services apply annotation with a flexible schema, not a fixed one. A radiology DICOM file gets tagged with modality, body part, and study type extracted from its imaging headers. A genomics BAM file gets a different set of tags entirely, extracted from its own header format. A contract gets clause and party tags pulled from its text. Each object type keeps its own metadata schema, because a single one-size-fits-all label set cannot capture what makes each of those file types useful to a model. Because the Global Metadatabase supports custom, per-object metadata rather than a rigid fixed schema, new tag fields can be added as new use cases come up, without re-architecting the index or re-annotating everything that came before.
This also lets annotation happen progressively. A team does not need to annotate an entire repository before it can use any of it. Deep Analytics can identify and annotate a first, high-value subset, put it to work, then progressively expand the annotated set as new needs emerge, rather than blocking on a single all-or-nothing labeling project. Smart Data Workflows can also filter out ROT data and files matching sensitive-data patterns identified by Smart Data Workflows PII scanners before annotation ever runs, so labeling effort and downstream AI compute go toward files that are actually worth the tokens spent processing them.
Choosing an Approach to Data Annotation at Enterprise Scale
Enterprise teams generally choose between three approaches: fully manual annotation, a fixed-schema annotation tool applied uniformly across all data, or a metadata-driven approach that targets and schemas annotation per object type. The table below outlines what to evaluate.
| Evaluation Criteria | Without Progressive, Metadata-Driven Annotation | With Komprise |
|---|---|---|
| Schema flexibility | One fixed label schema applied to every file type, regardless of fit | KAPPA data services extracts a custom metadata schema per file type (DICOM, BAM, contracts, and more) |
| Scope of annotation | Entire repository annotated upfront, or annotated ad hoc with no prioritization | Deep Analytics queries the Global Metadatabase to identify and annotate targeted subsets first |
| ROT and low-value data | Labeling budget and AI compute spent on redundant, obsolete, and trivial files | Smart Data Workflows filter out ROT data before annotation and AI ingestion |
| Sensitive data handling | Sensitive content annotated and exposed with no upfront check | Smart Data Workflows scan file content for PII using 68 built-in scanners plus custom regex before annotation |
| Path to scale | Manual reviewers labeling files one at a time across disconnected storage systems | Workflows run across every NAS, cloud, and object storage silo from one Global Metadatabase index |
| Adding new label types later | Re-annotating the dataset from scratch as new use cases emerge | New custom metadata fields added progressively without re-processing prior annotation |
Data Annotation Frequently Asked Questions
What is data annotation?
Data annotation is the process of labeling raw data, such as text, images, video, or audio, with tags that identify what the data represents so a machine learning model can learn from it. The labels serve as ground truth during supervised model training.
What is the difference between data annotation and data tagging?
Data tagging generally adds metadata for search, retrieval, and management, such as project name or sensitivity level. Data annotation applies labels specifically so a model can learn from them, and the label becomes part of the training signal itself, not just a way to locate the file later.
Do you need to annotate all of your data before using it for AI?
No. Annotating an entire data estate upfront is expensive and often wasteful, since much of that data may be redundant, obsolete, trivial, or simply irrelevant to the model being trained. A metadata-driven approach identifies which files are worth annotating first, applies the right label schema to each, and expands the annotated set progressively as new use cases emerge.
Why does data annotation need a flexible schema instead of one standard set of labels?
Different object types carry fundamentally different information. A medical image, a genomics file, and a legal contract each need their own label set extracted from their own structure. Forcing every file type through one fixed schema either produces labels too generic to be useful or misses the attributes that make a specific file type valuable to a model.
How does Komprise support data annotation for AI at enterprise scale?
Komprise uses the Global Metadatabase to index file and object metadata across every storage silo, then uses Deep Analytics to query that index and identify which files should be annotated. Smart Data Workflows and KAPPA data services then apply a custom metadata schema per object type and filter out ROT data and sensitive content before annotation happens, so annotation effort and AI compute go toward files worth processing.
Where does data annotation fit in the AI data platform?
Data annotation sits at Layer 4 of the AI data platform: Enrichment and Curation. It follows Layer 2, Metadata and Discovery, which builds the index annotation is queried against, and Layer 3, Classification and Governance, which determines what is sensitive and what is ROT before labeling effort is spent. See AI Data Platform for how all five layers work together.