Data quality in practice: how to know if your data is really reliable
Dimensions, metrics, rules and a treatment cycle: a practical guide to identifying, measuring and fixing data quality problems in industrial environments.

In this article
A filled-in field is not necessarily a correct one.
The most banal example shows up in any customer or supplier database. Three systems at the same company record the same entity like this:
NORVALE | NORVALE S.A. | Norvale Indústria Brasileira S.A.
None of the three is wrong. Every field is filled in, no validation was violated. But when someone consolidates revenue by customer, three customers show up where there is really one. The ranking changes, the total per account changes, and the revenue-concentration analysis ends up describing a reality that doesn't exist.
The problem is not typing: it's the absence of a unique entity identifier and a writing standard. Without a stable identifier — a tax ID, a master code — no manual fix solves it, because the next record will arrive with a fourth variation. This pattern repeats in industrial environments with equipment, materials, cost centers and suppliers.
1. What data quality means
Data quality is not a single attribute — it's a set of dimensions that can fail independently. A piece of data can be complete and inaccurate, precise and outdated, current and inconsistent with another system. Treating “quality” as one single thing is what keeps fix-it projects from converging.
| Dimension | Definition | Example of failure | Impact |
|---|---|---|---|
| Completeness | Required fields are filled in | 30% of production orders with no equipment code | Impossible to calculate availability by asset; silently partial analyses |
| Accuracy | The value matches the real-world fact | A record of 500 units when the line produced 480 | Distorted productivity and unit-cost indicators |
| Consistency | The same fact has the same value everywhere | Shift production diverging between the MES and the ERP | Meetings spent reconciling numbers instead of deciding |
| Timeliness | The data is available within the window where it's useful | Sensor with no new reading for 6 hours on a real-time dashboard | Decisions made based on a past state of the process |
| Uniqueness | Each entity appears only once | The same equipment registered under two codes | Split maintenance history; recurring failures go undetected |
| Validity | The value follows rules, formats and domains | Unit of measure out of standard, date in a divergent format | Rejected loads, wrong conversions, meaningless averages |
2. A quality problem takes many shapes
In industrial environments, the cases below show up frequently and rarely trigger any alert — the systems accept all of them:
Equipment with no code
The maintenance order exists, but isn't linked to an asset in the master registry. The cost gets recorded and can't be attributed.
Invalid maintenance date
A record with a future date or one earlier than the equipment's installation. The MTBF calculation ends up including negative intervals.
Inconsistent unit
The same quantity arrives in bar and in kPa depending on the gateway. The column average stops meaning anything.
Sensor with no update
The last valid value keeps being shown as if it were current. The dashboard looks healthy while collection has stopped.
Duplicate record
Load reprocessing generates the same entry twice; shift production appears inflated.
Order with no plant
Production order with no link to a plant or line. It disappears from analyses filtered by unit, without anyone noticing the gap.
Divergence between systems
Inventory in the ERP differs from what's recorded in the MES, with no clear rule on which source prevails.
This means small, localized problems propagate. A missing link between an order and a plant doesn't look serious until the moment a per-unit report omits 4% of production — and the capacity-allocation decision is made using that number.
3. How to measure data quality
Measurement is what separates perception from management. The formulas below are deliberately simple: the value is in calculating them automatically, in the same place, with the same definition, over time.
| Metric | Calculation | What to watch |
|---|---|---|
| Completeness | records filled in / expected records | Measure by critical field, not the whole table |
| Duplication | duplicate records / total records | Depends on a defined business key; without a key, there's no identifiable duplicate |
| Timeliness | average data age or delay versus expected | Compare against the decision window, not a generic target |
| Validity | records that meet the rules / records evaluated | Only makes sense with declared, versioned rules |
| Accuracy | correct records / records evaluated | Requires an external reference: inspection, physical count, authoritative system |
Two warnings. First: these formulas are examples and should be adapted to context — what counts as “expected” for completeness depends on the business rule for each field. Second: accuracy is the hardest dimension and the one most often faked. Without an independent source of truth to compare against, what gets measured isn't accuracy, it's internal consistency.
4. Data quality rules in practice
Quality rules are declared checks, executed automatically, tied to an action when they fail. Some examples applicable to an industrial environment:
| Rule | Type | Action on failure |
|---|---|---|
| Temperature cannot be lower than -273.15 °C | Validity — physical limit | Reject the record and alert the collection team |
| Maintenance date cannot be later than the current date | Validity — temporal | Block the entry at the source |
| Equipment code must exist in the master registry | Referential integrity | Send to quarantine and notify the registry owner |
| Every production order must be linked to a plant | Mandatory completeness | Prevent the order from being closed |
| Unit of measure must belong to the standardized domain | Validity — domain | Convert when a rule exists, or reject |
| A critical sensor reading cannot be older than 5 minutes | Timeliness | Mark the indicator as unreliable on the dashboard |
| Vibration must be between 0 and 50 mm/s | Range plausibility | Flag for analysis without discarding the value |
The most important column is the third one. A rule with no defined action turns into a report no one reads. And choosing the action is a business decision, not a technical one: rejecting an implausible temperature record might be correct for a cost calculation and wrong for a failure investigation, where that exact anomalous value is the point of interest.
5. What to do when a problem is found
- 01DetectAutomated rule flags the failure
- 02ClassifyDimension, criticality, affected volume
- 03Identify originWhich layer the deviation is born in
- 04FixAdjustment at the point of origin
- 05PreventValidation or control that stops recurrence
- 06MonitorOngoing tracking of the metric
- Detect: the failure shows up via an automated rule, not because someone noticed a strange number in a meeting.
- Classify: which dimension failed, how many records were affected, which decisions depend on them. Classification is what allows prioritization.
- Identify origin: find out at which layer the deviation first appears — collection, integration, transformation or consumption.
- Fix: act at the point where the problem is born; fixing downstream copies is only a temporary remedy.
- Prevent: add input validation, a domain constraint, a unique key, or adjust an operating procedure.
- Monitor: keep the metric visible to confirm the fix worked and to catch recurrence.
The most frequently skipped step is “prevent”. Without it, the cycle turns into a routine of manual fixes that consumes the team's time and never reduces the incidence rate — and the internal perception becomes “the data is bad”, with no one able to point to where.
6. Quality and root cause
Consider the following scenario. The management dashboard shows production higher than reality on one of the lines. An analyst identifies the problem, adjusts the dashboard's query to disregard duplicate records, and the number returns to what was expected. Three weeks later, the same deviation reappears — now also in a cost report no one had adjusted.
The dashboard fix wasn't a correction: it was a filter that hid the symptom at a single point of consumption. The duplication kept being recorded, and every consumer that didn't receive the same filter kept showing the inflated value.
- 01SourceDuplicate entry, uncalibrated sensor
- 02IntegrationResend without idempotency, overlapping window
- 03PipelineReprocessing without a unique key
- 04TransformationA join that multiplies rows
- 05StorageNo uniqueness constraint
- 06ConsumptionA filter that only masks the symptom
The investigation follows the same layer-by-layer descent used in traceability: check at which layer the duplication is already present. If the raw layer already contains two identical records, the cause predates the pipeline. If it contains one and the analytical table contains two, the cause is in the transformation — most likely a join with unverified cardinality.
Quality problems often arise during integrations and transformations. The content on data lineage explains how to trace that path layer by layer.7. How to build a data quality indicator
A quality dashboard doesn't need to be sophisticated. It needs to be specific, always calculated the same way, and have someone responsible for each line. A minimal functional structure:
| Indicator | Suggested scope | What it's for |
|---|---|---|
| Completeness by critical field | By table and by required field | Show where the data that supports indicators is missing |
| Sample accuracy | Comparison against a physical count or authoritative system | Estimate real deviation, when a reference is available |
| Cross-system consistency | Pairs of sources describing the same fact | Catch divergences before the results meeting |
| Timeliness | By source, with a declared expected window | Tell 'nothing new' apart from 'collection stopped' |
| Duplication | By business key | Detect reprocessing and missing uniqueness constraints |
| Error rate by rule | By rule and by period | Prioritize which rules cause the most rework |
| Rejected / quarantined records | Volume and time on hold | Prevent quarantine from becoming a permanent dump |
| Average time to fix | By problem type and owner | Assess response capacity, not just incidence |
Deliberately, there are no target values here. Quality targets only make sense based on the operation's own baseline and the criticality of each piece of data: requiring 99% completeness on a field that supports no decision wastes effort, and accepting 90% on a field that feeds a cost calculation compromises the result. Measure first, set the target later.
8. Final checklist
- Are there defined and documented quality rules for the critical data?
- Are those rules executed automatically, rather than through manual checking?
- Is there continuous monitoring of quality metrics?
- Does each type of problem have a named owner?
- Can the source of errors be identified layer by layer?
- Is there a history of the problems found and the fixes applied?
- Do critical data points have declared minimum quality criteria?
- Is quality tracked continuously, rather than only in one-off projects?
If most answers are no, the most efficient starting point isn't a broad quality program. It's choosing the three data points that support the most expensive decisions, declaring their rules, automating the check, and measuring. The results of that measurement usually justify — with numbers from the operation itself — extending the work to the rest of the environment.
Conclusion
Reliable data doesn't come only from good collection. It depends on declared rules, automated monitoring, traceable origin and continuous improvement processes. Collection solves availability; reliability is built afterward, and maintained.
This means quality is an operational practice, not a project with an end date. Every new source, every rule change and every added integration reintroduces the same questions — and the difference between a reliable environment and one in constant reconciliation lies in having automated answers for them.
See how Athenas organizes your operation's data
A roughly 40-minute demo showing source connectivity, the catalog, the ISA-95 UNS and real-time data distribution.
Keep reading

