Why Data Quality Starts Before the Dashboard

A plausible chart can still be wrong

Imagine a fictional merchant receiving sales exports from two branches. Both files contain an amount column, but one branch records local currency and the other records a different currency. Adding the amounts produces a mathematically valid total with no coherent business meaning. A polished chart does not repair the missing definition.

This scenario is illustrative. It is not an investigation of a named merchant or a claim about a particular African payment system. Its purpose is to make a common design question visible: what must be true before a metric is worth displaying?

Define one row

Decide whether a row is an order, an item, a payment attempt or a settled payment. These are different events. An order containing three items may appear in three rows without representing three separate purchases.

Write the definition beside the dataset. Then specify the identifier used to count events. If two source systems reuse the same numerical identifier, include the source in the key rather than assuming global uniqueness.

Make reconciliation a first-class output

For every import, record input rows, accepted rows, rejected rows and duplicate rows. State whether duplicates are part of the rejection count; otherwise your reconciliation can double-count them. Keep the input batch identifier so that someone can trace a report back to its source file.

An illustrative batch might contain 100 rows, of which 93 are accepted, five fail validation and two are duplicates. Under that exclusive classification, 93 plus five plus two should equal 100. A discrepancy deserves investigation before reporting business totals.

Enforce the rules you can express

Relational databases can express requirements through constraints such as uniqueness, non-null values and references to related records. Transactions let a set of database changes succeed together or be rolled back together. These mechanisms help enforce specific rules; they cannot decide whether the business definition itself is sensible.

For a learning exercise, add a repeated order identifier and a reference to a missing customer. Observe which rules reject them. Record what happens to the rest of the import when one row fails.

Separate missing information from zero

A missing payment amount is not necessarily a zero-value payment. A missing date is not today. Define explicit handling for absent information instead of allowing convenient defaults to change meaning.

Keep rejected records and reasons in a controlled location. If the data is sensitive, apply appropriate access restrictions and retention decisions rather than putting the raw exception file into a public repository.

Explain the report’s boundary

A report should state the period covered, time-zone rule, currency treatment, excluded records and last successful import. If it reports orders rather than collected cash, say so. Let a reader distinguish measured values from estimates.

For teams working across countries, the names of the columns are not enough to establish comparability. Capture the definitions alongside the numbers. That documentation often delivers more value than another dashboard visualization.

Sources and further reading

PostgreSQL: Constraints

PostgreSQL: Transactions