Hardware FixRecommendedDevice not working? Your driver may be the problemCheck updates for common hardware issues.Fix DriversOctober DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsWindows FixRecommendedWindows errors stealing your time? Find the fix fastScan stability, cleanup and performance issues.Fix Now×
Skip to content
World desk5 min

SQL Data Quality: Profile Records, Set Rules, Prevent Errors

A practical PostgreSQL workflow for inspecting data, identifying duplicate candidates, handling NULLs, analyzing aggregates, and enforcing confirmed rules.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Use SQL to profile a table, identify questionable records, apply explicit cleaning rules, and summarize the results. The safest approach is to inspect first, preview every proposed change, and separate three jobs: finding anomalies, deciding what they mean, and preventing invalid data from entering again. The examples below use PostgreSQL; check your database’s documentation before relying on syntax or edge-case behavior in another engine.

Start by defining what a row represents

Before changing data, establish the table’s grain: what one row is supposed to represent. A repeated customer ID, for example, may be an error in a table intended to have one row per customer, but entirely valid in an orders table. Also identify the database engine, table definition, key columns, and any existing rules.

As an Amazon Associate I earn from qualifying purchases.

Inspect representative rows and column types, then profile the table. This PostgreSQL query gives a row count and counts NULL values in selected columns:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
SELECT
  COUNT(*) AS row_count,
  COUNT(*) FILTER (WHERE customer_id IS NULL) AS missing_customer_ids,
  COUNT(*) FILTER (WHERE order_total IS NULL) AS missing_order_totals
FROM orders;

COUNT(*) counts rows; COUNT(column) counts only rows where that expression is not NULL. PostgreSQL’s built-in aggregates generally ignore NULL inputs, so the two counts answer different questions. See the PostgreSQL documentation on aggregate functions.

Profile values and suspected keys before deciding what to change:

SELECT status, COUNT(*) AS rows_per_status
FROM orders
GROUP BY status
ORDER BY rows_per_status DESC;

SELECT customer_id, COUNT(*) AS rows_per_id
FROM customers
GROUP BY customer_id
HAVING COUNT(*) > 1
ORDER BY rows_per_id DESC;

These queries surface distributions and candidate duplicates; they do not establish that a value is wrong. Write down the business rules for required fields, valid ranges, and how to select a canonical record before editing.

Distinguish missing values from zero and other values

In PostgreSQL, NULL represents an unknown or absent value, not zero, an empty string, or a literal word such as “unknown.” Decide what a missing value means in the specific column before replacing or excluding it. For example, a missing order total is not necessarily equivalent to a zero-dollar order.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Most PostgreSQL aggregates ignore NULL inputs. In addition, SUM over no selected rows returns NULL, not zero. Use COALESCE only when a fallback is meaningful for the question:

SELECT COALESCE(SUM(order_total), 0) AS total_order_value
FROM orders
WHERE order_date >= DATE '2026-01-01';

Here, the query deliberately reports zero if there are no qualifying non-NULL totals. That choice changes the presentation of “no aggregate result” into a numeric zero; it should match the intended interpretation. PostgreSQL documents these aggregate behaviors in its aggregate reference.

Find duplicate candidates and choose a record rule

SELECT DISTINCT removes identical rows from a query’s output. It does not decide which source record to preserve when rows share a business key but differ in other fields. In PostgreSQL, DISTINCT ON can return one row per key, but the chosen row is unpredictable unless the ordering determines which record comes first.

For instance, if the rule is “keep the most recently updated customer record, breaking ties by highest customer row ID,” express both parts of that rule in the ordering:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
SELECT DISTINCT ON (customer_id)
  customer_id, email, updated_at
FROM customer_imports
ORDER BY customer_id, updated_at DESC, import_row_id DESC;

This is a selection rule, not proof that the chosen record is correct. Confirm that the key and tie-breakers reflect the business definition, and inspect the candidates before deleting or overwriting anything. PostgreSQL explains duplicate elimination in SELECT lists and the ordering requirements for SELECT and DISTINCT ON.

Understand query order before interpreting a summary

A summary describes the rows that reach its grouping and aggregate stage, not necessarily every row in the table. In PostgreSQL’s documented SELECT processing, filtering determines the input rows; grouping and aggregate calculations operate on that input; result expressions and duplicate elimination shape the output; then ordering and limiting affect what is returned. Moving a filter or limit can therefore change the meaning of an analysis.

For example, this query counts orders by status only after restricting the date range:

SELECT status, COUNT(*) AS order_count
FROM orders
WHERE order_date >= DATE '2026-01-01'
GROUP BY status
ORDER BY order_count DESC;

Do not limit the input records before calculating a table-wide total unless a subset is what you intend to analyze. PostgreSQL’s SELECT documentation describes the processing behavior and syntax.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Turn confirmed rules into constraints

Queries can detect suspicious values and help clean existing rows. Constraints express rules the database should enforce on future writes. PostgreSQL supports NOT NULL, CHECK, UNIQUE, primary-key, and foreign-key constraints. Use them only after confirming the rule and resolving existing violations; a constraint cannot infer what the business considers valid.

  • Required value: use NOT NULL when a value must be present.
  • Allowed range or condition: use CHECK for a condition such as a nonnegative amount.
  • Unique identity: use UNIQUE or a primary key when the key must identify one row.
  • Relationship: use a foreign key when a value must refer to a valid row in another table.

A PostgreSQL CHECK constraint passes when its expression evaluates to NULL, so it does not by itself require a value. Pair it with NOT NULL when presence is part of the rule. Also, ordinary PostgreSQL uniqueness permits multiple rows with NULL in a constrained column, so a unique constraint is not a substitute for a required-value rule. Review the PostgreSQL constraints documentation for these behaviors and syntax.

Apply changes reversibly and validate the result

  1. Identify the engine, table grain, types, and keys. Confirm what each row represents and which columns define identity.
  2. Profile before editing. Record row counts, NULL counts, value distributions, and duplicate-key candidates.
  3. Write the rule. Specify valid values, required fields, and the selection rule for records that share a key.
  4. Preview candidate changes with SELECT. Inspect the exact rows that would be changed or excluded, and compare them with the rule.
  5. Plan recovery. Use an appropriate backup and transaction strategy for the size and risk of the change; do not run destructive statements before confirming how to recover.
  6. Make the change, then compare. Recheck row counts and key distributions, run validation queries, and add constraints where confirmed rules should apply to future writes.

For order-sensitive aggregates, specify input ordering when the aggregate’s result depends on sequence. PostgreSQL documents ordering within aggregate calls in its aggregate reference. Syntax and edge cases can differ across database engines and versions, so validate examples against the documentation for the system you use.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Windows Errors? Fix Them Before They SpreadFree repair scan
Crashes, No Sound, or Screen Glitches?Free driver scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.