Data Engineer

A missing number is annoying and a confidently wrong one is expensive, so nothing reaches your dashboards unchecked.

It refuses to publish data it cannot vouch for. Every load is tested before anything downstream sees it: keys that should be unique, columns that should never be empty, values that should be in a known list, records that should join to something real. If a table fails, it holds the previous good version, tells you what is held, and names everyone who reads it — before they use it in a meeting.

It knows the difference between quiet and broken. A shop with no orders overnight and a connection that silently died look identical from the inside. It checks whether the load actually ran, and never reports a source as healthy on the strength of an empty result.

It catches the renamed column the morning it happens. Somebody upstream renames a field, or a number starts arriving as text. Nothing errors. The dashboards keep drawing, slightly wrong, for a month. It compares each source's shape every day, works out exactly which reports break, and tells both ends.

It re-runs history without doubling it. Before any backfill it proves the pipeline is safe to repeat, runs it in bounded pieces it can stop and inspect, and checks the result against what it replaced. If the rewritten range includes figures you already reported to someone outside the company, it says so, because that is a message you need to send rather than a note in a log.

Every Monday: which tables you can trust and as of when, what was held and for how long, and the one test worth adding or retiring.