Data engineer job description
Data engineering adverts have converged on one shopping list of tools, and so have the resumes answering them. The list is close to useless as a filter, because every tool on it is documented, certifiable and installable in an afternoon. What actually breaks data hires is different. Nobody trusts the numbers, and the person who owns the pipelines is not the one who finds out first. Below is a description you can post, and how to screen for the difference.
The job description
We are looking for a data engineer who owns whether the numbers can be trusted. You will inherit pipelines as well as build them, and you will be the person who finds the problem before the business does.
What you will own
- Ingestion and transformation pipelines, including the ones you did not write
- Data quality checks, and what happens when one fails at an inconvenient hour
- The warehouse model, and the definitions that sit on top of it
- Coordination with the engineers whose changes break you, ideally before they make them
- Deprecation: finding out who depends on a table before it disappears
What the job needs
- Strong SQL and one general purpose language used in anger rather than in a tutorial
- Experience with an orchestration tool and an opinion about what belongs inside it
- Comfort with the human half: chasing a definition until somebody commits to it in writing
Signals we weight heavily
- You have been paged for a pipeline and changed something so it stopped paging
- You have deprecated a table and survived the aftermath
- You can say what an active user meant at your last company and who decided that
What a data engineer is actually accountable for
Pipelines are the visible artefact. The outcome is trust: the number in the report is right, it arrives before the meeting, and it means the same thing to two teams reading it in different rooms.
- Data that lands on time, and an alert that fires before a stakeholder notices it did not
- Definitions that hold across teams, so two dashboards stop disagreeing about the same word
- Knowing which tables are load bearing and which ones nobody has opened in a year
- Absorbing an upstream change made by a team with no particular reason to warn them
- Leaving pipelines somebody else can debug at seven in the morning
Four ways a data engineer hire looks right and is not
Every one of these passes a resume screen. Ladders eye-tracking study timed the average initial scan at just 7.4 seconds, which is long enough to check the four signals below and not long enough to check anything that would contradict them.
Looks right
Every tool in the modern data stack listed on the resume
That list is a shopping catalogue every resume can carry and every vendor certifies for. It records which tools were installed, not whether the person has been the one awake when a job failed silently and the morning report went out wrong anyway. Ask what has broken, not what has been used.
Looks right
Built a data platform from scratch
Greenfield builds are the most enjoyable and least representative work in this discipline. The usual job is inheriting pipelines with unclear ownership and business rules buried inside a query nobody remembers writing. Ask what they migrated, and what they kept running while migrating it.
Looks right
Very large scale on the resume, high event volumes
Scale problems and correctness problems are different disciplines. At most companies the failures are late-arriving data, an upstream schema change nobody announced, and two teams meaning different things by one word. An engineer who has only optimised for volume may have met none of them.
Looks right
Clean code and a solid software engineering background
Welcome, and not the binding constraint. Data engineering fails at the human contract more often than at the code: whether they will chase down the product engineer who renamed a field, and whether they will refuse to publish a table nobody can define. Neither behaviour is visible in a code sample.
What to ask for evidence of instead
Four questions, and what the answer actually tells you. Take these into your own process whether or not you ever talk to us.
- Tell me about the last time a stakeholder found a data problem before you did. What changed afterwards?
- What it tells you: Every data team has had this happen. The answer separates people who added a check from people who added an apology. A candidate who says it has never happened is usually not close enough to what the business actually reads.
- Describe an upstream schema change that broke you.
- What it tells you: The defining failure mode of the role. Listen for what they did the second time, which is either a contract, a test, or a relationship with the team making the changes. One of those three is the real answer.
- What have you deprecated, and how did you work out who was using it?
- What it tells you: The hardest unglamorous task in a warehouse. Anybody who has done it can describe the archaeology involved. Anybody who has not will underestimate what an undeprecated table costs you two years later.
- How do you decide what gets a quality check, and what happens when one fails?
- What it tells you: A check that alerts into a channel nobody reads is not a check. You want somebody who has thought about who acts on the failure, how quickly, and what the report downstream should do in the meantime.
How Continuity1 runs this funnel
The screening above is the job. These are the numbers it produces when a function owns it end to end, set against the published benchmarks for the same market.
- 1 in 3
- Shortlisted candidates you meet who become the hire
- Aligned engagements run nearer 1 in 2, distant ones nearer 1 in 10. The market takes about 180 applicants to make one hire, and that sifting lands on your team rather than ours.
- Continuity1 tracked engagements
- ~3
- Interviews your team sits in, per hire
- Ashby puts technical roles at 17.6 interviews per hire across the whole process, up 52% since 2021. The rest of that load sits with the function, not with you.
- Ashby talent-trends report
- 1 in 9
- Accepted offers that ghost before joining
- Indian employers report nearly 4 in 10 offers dropped. We lose 1 in 9.
- nasscom community
- 95%
- Offers that close inside your stated band
- 20 of the last 21. A flat fee earns nothing from an inflated offer; a percentage of CTC earns more.
- Continuity1 tracked engagements
Every brief becomes a success profile before sourcing starts, calibrated with the people who will manage the role. That calibration is the step most hiring skips, and it is why a shortlist either matches the job or matches the job advert.
You review a scored shortlist and make the calls. The filtering never lands on your calendar.
Questions teams ask
Data engineer or analytics engineer?
A data engineer moves and lands data reliably. An analytics engineer models it into the tables analysts query and owns the definitions on top. Small teams often need the second more urgently, particularly where the raw data already arrives and the actual problem is that nobody agrees what it means.
Should we hire a data engineer before a data analyst?
Only if the data is genuinely unusable. Hiring the pipeline first is common, and often produces excellent infrastructure serving questions nobody has asked yet. Where an analyst is already working around the data by hand every month, that hand work is the specification for the engineering hire.
Should the data engineer own warehouse cost?
Yes, if they own what runs. Compute is one of the few costs where a single engineering decision changes the bill immediately and visibly. When nobody owns it, finance discovers the number a quarter later and the engineer who could have prevented it never saw it.
How do we test for this in an interview?
Describe one of your real pipeline failures and ask what they would have done differently. It is cheap, hard to fake, and far more revealing than a modelling exercise, because the strong candidates immediately ask what the data was being used for downstream.
Related