Why AI Coding Agents Fail in Data Engineering Without Upfront Specs

Your coding agent can ship a dbt model in five minutes. The same model that used to take a day. If it can't, we should talk.

And while it is easy to celebrate the speed, we also have to be talking about the impacts when the model is wrong.

AI coding agents have made poor planning more expensive, not less. The blast radius of a vague spec compounds faster than ever. And most organizations are doubling down on the wrong side of the equation.

The industry is obsessed with "vibe coding," prompt tricks, and demos of agents writing entire pipelines from a one-sentence description. That makes for great conference talks. It makes for terrible production infrastructure.

Why Faster AI Code Generation Increases Rework and Liability

A coding agent going full speed on the wrong thing is not a productivity gain. It is a liability generator.

Think of it this way: a consultant writing bad code by hand produces one broken model per day. An agent writing bad code produces a dozen before lunch. The failure mode is not slower. It is faster, wider, and harder to catch.

We are hearing this pattern from engineering leaders repeatedly. One VP of Engineering described it plainly: code review cycles are getting longer, not shorter. PRs ship faster, but they bounce back at a higher rate. The net throughput improvement is marginal, and the frustration is real. Senior engineers are saying "AI code doesn't work" because the code they review has no relationship to the business problem it was supposed to solve.

The code works syntactically. It passes linting. It even runs. But it answers the wrong question, models the wrong grain, or makes assumptions about business logic that nobody validated. That is not a tooling problem. That is a planning problem.

Where AI Agents Hallucinate Business Logic in Data Pipelines

Every data engineering project starts with a compressed idea. A stakeholder says, "I just want a dashboard." A product manager writes, "We need churn metrics." A CFO asks, "Why don't our numbers match?"

Between that compressed idea and production code, there is a gap. That gap used to be filled by a senior engineer spending two days thinking, asking questions, sketching schemas, and writing code that reflected their understanding of the business context. Slow, yes. But the thinking and the building happened together.

Agents separate those two activities completely. The building is instant. The thinking is not. And if you do not fill that gap deliberately with structured planning, the agent will fill it for you. It will hallucinate business logic. It will invent grain assumptions. It will make choices about edge cases that nobody discussed because nobody wrote them down.

This is what we call the "tactical expansion" problem.

The distance between a stakeholder's one-sentence request and correct, production-ready code is enormous. Agents do not shrink that distance. They just cross it faster, often in the wrong direction.

What Spec Coding Is and How It Differs From Vibe Coding

At Mammoth Growth, we invested heavily in solving this problem because our entire operating model depends on it. We run 65+ custom Claude Code skills daily. Our agents produce 95% of all text output across the business, from code to documentation to analysis artifacts. We are not skeptical about agents. We are all in. And that is exactly why we treat planning as the highest-leverage activity in every engagement.

We call our approach "spec coding." The premise is simple: an agent with a detailed specification produces better code faster than an agent with a vague prompt, every single time. No exceptions.

Here is the workflow. A senior consultant spends one to two hours building two documents before the agent writes a single line of code:

The Business Spec. What question are we answering? What is the grain? What are the edge cases? What does "correct" look like, stated in business terms a stakeholder can validate? This is not a requirements doc that gathers dust. It is the agent's operating contract.

The Technical Spec. What sources feed this model? What is the join logic? What transformations apply? What tests confirm correctness? This is not architecture documentation written after the fact. It is the blueprint the agent executes against.

With those two documents in hand, the agent produces code in minutes. The review process becomes: "Does this match my original specification?" That is a tractable question. Without those documents, the review process becomes: "Did the agent hallucinate anything?" That question is nearly impossible to answer quickly, especially at scale.

Why Planning-First Teams See Durable 4x Productivity Gains From AI Agents

Teams that skip planning and go straight to prompting see initial speed gains that decay within weeks. The pattern is predictable: fast first draft, slow review, rework, another fast draft, another slow review. The cycle compresses calendar time on the front end and expands it on the back end. Net improvement: marginal.

Teams that reinvest saved engineering time into planning quality see a different curve entirely. The spec takes an hour or two. The agent output is right the first time, or close to it. The review is fast because the reviewer is comparing output to a known standard, not reverse-engineering intent from generated code. Rework drops. Cycle time drops. Quality goes up.

The math is straightforward. If an agent compresses a day of coding into five minutes, you did not save a day. You freed a day. The question is where you reinvest it. Most teams reinvest it in more prompting, more agent runs, more volume. We reinvest it in more planning, more specification, more alignment with stakeholders. The compounding effects of those two choices diverge fast.

Why Standardized Spec Templates Compound AI Agent Output Quality

One specification is useful. A hundred specifications that follow the same structure are transformative.

We enforce absolute repeatability in our technical specs. Every spec follows the same template. Every section answers the same questions. Every agent consumes the same format.

This is not bureaucracy. It is the foundation for compounding quality at scale.

• When your specs are consistent, your agents produce consistent output.
• When your agents produce consistent output, your review process accelerates because reviewers know exactly where to look.
• When your review process accelerates, your cycle time drops.
• When your cycle time drops, you can take on the next business question instead of relitigating the last one.

This is the flywheel that most teams miss. They optimize the agent. They should optimize the input to the agent.

Five Actions VPs of Engineering Should Take to Fix AI Code Quality

If you are a VP of Engineering or Head of Data watching code quality decline despite faster PR velocity, the fix is not a better model or a more sophisticated prompt chain. The fix is upstream.

• Audit your rework rate. Track how many PRs bounce back from code review on the first pass. If that number is climbing, your planning process is the bottleneck, not your agent configuration.
• Mandate specs before prompts. No agent run without a business spec and a technical spec. Your senior engineers may push back on this. They will say it slows them down. But it only feels slow for about two weeks. Then it feels like the only sane way to work.
• Standardize your spec templates. One format. Every project. Every engineer. Consistency in the input produces consistency in the output. This also makes it possible for the specs themselves to become agent-readable context in future iterations.
• Reinvest time savings into planning, not volume. This is where leadership discipline matters. When an agent saves your team eight hours on implementation, the instinct is to ship eight more features. But if those four of those hours are spent writing better specs for the next four features. You will ship more total value and carry less technical debt.
• Measure cycle time, not velocity. PRs merged per week is a vanity metric if half of them require rework. Measure time from business question to validated, production answer. That is the number that matters to your stakeholders, and it is the number that planning improves.

Why AI Coding Demos Do Not Translate to Production Data Engineering

The AI coding discourse is dominated by people who have never shipped production data infrastructure. They show demos. They write threads. They build toy projects in sandboxes.

Production data engineering is different. The edge cases are real. The business logic is messy. The cost of being wrong shows up in board decks, financial reports, and customer-facing dashboards. Getting it wrong faster does not help.

Agents are the most significant force multiplier we have seen in a decade of building data infrastructure. We are not arguing against agents. We are arguing for treating them like what they are: execution engines that amplify the quality of whatever you feed them.

Feed them vague prompts, and they amplify ambiguity. Feed them rigorous specifications, and they amplify precision.

Planning is not overhead. Planning is the new highest-leverage work in data engineering. The teams that figure this out first will build better, ship faster, and compound their advantage every week. The teams that do not will keep wondering why their 10x coding agent is producing 0.5x outcomes.