51 pages, 32 endnotes and a closing line that went around the industry: “generation is solved.” The paper gets the diagnosis right and fails at exactly the thing it preaches. Here’s the full review, with a red pen over every figure.
In June 2026 Google published The New SDLC With Vibe Coding: From ad-hoc prompting to Agentic Engineering, a 51-page whitepaper by Addy Osmani — a Director at Google Cloud AI — Shubham Saboo and Sokratis Kartakis. The document is dated May, but it went public the week of June 15 as Day 1 material for Kaggle’s free 5-Day AI Agents Intensive. The previous edition of that course drew over 1.5 million learners, so we’re talking about a paper with distribution most academic work can only dream about.
It isn’t a product or a repository: it’s a framework document. And its thesis fits in the last line of the body text, right before the endnotes:
“Generation is solved. Verification, judgment, and direction are the new craft.”
I read the whole thing and broke it into layers, because their reuse value differs sharply and conflating them is the most expensive reading error available. Verdict up front: the conceptual framework is among the best writing on this subject, and the statistical apparatus collapses at the first question. The irony is hard to miss — a document proclaiming verification as the new discipline never applies that standard to its own numbers.
Layer 1 — The conceptual framework
This is where the real value sits, and the adoption cost is zero.
The spectrum: verification as the only variable
The paper’s central move is to refuse the binary. Vibe coding and agentic engineering aren’t two tribes: they’re two ends of a single spectrum. And what fixes your position on it isn’t the tooling — it can be identical at both ends — but how much structure, verification and human judgment surround the model’s output.
The same developer, with the same agent and the same model, can sit at both ends in the same afternoon. What changes is the scaffolding. The text is explicit: without tests and evals, the practice is still vibe coding, no matter how sophisticated your prompts.
It’s a clean, non-trivial distinction, and it isn’t original to the paper: an independent academic taxonomy — Sapkota, Roumeliotis and Karkee, May 2025 — drew essentially the same map a year earlier. Two unrelated pieces of work converging is itself a signal that the distinction captures something real.
Agent = Model + Harness
This is the contribution with the most operational leverage, and the one I’d take into a meeting tomorrow.
The dominant intuition — that the model is the system — is false, and it sends investment to the wrong place. The model is one input. Everything else is the harness: instruction files (AGENTS.md, CLAUDE.md, GEMINI.md), tools and MCP servers, sandboxes, orchestration logic, deterministic hooks and observability. The split the paper proposes is roughly 10% model / 90% harness.
The corollary is the single most useful sentence in the text for anyone running a team: most agent failures, examined honestly, are configuration failures. A missing tool, a vague rule, an absent guardrail, a context window stuffed with noise.
And here there’s an actual receipt. LangChain published an experiment in February 2026 where they touched only the harness, holding the model fixed (gpt-5.2-codex): on Terminal-Bench 2.0 they went from 52.8% to 66.5% — 13.7 points — and from just outside the Top 30 into the Top 5. It’s one measurement, on one benchmark, with one agent. But it’s a real measurement, with its methodology on the table.
Static context vs dynamic context
The paper enumerates six context types — instructions, knowledge, memory, examples, tools and guardrails — and draws a boundary that is, in practice, the most consequential architectural decision in agent design: what goes into static context (always loaded, paying tokens on every interaction) and what goes into dynamic context (on demand, paying only when needed).
The concrete ask is that this boundary be treated as first-class configuration: reviewed in a pull request and versioned like code. The pattern that makes dynamic context scale is skills with progressive disclosure — the agent sees lightweight metadata at startup, loads full instructions when a task matches, and pulls heavy material only when it genuinely needs it. The paper credits their fast adoption to solving four problems at once: context rot from overloaded prompts, the absence of procedural memory in LLMs, the operational overhead of multi-agent architectures, and the need for portability across tools.
The factory model
The synthesis of all of the above: the developer’s primary deliverable stops being code and becomes the system that produces code. Specifications and context, agents that translate them into implementation, tests and quality gates, feedback loops that route failures back to the agent, guardrails that bound behavior. A factory manager doesn’t assemble each widget: they design the line and own quality control.
Layer verdict: ADOPT. Shared vocabulary, defensible criteria in front of non-technical stakeholders, zero dependency on anyone.
Layer 2 — The SDLC, phase by phase
The paper keeps the six classic phases and redistributes time inside them. The key observation is that compression is dramatic but uneven: implementation goes from weeks to hours while requirements, architecture and verification stay human-paced. The result isn’t a faster SDLC; it’s a different workflow, with blurred boundaries between phases.
Two ideas from this chapter are especially worth keeping.
The two operating modes. The conductor directs in real time inside the IDE — complex logic, hard debugging, unfamiliar codebases — and their dominant skill is fine-grained understanding of the change; their risk is becoming the bottleneck themselves. The orchestrator delegates asynchronously across several agents — known bugs, features on established patterns, migrations, test generation — and their dominant skill is specifying, decomposing and evaluating; their risk is losing comprehension if review discipline slips. These aren’t two kinds of person: they’re two gears of the same developer.
The 80% problem. Agents rapidly generate roughly 80% of a feature’s code, and the remaining 20% — edge cases, error handling, integration points, subtle correctness requirements — demands deep contextual knowledge. What matters isn’t the split, it’s that the nature of errors has changed: from syntax mistakes to conceptual failures (wrong assumptions about business logic, not asking for clarification on ambiguity, architectural decisions that leave debt). And those are harder to catch precisely because the code looks right and passes the basic tests.
Layer 3 — Economics
For anyone accountable for delivery this is the most actionable chapter. It’s also the most weakly evidenced.
The argument recasts the spectrum as a cost structure. Vibe coding is low CapEx, high OpEx: essentially zero barrier to entry, then token burn in fix-it-again loops, a maintenance tax on structurally inconsistent code, and security remediation after the fact. Agentic engineering inverts the equation — high CapEx, low OpEx: you pay upfront in specs, tests and context design, and marginal cost per feature collapses. There’s a crossover point.
Two concrete levers that survive scrutiny: context engineering as a financial strategy (passing a 100,000-token repository into every prompt doesn’t scale; a dense payload raises first-pass success and avoids expensive loops) and intelligent model routing (frontier models for requirements, architecture and initial implementation; small, cheap models for test generation, review and CI/CD monitoring).
What doesn’t survive is the number. The paper claims that past the crossover point, vibe coding costs 3 to 10 times more per feature. I went looking for the measurement behind it. There isn’t one: everything available is consultancy blog posts selling remediation services, with no methodology and no data. It’s an illustrative figure that turned into a number along the way. Useful as a mental model, unacceptable in a commercial proposal.
Layer verdict: ADOPT the model, WATCH the figure.
Layer 4 — The evidentiary apparatus, with a red pen
This is where the paper falls over, and where this article gets uncomfortable. I traced every headline statistic to its primary source. Here’s what survived.
| Claim in the paper | What I found tracing it |
|---|---|
| 85% of developers use coding agents | Reword. The real 85% comes from JetBrains, State of Developer Ecosystem 2025 (24,534 developers, 194 countries) and says “AI tools”, not agents. JetBrains’ closest agent figure is 62% |
| 51% daily use | Holds, with a caveat. Stack Overflow 2025 (49,009 responses, 177 countries) reports 47.1% daily use overall and 50.6% among professional developers. The 51% is that 50.6% rounded up |
| 90% adoption | Holds. DORA 2025, nearly 5,000 professionals: 90% use AI at work, up 14 points year over year. Note: AI use at work, not penetration into the development lifecycle |
| 41% of new code is AI-generated | No primary. No locatable source; it’s a figure that circulates by citing itself. What is attributable: Google ~25% (Pichai, 2024), Microsoft 20-30% (Nadella, 2025) |
| 25-39% productivity improvements | Misread. It comes from the Copilot field experiments (MIT/Princeton/UPenn, 4,867 developers). The headline result is 26.08% more tasks completed; the 27-39% applies only to junior developers, while seniors see 7-16%. The paper itself warns that this heterogeneity isn’t significant at conventional levels |
| Deloitte 30-35% projection | Holds as what it is: an expectation, not a measurement |
| METR: experts 19% slower | Real. July 2025, randomized controlled trial, 16 expert maintainers, 246 issues, mature repos and early-2025 tooling |
| Top 30 → Top 5 and +13.7 points from harness tuning | Double-counted. Presented as two results from two teams. It’s one experiment: LangChain, February 2026, same fixed model |
| 3-10x cost past the crossover | Illustrative. No associated measurement |
| The 10/90 split | A heuristic with an author, not a datum. Uncited in the paper itself |
Two things strike me as worse than any single row above.
First: the paper cites the METR study — experts 19% slower — and never reconciles it with its own 25-39% improvement figure. Two incompatible claims in the same text, a few pages apart. And the missing nuance is delicious: METR changed the experiment design in February 2026 because it can no longer get a control group. Between 30% and 50% of developers avoided submitting tasks when their turn came to work without AI. The experiment didn’t break on the data: it broke because people refuse to go back.
Second: the paper argues generation is solved, mentions security remediation and slopsquatting in endnotes, and never reconciles the evidence on vulnerabilities in generated code with that thesis. If the hard 20% includes “subtle correctness gaps that pass the tests”, generation isn’t solved: it’s displaced.
Layer verdict: WATCH. Don’t quote any of these figures in a proposal, an RFP or commercial material without substituting the primary source. Where no primary exists — 41%, 3-10x — simply don’t use it.
Layer 5 — Coupling and sovereignty filters
These need separating precisely, because the risk profile is opposite by layer.
- Conceptual framework and governance recommendations: zero coupling. Reusable without restriction, vendor-agnostic, applicable as-is on any stack.
- Interoperability standards: low coupling. The paper explicitly recommends adopting MCP and A2A to keep the option of mixing vendors. Watch a detail almost everyone conflates: MCP was donated by Anthropic to the Agentic AI Foundation (under the Linux Foundation) in December 2025, while A2A has been an independent Linux Foundation project since June 2025. Both open, different governance. And
AGENTS.mdis also under AAIF stewardship, with over 60,000 projects using it. - Reference implementation: high coupling at the deployment stage.
google/agents-cliinstalls seven skills into your coding assistant — workflow, ADK code, scaffolding, evals, deployment, publishing and observability — and also runs standalone. Locally an AI Studio key is enough for scaffold, run and eval; deployment and cloud features require Google Cloud. ADK is Apache 2.0 and containerizable on your own infrastructure, which is more than almost anyone else offers. - Managed deployment chain: this is the boundary. Agent Runtime — a component of Gemini Enterprise Agent Platform, the brand that replaced Vertex AI at Cloud Next ’26 — is a managed service with no self-hosted equivalent. An important caveat so as not to overstate it: you can package your ADK agent and run it on GKE, Cloud Run or any container runtime. The hard dependency isn’t in executing the agent, it’s in the adjacent managed pieces — sessions, memory — and in the path the paper presents as natural: a local prototype that “graduates” to Agent Runtime, i.e. platform lock-in exactly at the production boundary.
For regulated, on-premise or data-residency-constrained environments the recommendation is the usual one: keep the patterns — the lifecycle phases, trajectory-scored evalsets, observability from day one, per-agent scoped permissions — and reimplement them on your own orchestration. The value is in the ideas; the reference code is adoptable only where coupling to US cloud infrastructure is acceptable.
The fine print
Beyond the numbers, three things worth knowing before you cite the paper.
It ages at different speeds. The authors admit this with unusual honesty: the phase-by-phase snapshot reflects mid-2026 and the boundaries may look different in twelve months. Hold the framework, not the diagram. A concrete example: Terminal-Bench 2.0, the benchmark behind the LangChain result, is already at 2.1 with a version 3 in circulation.
Vendor neutrality is partial. The text names third-party tools generously — Copilot, Cursor, Windsurf, Claude Code, Jules — but the only concrete implementation path it offers is ADK + Agents CLI + Agent Runtime on Google infrastructure. The institutional bias is disclosed, not hidden. But it’s there.
Be careful with second-hand chronology. Tracing the concept’s dates, I found secondary coverage repeats them wrong quite often. For the record: Karpathy coined “vibe coding” on February 2, 2025, and the “agentic engineering” formulation comes from his retrospective of February 4, 2026 — the anniversary of the original term — not April. If you’re building a timeline, go to the original posts.
Verdict by layer
| Layer | Verdict | Action |
|---|---|---|
| Conceptual framework (spectrum, harness, factory, conductor/orchestrator) | ADOPT | Fold the vocabulary into internal documentation and client conversations |
| Governance recommendations (evals as gate, versioned harness, adapted review) | ADOPT | Start with the written prototype-vs-production policy |
| Static/dynamic context engineering | PILOT | Measure tokens per task before and after on a pilot repository |
| Quantitative apparatus (85/51/41%, 25-39%, 10/90, 3-10x) | WATCH | Don’t quote without substituting the primary. Where there is no primary, don’t use it |
| Reference implementation (Agents CLI + ADK) | PILOT | The seven-skill pattern and trajectory-scored evalsets are worth evaluating |
| Managed deployment chain (Agent Runtime) | DROP | For accounts with sovereignty, isolation or data-residency requirements. No self-hosted equivalent |
What I’d do on Monday
Four things, in order of immediate return.
- Write the working-mode policy. Which projects, branches and environments permit vibe coding and which demand agentic engineering. The paper warns that teams leaving this boundary blurry produce prototypes that ship by accident. It’s the best effort-to-impact recommendation in the whole text.
- Treat the harness as code.
AGENTS.md, system prompts, eval suites and skill libraries: reviewed in pull requests, versioned with the project, owned by named engineers. Without that discipline the harness drifts and agent behavior stops being reproducible across the team. - Move the quality gate from demo to evals. Eval coverage with explicit rubrics — task success, tool-use quality, trajectory compliance, hallucination — as a precondition for any agent entering a shared workflow. The same way test coverage gates a service deployment.
- Redesign the code review checklist. Tuned to the failure modes of generated code: hallucinated dependencies, inadequate error handling, subtle correctness gaps that pass basic tests.
What I take away
This is a framework document with a statistics garnish. The framework is built to last; the figures are the part to trace to primary sources before using them anywhere someone might audit them. One hour of reading and a red pen.
And the central diagnosis is right, which is what matters: the bottleneck has moved from writing to specifying and verifying. What the paper doesn’t do is apply to itself the standard it preaches. Both facts are useful information, and the second maybe more than the first: if the industry’s reference text on why verification is the new craft doesn’t verify its own numbers, the conclusion isn’t that the framework is bad. It’s that the discipline it proposes isn’t yet a habit — not even among the people writing about it.
Ideas over code; evidence over hype.
Sources
- Osmani, A., Saboo, S. & Kartakis, S. (May 2026). The New SDLC With Vibe Coding: From ad-hoc prompting to Agentic Engineering — Google, 51 pp. (primary document)
- Osmani, A. (Jun 16, 2026). The New Software Lifecycle — companion essay, republished on O’Reilly Radar on July 15, 2026
- Sumi, A. (Jul 26, 2026). The new SDLC: what the Google paper actually says — independent fact-check of the figures
- LangChain (Feb 17, 2026). Improving Deep Agents with harness engineering — Terminal-Bench 2.0, 52.8% → 66.5% with the model held fixed
- METR (Jul 10, 2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity and the February 24, 2026 update
- Stack Overflow Developer Survey 2025 (49,009 responses) · Google Cloud DORA 2025 (nearly 5,000) · JetBrains State of Developer Ecosystem 2025 (24,534)
- Peng, Kalliamvakou, Cihon & Demirer. The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers
- Sapkota, R., Roumeliotis, K. & Karkee, M. (May 26, 2025). Vibe Coding vs. Agentic Coding — arXiv:2505.19443
- Google. Agents CLI · Agent Development Kit · Gemini Enterprise Agent Platform
- Model Context Protocol joins the Agentic AI Foundation · Google donates A2A to the Linux Foundation · AGENTS.md
- Karpathy, A. — “vibe coding”, Feb 2, 2025 and “agentic engineering”, Feb 2026
Leave a Reply