All articles

Engineering

AI Does Not Fix Weak Engineering Systems. It Amplifies Them.

AI coding adoption is not a tool rollout. It is an operating-model change.

By Brandon W. Lee · Published · Updated

Most companies are still talking about AI coding adoption as if it were a purchasing decision.

Pick a tool. Buy seats. Announce guidelines. Encourage experimentation. Measure usage. Hope productivity shows up in the roadmap.

That frame is too small.

AI coding tools are no longer just autocomplete. They are moving into issue triage, code generation, test writing, documentation, refactoring, pull request creation, and review support. GitHub's coding agent is a clear signal: agents are being embedded directly into the work system, not kept in a side window. Assign an issue, let the agent create an environment, make changes, run checks, and produce a pull request. That is not a better text editor. That is a change to the engineering operating model.

The question for engineering leaders is not "Which AI tool should we roll out?"

The better question is: "What kind of engineering system will this tool amplify?"

DORA's 2025 AI-assisted software development research makes the point directly: AI acts as an amplifier of existing organizational strengths and weaknesses. That should change how leaders think about adoption. AI does not automatically convert a weak delivery system into a strong one. It increases the throughput of whatever system already exists.

If the system has clear ownership, good tests, fast review, small batches, usable internal documentation, thoughtful platform support, and strong feedback loops, AI can accelerate real delivery.

If the system has brittle tests, vague requirements, slow review, unclear ownership, hidden tribal knowledge, overloaded maintainers, and weak release discipline, AI can accelerate confusion.

This is why the next phase of AI adoption belongs less to tool selection and more to AI Systems Engineering: turning AI tools, agents, and models into reliable, repeatable engineering systems.

The Failure Mode Is Not That AI Writes Bad Code

AI can write bad code. So can humans.

But the larger failure mode is structural: AI makes it easier to generate more work than the system can safely absorb.

That matters because software delivery is constrained by flow, feedback, and risk, not typing speed. If developers produce code faster but the review system stays the same, batch sizes grow. If batch sizes grow but tests stay weak, more defects reach integration. If agents touch parts of the system they do not understand, maintainers inherit work they did not shape. If teams accept generated code without better verification, "done" becomes a claim instead of evidence.

DORA's generative AI report found the same tension. AI adoption can improve individual experience and perceived productivity, but it can also hurt delivery throughput and stability when teams do not reinforce fundamentals like automated testing, continuous integration, and fast review. The lesson is that AI changes the load placed on the delivery system.

An engineering organization is not a collection of people typing code. It is a system for turning intent into reliable production change.

AI changes several parts of that system at once:

  • The cost of generating code drops.
  • The volume of proposed change rises.
  • The boundary between design, implementation, and review blurs.
  • The amount of code reviewers must inspect can increase.
  • The need for high-quality context becomes more important.
  • The need for explicit permissions becomes more urgent.
  • The cost of weak verification compounds faster.

When leaders treat AI as a tool rollout, they optimize for access and usage. When they treat AI as an operating-model change, they optimize for delivery outcomes.

Context Architecture Becomes Infrastructure

Most teams underestimate how much AI performance depends on context.

Developers carry implicit context: why a service exists, where the dangerous code paths are, which tests are meaningful, which migration patterns are safe, which product constraints matter, and which parts of the codebase are legacy for a reason. Agents do not inherit that context unless the system gives it to them.

The naive answer is to stuff more context into prompts. That does not scale.

Teams need context architecture: a repository-native system that tells agents what to read, what to ignore, how work moves through stages, and what evidence proves completion.

This is not glamorous work, but it is leverage. A well-structured `AGENTS.md`, `CONTEXT.md`, or equivalent context map can keep agents from wandering through the entire repository. Stage-specific instructions can distinguish planning from implementation, review, verification, and release. Reference files can capture coding standards, architecture boundaries, domain language, security rules, and operational constraints. Working artifacts can preserve decisions across sessions.

The strategic point is simple: your context file is not a prompt. It is infrastructure.

For engineering leaders, this means AI adoption should include a context audit:

  • Which repositories have clear routing instructions for agents?
  • Which services have current architecture notes?
  • Which review rules are written down instead of stored in senior engineers' heads?
  • Which setup, test, and release commands are agent-readable?
  • Which areas require explicit human approval?
  • Which artifacts prove planning, implementation, review, and verification?

Weak context architecture produces expensive agent behavior: too much reading, missed constraints, repeated setup, vague plans, local drift, and unclear intent. Strong context architecture makes the work inspectable. It gives agents a route through the system and gives humans a visible surface to improve when outputs fail.

Permissions Are Part Of The Product

The more capable the agent, the more important the boundary around it.

Early AI coding assistants mostly suggested snippets inside a developer's active file. Modern agents can inspect repositories, create branches, run commands, call tools, open pull requests, and interact with development environments. That makes permissions a first-class design concern.

The Google DeepMind AI Control Roadmap is focused on frontier AI lab risk, but its systems lessons apply broadly: use defense in depth, access controls, environment hardening, monitoring, prevention, and response. Treat agent capability as something the surrounding system must govern.

For normal software teams, this does not mean panic. It means design.

An AI agent should not automatically inherit every permission the human has. A developer may have broad access because they are accountable, trained, and embedded in organizational norms. An agent is an execution system. It should receive the minimum access needed for the task, with actions logged, reviewable, and reversible.

Practical controls include:

  • Separate identities for agent activity where the platform supports it.
  • Branch and pull request workflows instead of direct pushes to protected branches.
  • Restricted secrets access by default.
  • Sandboxed or ephemeral development environments.
  • Clear rules for network access, dependency installation, and external data.
  • Human approval for production, security-sensitive, data-sensitive, or irreversible actions.
  • Logs that show what the agent read, changed, ran, and claimed.

The point is not to slow every workflow down. The point is to align permissions with risk.

AI agents tolerate friction differently than humans do. An agent can operate inside narrower boundaries if the workflow is designed well. That creates an opportunity: teams can make agent permissions more granular than human permissions without destroying developer experience.

This is where AI adoption becomes platform work. The platform must make the safe path the easy path: approved tools, standard environments, reusable workflows, visible logs, and policy-backed defaults.

Review Has To Change

If AI increases the amount of code entering pull requests, code review cannot remain a heroic human bottleneck.

The answer is not to remove human review. The answer is to redesign review as a layered system.

Human reviewers should spend less time discovering basic issues and more time judging architecture, product fit, risk, maintainability, and operational impact. Agents and automation should handle cheap, repetitive, evidence-based checks.

A useful AI-era review system has at least four layers:

  • Pre-change review: Is the plan coherent? Is the task scoped correctly? Is the agent reading the right context? Is this work safe for an agent to attempt?
  • Mechanical review: Do formatters, linters, type checks, dependency checks, and tests pass?
  • Semantic review: Does the implementation match the requirement? Did it preserve architecture boundaries? Did it change behavior intentionally?
  • Release review: Is the rollout safe? Are migrations, flags, monitoring, documentation, and rollback paths handled?

Most teams over-focus on the pull request because that is where review already lives. But agents need review before and after the diff. Before the diff, a bad plan implemented quickly is faster rework. After the diff, a clean-looking change without evidence is merely plausible.

Review also needs explicit ownership. If an agent opens a pull request, who owns the design, risk, scope, and incident explanation? AI can assist with work, but accountability cannot be delegated to a model.

Leaders should make this boring and explicit: every AI-assisted change has a human owner.

Verification Is The Completion Standard

The agent is not done when it says done.

The developer is not done when the code compiles.

The team is done when the relevant stage contract has evidence.

That evidence depends on the work. A UI change may need screenshots and accessibility checks. A backend change may need unit tests, integration tests, migration validation, and logs. A security-sensitive change may need threat-model notes, dependency review, and approval from the right owner.

AI adoption should force teams to make their definition of done more concrete.

A useful verification standard answers five questions:

  • What commands prove the change works?
  • What user or system behavior was checked?
  • What risks remain?
  • What evidence is attached to the pull request or artifact?
  • What would cause us to revert, revise, or escalate?

This is also where metrics matter. DORA's delivery metrics remain relevant because AI adoption should improve the system, not just individual activity. Track lead time, deployment frequency, failed deployment recovery time, change fail rate, and deployment rework rate at the service or application level. Add AI-specific leading indicators where they help: review queue time, batch size, test failure patterns, agent rework rate, human edit rate, and percentage of AI-assisted changes with verification evidence.

Avoid vanity metrics. Seat count, token volume, generated lines, and accepted suggestions can be useful operational signals, but they are not delivery outcomes. A team can use a lot of AI and still ship worse software.

The goal is not more AI usage. The goal is better software delivery.

Training Is Operating Practice, Not Tool Tips

Many AI rollouts fail because training is treated as onboarding material: here is the tool, here are the prompts, here are the acceptable-use rules.

That is not enough.

Developers need practice inside the actual engineering system. They need to learn when to use an agent, how to write task briefs, how to inspect plans, how to constrain scope, how to review generated code, how to verify outputs, and how to recover when the agent goes wrong.

Teams also need shared patterns. If every developer invents a private AI workflow, the organization gets local productivity experiments but no durable operating capability. The best teams will share systems: context maps, task templates, review checklists, verification commands, permission rules, and examples of good agent runs.

Training should include:

  • Task selection: which work is suitable for agents, assisted workflows, or human-only execution.
  • Context writing: how to package requirements, architecture notes, constraints, and examples.
  • Plan review: how to catch flawed assumptions before implementation starts.
  • Diff review: how to inspect AI-generated code without rubber-stamping it.
  • Verification: how to demand evidence and reproduce it.
  • Security and privacy: what data, secrets, systems, and actions are off limits.
  • Failure analysis: how to turn bad agent runs into better instructions, tests, and workflow gates.

This is an organizational learning loop. Every repeated human correction is a signal that the source system needs improvement. If reviewers keep telling agents the same thing, encode it in the context architecture. If generated changes keep becoming too large, adjust task templates and pull request policy.

AI adoption improves when the system learns from use.

Governance Should Enable Flow

AI governance cannot be only a policy PDF. It has to be embedded into the workflow: which tools are approved, which repositories are enabled, which data can be used, which actions require approval, which logs are retained, which review gates are required, and which metrics leadership will inspect.

Good governance gives teams confidence. Developers should not have to guess whether they are allowed to paste a stack trace, use an agent on customer-facing code, generate tests from production incidents, or let an agent install dependencies.

The executive stance should be practical:

  • We will use AI where it improves delivery outcomes.
  • We will not measure success by adoption theater.
  • We will protect customer data, secrets, and critical systems.
  • We will require human accountability for AI-assisted work.
  • We will strengthen tests, review, and verification as code generation gets faster.
  • We will improve the workflow when failures repeat.

That kind of governance enables speed because it removes ambiguity.

The Leadership Checklist

AI adoption should start with a systems review, not a vendor comparison.

Before expanding rollout, leadership should be able to answer:

  • Context: Do our repositories tell agents what matters?
  • Scope: Do we have task types and boundaries for AI-assisted work?
  • Permissions: Can agents operate with least privilege?
  • Review: Do we review plans, diffs, and release risk?
  • Verification: Do changes carry evidence, not just claims?
  • Metrics: Are we measuring delivery outcomes, not just usage?
  • Training: Are teams practicing shared workflows?
  • Governance: Are policies embedded into daily engineering work?
  • Ownership: Is a human accountable for every AI-assisted change?
  • Learning: Do failures improve the system, or do they just create cleanup work?

This checklist is not bureaucracy. It is how leaders protect the value of the rollout. AI coding can be a real advantage, but the gains are not automatic. They appear when the engineering system can absorb higher generation speed without losing control.

The teams that win will not be the teams with the most prompts, the most seats, or the most demos.

They will be the teams that turn AI into an operating capability: context architecture, bounded permissions, layered review, verification evidence, delivery metrics, workflow training, and governance embedded into engineering work.

That is the core shift. AI adoption is not about adding a smarter assistant to the same old process. It is about redesigning the process so smarter assistants can be useful, controlled, and tied to production outcomes.

AI does not fix weak engineering systems.

It amplifies them.

So build the system you want amplified.

keep going

This article covers one part of a larger system. The AI Systems Engineering Handbook is the whole operating model — fourteen chapters on context, stage contracts, validation, evaluation, cost, and governance.