Introduction
Every team is about to run the same experiment:
Someone types "@agent implement this" in Slack.
Ten minutes later, there's a pull request.
Not a prototype. Not a spike. A PR—shaped like a decision.
This is no longer hypothetical. GitHub's Copilot coding agent can be invoked from Slack; when you mention `@GitHub` in a thread, it can capture the thread as context and store that context in the resulting pull request. GitHub Docs
Anthropic is pushing the same direction with Claude Code inside Slack: tag the bot in a thread and it'll use the conversation as context to help execute coding tasks. The Verge
So if your organization "embraces AI," the question isn't _whether_ non-engineers will trigger code changes.
It's _when_, _how often_, and _what happens next_.
Because the economics of software change just flipped:
- Code is cheap.
- Pull requests are cheap.
- Novelty is cheap.
- Evidence is expensive.
- Coherence is expensive.
Managing AI development at scale is the craft of turning cheap output into reliable outcomes—without becoming a process bottleneck or a taste police department.
And the punchline is uncomfortable:
In the agent era, "good engineering" is less about how code looks
and more about how confidently you can change it again tomorrow.
The new failure mode: PR inflation
We used to be constrained by engineering throughput.
Now we're constrained by the ability to review, verify, release, and support changes at the pace they're created.
That's PR inflation:
- the number of PRs grows faster than the organization's ability to metabolize them
- review standards get inconsistent
- "green CI" becomes a fig leaf
- product surfaces start to drift
- support/documentation lags reality
- the most "precious" engineers become blockers (or quit quietly)
This doesn't feel like a tooling problem.
It feels like chaos.
But chaos is just the symptom. The disease is unpriced change.
When you can generate changes instantly, you stop paying for the _thinking_.
The candy store problem: velocity kills quality (especially with ADHD brains)
AI development is a candy store for builders.
You can ship anything:
- a tweak
- a micro-feature
- a new component
- a refactor
- a "small UX improvement"
- a "quick cleanup"
And if you have an ADHD personality (or founder brain), this is a trap:
- novelty feels like progress
- progress feels like shipping
- shipping becomes the goal
- testing becomes optional homework
- edge cases become "later"
- docs/support become "someone else's problem"
So you ship vibe-coded slop:
- a component change silently breaks 6 call sites
- a flow "works" but feels wrong
- error states are missing
- performance degrades in ways nobody notices
- the design system gets bypassed because the agent grabbed raw primitives
- regressions land, then sit in production until a user reports them
This is the core paradox:
AI increases _build speed_ faster than it increases _verification speed_.
If you don't deliberately upgrade verification, quality doesn't gradually decline.
It cliffs.
Quality has moved: stop worshipping internals
Here's the thing that will annoy the code aesthetes:
Internals matter less than they used to.
Not because internals don't matter at all—but because they're no longer where the leverage is.
In an agent-heavy world, your repo starts behaving like a compiler backend:
- humans and agents generate lots of code
- most of it is locally "fine"
- the organization wins or loses on whether the output is correct, coherent, performant, and safely released
So the new hierarchy is:
1. Input quality
Are we good at expressing intent? Do we provide constraints, examples, and context?
2. Output quality
Does it work? Does it work _well_? Is the UX coherent? Is it accessible? Is it fast?
3. Operational quality
Can we release safely? Detect regressions quickly? Roll back instantly?
4. Internal aesthetics
Important, but mostly solvable by canonicalization and automation.
OpenAI's own prompting guidance is basically a quality manifesto: you improve outputs by providing clear instructions, examples, and the context the model doesn't already have. OpenAI Platform
That's the shift:
- Quality is not a property of the code.
- Quality is a property of the feedback loops around the code.
If you're still spending most of your energy on style debates, you're managing the wrong layer.
"Tests pass, but UX breaks" is the default now
You called out a common pain:
tests pass, but UX breaks
That's not rare. It's the expected outcome of a test strategy that mostly validates:
- types
- units
- happy paths
- APIs
...and doesn't validate:
- visual layout regressions
- interaction regressions
- cross-component coupling
- "this feels broken" journeys
AI makes this worse because it's often locally competent and globally careless:
- it changes the component you asked for
- it doesn't understand every downstream consumer unless you force it to
- it optimizes for compile-time success, not product coherence
So "green CI" becomes false confidence.
Scaling agent output without scaling regressions requires one move:
Upgrade what counts as evidence.
Rigor is release empathy
There's another trap in AI + CI/CD:
If you can ship continuously, you will.
Engineers love this. It removes ceremony and makes change cheap.
Users don't experience commits. They experience change.
If your product shifts every few hours:
- users feel lost ("wait, where did that go?")
- docs become fiction
- support becomes the changelog
- PMs lose control of the narrative
- designers lose control of the surface area
This isn't "agile." It isn't "velocity." It's unpriced entropy.
Rigor is not about slowing down.
It's about being responsible with what you're speeding up.
Rigor means:
- impact analysis before implementation
- edge cases and error handling as first-class work
- tests that match risk (not vibes)
- intentional release strategy (merge != exposure)
- documentation and support readiness (because users exist)
The playbook: build an Evidence Loop (not a bureaucracy)
Most companies try to solve this with more process:
- more approvals
- more meetings
- more checklists
- more "please coordinate"
That works when change is scarce.
When change is abundant, gates turn into traffic jams.
The scalable model is: guardrails, not gates.
Your goal is simple:
Make rigor the path of least resistance.
I like framing this as the Evidence Loop:
1. Shape inputs
2. Constrain outputs
3. Verify behavior
4. Release intentionally
5. Detect regressions fast
6. Learn and codify
Let's make this concrete.
1) Shape inputs: treat intent like code
In an agent workflow, prompts _are_ requirements.
If you want quality, you need better "upstream artifacts":
- clearer intent
- clearer constraints
- clearer acceptance criteria
- clearer examples
- clearer non-goals
This is exactly what your Context-Driven Development essay was pointing at—LLMs can't infer tacit knowledge; they need explicit, structured context. OpenAI Platform
Practical moves
- Spec stubs for user-visible changes
- not a PRD, a forcing function (1–2 pages max)
- "what/why", success criteria, non-goals, screenshots/wireframes, rollout plan, support/docs impact
- Repo-level agent instructions
- you already have these—good
- now treat them like production code: version, review, and keep them sharp
- Golden paths
- example implementations for the patterns you want repeated
- "copy this, don't invent"
- Design system catalog that agents can actually use
- if the DS is "in people," agents will route around it
- give them examples, not philosophy
Rule of thumb:
If a human would ask a follow-up question, your prompt/spec isn't done.
2) Constrain outputs: canonicalize style and stop litigating it
If you want to prevent "precious devs" from becoming blockers while keeping consistency:
- one formatter
- one lint profile
- autofix on save + in CI
- delete rules that cause agent thrash (fix -> break -> fix)
- convert "style" into compilation
This isn't lowering standards. It's reallocating attention:
- stop spending review energy on aesthetics
- spend it on behavior, UX, and risk
In the agent era, style debates are a tax.
Pay it once. Automate it forever.
3) Verify behavior: evidence-based merges
The core shift: no more vibes-based merges.
Every PR should answer:
- What changed?
- What could this break?
- What evidence proves it doesn't?
Make that unavoidable with:
- a PR template agents fill by default
- required CI checks matched to risk
Risk tiers (example)
- Tier 0: Refactor / no behavior change
Evidence: typecheck + unit suite
- Tier 1: Small UI change (local)
Evidence: screenshots + story/interaction coverage
- Tier 2: Shared components / design system changes
Evidence: visual regression across stories + consumer checks
- Tier 3: New user journey / feature
Evidence: end-to-end journey test + error states + analytics hooks
- Tier 4: Critical flows (auth, payments, onboarding)
Evidence: staged rollout + synthetic monitoring + manual QA sign-off
This is how you bake rigor into culture without turning PM/eng leads into approval clerks.
"But UX is hard to test"
Yep. That doesn't mean you don't try.
Visual testing is one of the most leverageful upgrades you can make for frontend regressions:
- Storybook's visual testing docs explicitly describe PR checks you can require to prevent accidental UI bugs from being merged. Storybook
- Tools like Chromatic position this as "catch visual and functional bugs automatically" across browsers/viewports/themes. Chromatic
You're not trying to test everything.
You're trying to protect:
- shared components
- critical journeys
- the places regressions are expensive
4) Release intentionally: merge != expose
Feature flags stop being "nice to have" once PRs are cheap.
They're how you separate:
- engineering velocity
- from user experience stability
Martin Fowler describes "release toggles" as a way to separate feature release from code deployment in continuous delivery. martinfowler.com
That's not theory. That's the basic mechanism that lets you ship code today and decide who sees it tomorrow.
What this solves in an agent world
- PM coherence without blocking engineering
- merge is cheap
- exposure is controlled
- Docs/support alignment
- you can time user-visible change to when docs/support are ready
- Blast radius control
- staged rollouts
- quick kill-switch on regressions
The cultural shift here is subtle but important:
"Continuous deployment" is an engineering choice.
"Continuous change" is a product choice.
Treat them differently.
5) Detect regressions fast: the system should scream before users do
Your release pain is regressions that linger.
That's not an engineering failure. It's an operational failure:
- detection is too slow
- signals are too weak
- rollback isn't automatic
If you can ship in minutes, you need to detect in minutes.
Practical moves
- Synthetic monitoring for critical journeys
- log in, create X, search Y, checkout Z
- run on a schedule, alert on failure
- Error/latency budgets per surface
- if error rate spikes, rollback or disable flag
- Ownership mapping
- every alert routes to a human team (or it routes to nowhere)
In the agent era, reliability is a product feature.
Treat it like one.
6) Learn and codify: every regression becomes repo context
Agent workflows produce the same kind of failure repeatedly:
- missed edge cases
- incorrect assumptions
- design system drift
- "works on my screen" regressions
Every time you fix one, you have a choice:
1. Patch it and move on (fast now, slow forever)
2. Turn it into a durable artifact (slow now, fast forever)
Durable artifacts are:
- a new test
- a new golden path example
- a clarified repo instruction
- a PR template improvement
- a spec stub requirement
- a lint rule that enforces a constraint
This is how you stop "beating patterns into agents" via comments.
You don't teach the agent in the PR.
You teach the system that produces the agent's context.
Product coherence: infinite contributors need sponsorship, not permission
When anyone can create changes, PMs get squeezed:
- block changes -> become the speed bump
- allow changes -> product becomes a junk drawer
The fix is not "more control." It's _better default ownership_.
Sponsorship model
- every user-visible PR must have a product sponsor
- not necessarily a PM; could be a designer, eng lead, domain owner
- every user-visible PR must link a spec stub
- intent, success criteria, non-goals, rollout plan, support/docs notes
This does two things:
1. it prevents "cool but out-of-band" features from drifting into production
2. it gives support/docs a place to anchor reality
Coherence is not a vibe.
It's a maintained artifact.
Design and support: stop absorbing entropy
AI development doesn't just stress engineering.
It stress-tests the entire org.
Designers
In an agent world, designers become stewards of constraints:
- tokens
- component contracts
- interaction invariants
- "this flow cannot change without review"
If the design system isn't being used reliably, assume the real problem is:
- it's not discoverable
- it's not enforceable
- it doesn't cover real product needs
If the DS can't express the product, people will route around it—human or agent.
Support
Support becomes the first sensor for incoherent change.
If support is getting "what changed?" tickets, treat that as:
- missing release notes
- missing docs
- too-frequent surface changes
- unclear ownership
Support shouldn't just absorb chaos.
Support should feed the Evidence Loop.
Who owns this? (The missing role: AgentOps / Quality Ops)
At small scale, you can do this informally.
At scale, you need an owner for:
- repo instructions
- golden paths
- PR templates / evidence tiers
- CI requirements
- visual/journey testing strategy
- rollout defaults
- regression detection + response playbooks
Call it AgentOps, Quality Ops, whatever.
But it's real work.
If nobody owns the loop, you get:
- inconsistent quality expectations
- "precious dev" gatekeeping
- PM resentment
- support exhaustion
- and a codebase that feels haunted
Conclusion
AI didn't eliminate engineering discipline.
It exposed which parts of engineering discipline were just taste.
If you want to manage AI development at scale:
- stop pretending style debates are quality
- treat intent as a first-class artifact
- require evidence that matches risk
- separate merge from exposure
- detect regressions before users do
- and turn failures into durable context
Because the agent era doesn't reward the teams that write the most code.
It rewards the teams that can change the system daily without fear.
Appendix: copy-paste templates
A. Spec stub (user-visible change)
B. PR template (evidence-based)
C. Definition of Done (agentic work)