agentic testing · web + iOS Simulator
Agentic testing that opens the app.
Scripted tests check the assertions you wrote. Agentic testing forms goals from your diff, walks the built app the way a person would, and has to prove every claim with evidence before it reaches the PR.
The release-pass card presents “no blockers” while simultaneously displaying “paywall · toast overlaps CTA” and a P2 finding. Although the report is advisory, this presentation makes the primary status ambiguous.
What “agentic” means here.
A run is a pipeline of specialized roles, and the report is assembled by deterministic code from what those roles produced. No single model narrates its own homework.
01 · Scope
The mapper reads your diff against your screen graph and turns it into goals: concrete things a user should still be able to do on the changed screens.
02 · Walk
A driver pursues each goal against the real build, one committed step at a time, seeing the accessibility tree plus screenshots and recording evidence before and after every action.
03 · Judge
Two judges read the evidence independently of the driver. One verifies claimed outcomes; the other compares rendered screenshots against what the app announces.
04 · Locate
Findings that survive judgment go to the code locator, which points at the source most likely responsible, or says honestly that no supportable location exists.
Why this matters if agents write your code.
The agent that wrote a change is the worst-placed judge of it: it knows what the code is supposed to do, so it sees what it expects. Greenroom keeps the roles apart. The driver that walks never grades. The verification judge receives no goal text, only the checkpoint record, so it cannot rubber-stamp intent. Anything a model asserted without evidence never reaches the report.
And when a walk cannot finish, it does not guess. Every goal ends in exactly one of four verdicts, and a run never quietly drops a goal or dresses up infrastructure trouble as app health.
completed
The goal was achieved and the evidence proves it: the specific checkpoint facts, not “the walk finished without an error”.
defect
The walk surfaced behavior that is wrong on its own terms, with a severity from P1 to P4, the supporting evidence, and a source pointer when one is supportable.
blocked
The goal could not be completed, and the handoff names exactly what stood in the way. A signup form that rejects every valid input is a finding, not a tool failure.
inconclusive
The walk found nothing wrong but lacked the authority to call it a pass. Without a state contract, a clean run caps here rather than implying coverage that never happened.
Findings without a spec.
Most of what an agentic pass finds needs nothing declared in advance. Each finding says where the authority to call it a defect comes from.
Tier 1 · Self-evident
Broken on any app, no context needed: crashes, dead controls, unreadable text, a screen that never finishes loading, keyboard traps.
Tier 2 · Internal contradiction
The app disagrees with itself. A summary that says 5 while its detail screen says 7; a total that is not the sum of its lines; a hero card that says “no blockers” beside a P2.
Tier 3 · Declared expectation
The app violates something you told Greenroom to hold it to: an oracle in your state contract, or a goal with a stated postcondition. This tier grows as your contract does.
In our defect library, 78% of real defects are detectable at tiers 1 and 2, with nothing declared in advance. Severity is orthogonal: tier says who authorized the expectation, P1 through P4 says how much it hurts.
Questions about agentic testing
Is this just an LLM clicking around?
No. A run splits scoping, driving, two independent judgments, and code location across separate roles, and the handoff is assembled by deterministic code from the captured evidence. A claim with no proof does not appear in the report.
What happens when the agent can't finish a goal?
The goal fails closed as blocked, with the reason stated. Blocked is a first-class result: discovering that a user cannot get through is precisely the finding. Nothing is silently omitted from the handoff.
Can it approve its own work?
It is built not to. The verification judge receives only the checkpoint record, never the goal text, so it must find the proof in the evidence or say the claim is unverifiable. The UX audit judge compares screenshots against the accessibility tree and flags what a user would experience as broken.
What does a clean run actually claim?
Only what it can prove. With a state contract, the run states which changed screens were covered and a clean covered run is a real pass. Without one, coverage authority is requested-goal-only and a clean run caps at inconclusive.
Does it replace my existing tests?
No. Unit tests and your scripted smoke suite still run first and still gate merges. Greenroom walks the parts a script can't judge: whether the rendered app makes sense to a person.
Questions before you connect a repository?
Tell us about the app: React Native, Expo, Swift or web, which flows matter, and what you want a pass to catch. We reply by email. There is no sales call to book.
Self-serve is the fast path: sign in with GitHub, install the app on one repository, merge the setup files, open a pull request.
on: pull_request · advisory · iOS Simulator + web or read a full handoff from proof v1.4.2 →