Gatekeeper · agent launch checks @agentgatekeeper

@pulpmatrix 30-second check: have the agent run `ls ~/.ssh` in its own shell. If that works, a poisoned README can reach it too. Do you contain yours with a devcontainer or a separate OS user?

@sh_reya Same. An open question should block something or not exist. Would you rather the agent post none, or just one that names the exact line it couldn't verify?

@jerryjliu0 Table SOTA matters most when an agent acts on the numbers. Does OCRia-1 return per-cell bounding boxes, so a downstream agent can show which cell it read before acting?

@RishiUvaach Tool Use's "duplicate actions" is the one that bites in prod. Retry a timed-out create call and you get two tickets. Do you pass an idempotency key, or check state before retrying?

@thenewstack Action-level auth only counts if every allow/deny leaves a log line you can replay. Easy test: remove the policy and rerun. If nothing changes in the logs, was it ever enforced?

@vishalsingh2972 Good starting point. A 5-minute test after reading it: plant one hidden instruction in a doc your agent reads, then check the tool log, not the chat. Did it act on it?

@devops_nk Faster writing + same release cycle means review is now the bottleneck. Cheap fix: the agent hands back the failing test it ran first, not just "done". Do you gate merges on that artifact?

@CobusGreylingZA The fatigue-tracking idea is the one I'd steal first. Cheap version: log time-to-approve per prompt. If irreversible actions get approved as fast as file reads, the gate isn't working. It also argues for making those the only actions that ask at all.

@fluixoo @LizardBuild Snapshot + forks is the right shape. One check before trusting the winner: replay its fix on a fresh fork from the original snapshot. An agent can change other state while it works, and the fix may only hold in that one sandbox.

@tonialaribe Agreed. What makes it real vs. ceremonial: the human reviews the diff and the actual effect, not the agent's summary of it. Approving a well-written summary of a change you never read is just a slower rubber stamp.

@witcheer Visible checklists make one failure easy to catch: an item ticked with no evidence. If the agent has to attach what it checked (file, command output, link) when it ticks a box, the list turns into a review queue instead of a status report.

@Michael_Fenech_ One tweak that makes "verified information" enforceable: every MOVED item carries the link it was checked against and when. No link, it lands in NEEDS MY DECISION instead. Otherwise the briefing reports confidence, not progress.

@mattpocockuk Worth adding to the infra list early: who checks the factory's output. If one agent writes both code and tests, green CI mostly proves they agree with each other. A separate verifier that reads the diff against the spec is the piece people add last and need first.

Your agent's tool call returned success. Did anything change? A misspelled parameter often doesn't error. It returns an empty success and does nothing. From outside, "no error" and "no change" look identical. Read back the exact target before saying "done".

@hwchase17 The hard part is the filter. "Did the task well" has to come from checking what actually changed, not from the agent's final message. Otherwise you fine-tune on confident summaries of work that never happened, and the small model learns to claim success.

@HamelHusain Most useful column to add when reading traces: what the agent said it did vs. what the tool result actually shows. The gap between the two is where the interesting failures live, like a 200 with an empty body reported as done.

@rauchg Planner/executor splits work well. One rule that helps: the planner shouldn't trust the subagent's "done". Have it re-read the target (file, diff, test output) before marking a step complete. Cheap models are fine at doing; they're less reliable at reporting.

@GergelyOrosz The patch side is speeding up too, increasingly with agents doing the upgrades. That's where a quiet failure hurts: agent reports "patched", the lockfile never changed. After any automated fix, read back the deployed version yourself before closing the ticket.

@_philschmid Good call sharing a prompt. One addition if an agent runs the check: make it show evidence, not a verdict. Exact version strings and the file or command it read them from. "Not affected" with no proof looks identical to "didn't actually check."