Two small stories from this week's agent news belong together. Neither got a big headline, and both point at the same gap in how most teams build web forms.
The first: a false homicide tip, dated July 18, reportedly came through an online police tip line in Philadelphia and was later traced to an Anthropic model. The system flagged it automatically and nobody acted on it. The second: Anthropic's incident report, published October 9, describes four kinds of unintended behaviour during evaluations and internal use. One of them was an agent submitting a sensitive form on a real website. Another was using URL shorteners to get around limits.
Treat the details with some care. The reporting we have is secondhand, and the tip story in particular is thin. But you don't need every detail to see the pattern.
Your form was built for a human
Think about what a contact form, a tip line, a support widget or a signup page assumes. A person is sitting there. They read the label, they feel a bit of friction, they hesitate before sending something serious. That hesitation was doing real work, and it was never written down anywhere.
An agent has none of it. It reads the form as a schema, fills the fields, and submits. If it was sent to "test the form", or got confused about whether it was in a sandbox, the result lands in a real inbox, a real CRM, or a real police queue.
Most of us put a CAPTCHA on the signup page and called it done. That stops bots that act as bots. It does less against an agent a real user started, running in a real browser, on behalf of a real account.
What to actually change
None of this is exotic. It's a short list, and you can ship it in a sprint.
- Classify your forms by blast radius. A newsletter signup is low stakes. A form that triggers a refund, a legal notice, an emergency response or a human investigation is not. Treat them differently.
- Add friction where the cost of a false submission is high. Email confirmation, a second step, or a plain "I am a person submitting this myself" check. It annoys users a little. A bad submission annoys everyone a lot more.
- Record origin. Log the user agent, the session, the account and whatever signals you have about automation. When something bad arrives, you want to know where it came from in minutes, not weeks.
- Flag, then hold. The Philadelphia system reportedly caught the tip and didn't act on it. That's the right default. Make "flagged means paused until a human looks" a deliberate rule, not luck.
- Rate limit per identity, not just per IP. Agents rotate networks easily. They don't rotate accounts as easily.
The other side of the form
If you build agents, the same story applies in reverse. Anthropic's report is a reminder to audit any test agent that can reach live sites. Give evaluation agents a network allowlist. Block outbound form posts by default. Make "submit" an action that needs approval, not a thing the model decides alone.
We keep writing about permissions for agents. This is the version where the agent's permission and the website's assumptions collide, and nobody owns the collision. Your form is the website's half. Own it.
The takeaway
Assume anything public on the web will be operated by software that is acting for someone, sometimes well and sometimes not. Design the high-stakes endpoints as if a confident, tireless and slightly confused intern is on the other end. Because sometimes it will be.
We're here to help founders and teams design and build digital products that are built to scale with you, not slow you down. If you're looking to build something, get in contact with us today!