On November 3, Anthropic opened a public beta for Message Threads on the Messages API. The idea is simple. Your agent sends the whole conversation once, gets a thread ID back, and from then on sends only the new messages. Anthropic says the median time to first token drops about 35% on requests in the 500k to 1M token range.
If you run long agent loops, that number matters. Re-uploading hundreds of thousands of tokens on every turn is slow, and it's mostly wasted work.
What actually changes in your code
Before, your client owned the conversation. Every call carried the full history, so any call could be replayed, retried or moved to another provider without thinking about it.
With threads, the history lives on Anthropic's side. Your client holds a thread ID and a small delta. That's a much lighter request, but your agent is now stateful in a way it wasn't yesterday.
Two details from the beta are worth reading twice. Threads sit in encrypted memory and expire after a short period of inactivity. And they don't change token billing, so you save latency, not money.
The catch is where the state lives
Short-lived server memory is fine until your agent pauses. A human approval step, a slow tool call or a queue backlog can all outlast the expiry window. When the thread is gone, the next request fails, and your agent has no history to send unless you kept it.
That's the trap. Teams that adopt threads and stop storing the transcript have quietly made a third party the only copy of their agent's memory, and that copy is designed to disappear.
There's also a compliance edge. Threads aren't eligible for zero data retention or HIPAA workflows. If one product has both a casual assistant and a regulated flow, the two paths now need different request shapes.
What we'd do before turning it on
- Keep writing the full transcript to your own store. Treat the thread as a cache, not a source of truth. Threads make requests faster. They shouldn't make you the owner of less data.
- Make expiry a normal code path. On a missing or expired thread, rebuild it from your transcript and carry on. Test this on purpose by deleting thread IDs in staging.
- Gate by workflow, not by app. Check each flow against the ZDR and HIPAA limits before it gets a thread ID. A feature flag per workflow is cheaper than an audit finding.
- Measure your own latency. The 35% is a median on very large requests. If your prompts are 20k tokens, the gain will be much smaller and probably not worth the added moving part.
Where it fits
Threads make the most sense for coding agents, research loops and any long session where context grows into the hundreds of thousands of tokens. They make the least sense for short chats and anything under strict retention rules.
The pattern here is a familiar one. A provider removes a cost you were paying every turn, and in exchange you depend on something you used to own. That's often a good trade, as long as you notice you're making it and keep an exit.
We're here to help founders and teams design and build digital products that are built to scale with you, not slow you down. If you're looking to build something, get in contact with us today!