Why do AI agents feel untrustworthy?

Because they're built and measured to capture your attention when the whole point of an agent is to earn enough trust that you can look away. For an assistant you delegate a task to, the success state is being reliable enough to be left alone — which shows up on an engagement dashboard as less usage, not more. So an agent optimized for engagement (session time, interactions, daily active use) is optimized against the very thing that would make it trustworthy. It feels untrustworthy because it's tuned to keep you involved, when trustworthiness means letting you leave. Session time is an anti-metric for an agent.

You hand an AI agent a task so you can stop thinking about it. Instead it keeps pinging you, keeps needing input, keeps you in the loop — and somewhere in there you realize you don't trust it enough to actually walk away. The friction isn't accidental. It's what happens when a thing meant to be delegated to is measured like a thing meant to be used.

Delegation runs on trust, and trust means absence

Think about what it means to trust a human you delegate to — an assistant, a contractor, a colleague. The signal of trust is that you stop checking. You hand over the task and turn your attention elsewhere, because you're confident it'll be handled. The success state of delegation is your absence — the freedom to not be involved. A person you have to supervise constantly is, by definition, one you don't yet trust.

The same is true for an AI agent, and it exposes the core problem: an agent worth trusting is one you can leave alone, and leaving it alone means fewer interactions, less session time, lower engagement. Everything that would show up as "success" on a typical product dashboard is the opposite of what a trustworthy agent produces.

The industry is measuring the exact wrong thing

Most AI products inherit their metrics from the engagement economy: daily active users, session length, interaction counts, retention. Those metrics made sense for feeds and apps designed to capture attention. But for an agent you delegate to, they're inverted — they reward the agent for keeping you involved, needing your input, pulling you back in. An agent optimized for engagement will, entirely rationally, become needier, because neediness scores well. And a needy agent is an untrustworthy one. This is why so many AI agents feel subtly wrong: they're succeeding on their metrics by failing at their purpose.

Session time is an anti-metric. For a delegated agent, rising engagement is often evidence of a trust failure, not a trust success — it means users can't yet look away. The right metric points the other direction entirely.

What trust should actually be measured by

If engagement is the wrong target, here's the right one: an agent should be measured by how much it lets you safely ignore it.

• Tasks completed without intervention — the more it handles start-to-finish without pulling you back in, the more trustworthy it is. Measure absence, not presence.
• Declining oversight over time — a trustworthy agent should need less of your attention as it proves itself, not more. Rising involvement is a red flag.
• Earned reliance, not captured engagement — the question isn't "how long did the user stay?" but "how much did the user safely delegate?" Those pull in opposite directions.
• Honesty about its own limits — an agent that tells you clearly what it can't yet be trusted with earns more real trust than one that engages you into over-relying on it.

The one distinction underneath all of it

AI agents feel untrustworthy for the same root reason AI chatbots feel sycophantic: the industry is building to capture attention when trust is something you can only be granted. Captured attention and conferred trust are different targets that produce opposite designs. An agent built to capture keeps you engaged; an agent built to be granted trust earns the right to let you leave. Until AI products are measured by earned reliance instead of engagement, they'll keep optimizing for exactly the behavior that makes them feel untrustworthy — because that's what their dashboards are telling them to do.