AI: Experience and Technology Can't Be Designed Separately

When we designed user experiences for applications before the AI era, two or three years back, the possibilities were always few and limited: a form submitted successfully or contained an error, a process ran successfully, got retried, or was marked as failed, a transaction rolled back or got confirmed.

Today, applications that incorporate AI functionality are far more complex to design for. It's no longer enough to design for when everything goes right or when the final result is the expected one. You also have to design for every possible intermediate outcome. And that's no longer a task that can be solved from just one side: experience and technology stop being two separate stages, where you first think through the design and then the system gets built, and instead get designed together, decision by decision.

Why this changed in the AI era

Today, one of the feature requirements for pretty much any application is to incorporate some kind of AI functionality, one way or another. AI lets us design more human experiences. It also lets us automate many processes, even very complex ones, freeing up people's time for other kinds of work.

But all the benefits AI brings also come with some considerations that, without being downsides exactly, need to be handled correctly.

The first is that AI is non-deterministic. Unlike software without AI, where conditions were explicitly programmed, AI can return different results even given the exact same input data. We always have to keep in mind that the outcome of a given process run by AI can be very different each time it executes.

The second is that executions can take minutes, hours, even days. AI agents need to know what the endpoint is where they can return a response. But to get there, they can run, if they decide to, dozens or even hundreds of steps before reaching it. It always depends on the task at hand, but it can range from querying a database to spinning up specialized subagents, creating a secure environment to run code, browsing the web for information, and more.

And it's not just a code-level problem, the user experience is what matters most

In processes with AI agents that run for a long time, the user can even close the browser or the app and come back the next day. Everything has to keep working and delivering the experience to the user, even under these circumstances.

Many AI operations also require the user to make a decision at certain critical points, where leaving it up to the agent's judgment could be risky.

So how do you deliver a good experience in systems with AI agents

The best approach is to first think about what the user needs to accomplish. Clearly defining what's called the "job-to-be-done," or the tasks that need to get done, is fundamental. This part hasn't changed compared to design from 2 or 3 years ago.

What has changed is AI's non-deterministic nature. No matter how much we try to reduce its randomness, there's always room for it to return different responses to the same input.

You have to account for it from both the technical side and the experience side

The best approach, then, is to think through the whole experience and the technical layer together.

From the technical side

Persisted state: if the process gets cut off midway, where does it resume from? Every step needs a checkpoint, not just the final result.
Idempotency: if a step gets retried, does it produce a duplicate effect (a charge, an email, a record), or is it safe to repeat?
Timeouts and retries: every call to a model or an external tool needs a time limit and a clear retry policy, not a "keep waiting."
Partial failures: an agent that runs five tools where the third one fails shouldn't lose the work from the first two. The data design has to allow for saving partial progress.
Observability: every step needs to be logged with its input, its output, and its intermediate reasoning, because when something goes wrong, the question isn't "what failed" but "at which step, with what context."
Cost control: a long-running execution stuck in a loop burns through tokens and tool calls. It needs hard limits, not just good intentions.
Decoupling from the session: execution can't live in the browser or depend on the user keeping a tab open. A job that takes hours or days has to run on the server, survive the user logging out, and stay recoverable from any device when they come back.

From the experience side

Progress visibility: the user needs to know the system is still working and, when possible, what it's working on. Total silence during a long execution reads as the system being broken.
Sync vs. async based on duration: a step that takes seconds can be shown live, with a spinner or a log that fills in as it goes. A process that takes hours or days needs the opposite pattern: the user leaves, and the system brings them back, via email, push, or an in-app notification, when something needs their attention or when the result is ready. Designing only the "live" mode for an execution that can take a full day is designing for something nobody will use that way.
Interruptibility: if an execution takes minutes, hours, or days, the user has to be able to cancel or pause it at any point, without leaving the data in an inconsistent state.
Independently queryable state: the user has to be able to ask "how's this going?" from anywhere, a dashboard, a link, a command, without depending on the session where the process started.
Human intervention points: not every agent flow should run end to end without supervision. You have to decide where the system asks for confirmation before an irreversible step.
Communicating failures, not technical errors: when a step fails, the user doesn't need a stack trace. They need to know what happened, what impact it has, and what they can do now.
Time expectations: a three-minute process with no progress signal at all feels slower than a ten-minute one with constant feedback. And a process that can take anywhere from twenty minutes to two days, depending on the case, needs to communicate that range up front, instead of leaving the user guessing whether something got stuck.
Graceful degradation: if a secondary step fails but the main result can still be delivered, the system should complete it with a warning, not cancel the whole process.

The real design work

Today's tools handle writing an agent's code at a speed that would have been unthinkable a few years ago. What they don't handle is this layer: deciding what happens at each step, both in the data and in the experience, when execution isn't instant and isn't deterministic. That diagnostic work, mapping out each step, its system contract and its experience contract, is what actually determines whether an agent-based product is reliable or just a lucky demo.

Thinking about the experience first and the technology second, or the other way around, is the mistake. In a system with AI agents, both disciplines get designed together, step by step, or the system doesn't work in production.