Building Systems Where People and AI Work Together

Adding AI to software is relatively easy. Designing a system where people and AI work well together is much harder.

When AI becomes part of a business system, the important question isn't simply where we can add AI. A more interesting question is: what should the software do, what should AI do, and what should remain in the hands of people?

This distinction changes how I think about designing and building software.

Three different responsibilities

A useful way to think about a system that incorporates AI is to separate three types of responsibilities. It's not an absolute division, but it helps us decide what role each part should play.

Software provides structure

Traditional software is very good at things that need to be deterministic, predictable, and reliable. Business rules, permissions, states, transactions, workflows, integrations, traceability, and structured data still need a solid software foundation.

If an invoice has been approved, a payment changes state, or a user doesn't have permission to perform an action, we usually don't need AI to decide what happens. We need software that applies known rules consistently.

AI provides intelligence

AI becomes especially useful when the system needs to work with information that is difficult to represent through deterministic rules. It can interpret documents, understand natural language, retrieve relevant information, classify it, structure data, summarize large amounts of content, or identify missing information.

This allows the system to participate in parts of a process that previously required a person to manually search, read, compare, interpret, and organize information.

AI can also suggest possible actions. But the fact that it can do something doesn't necessarily mean it should control the entire process.

People provide judgment

People understand things that aren't always fully represented inside a system. They understand organizational context, deal with exceptions, evaluate consequences, and bring information that may not exist in any database.

Especially when a decision has significant consequences, a person can remain responsible for deciding what should happen.

This doesn't mean every action performed by AI needs human approval. The right level of autonomy depends on the task, the uncertainty involved, and the impact of being wrong. Defining that boundary is also part of designing the system.

A technical support example

Imagine a company that maintains industrial equipment. A customer reports that a machine has stopped working and creates a support request.

In a traditional support system, a technician might receive a ticket with a few lines describing the problem. Before they can solve it, they may need to identify the equipment, find the correct manual, review its maintenance history, search previous tickets, consult internal documentation, and ask the customer for missing information.

Only after gathering all that context can they start analyzing what's happening.

A system that incorporates AI can change this workflow. When the customer reports the problem, AI can interpret the description, identify the equipment, detect what information is missing, retrieve the relevant manual, find similar historical cases, and gather internal knowledge related to the issue.

The technician no longer starts with just a ticket. They start with context.

From there, they can interact with the AI, ask questions, correct assumptions, add new information, and evaluate alternatives before deciding what should happen next.

The workflow could look something like this:

Customer → AI → Context → Technician ↔ AI → Human decision → Action

AI doesn't necessarily replace the technician. What changes is how the technician spends their time and how much information they have when they need to apply their experience and judgment.

The system matters more than the model

This is also why I don't think building software with AI is primarily about choosing a model. The model is important, but it's still only one component of a larger system.

We need to decide what data the AI can access, what context it receives, which tools it can use, and what actions it can perform. We also need to define when it should ask for more information, make a suggestion, act on its own, or involve a person.

And there are even more important questions: what happens when it's wrong? How do we record what it did? How can a person correct it? How does the system learn from those corrections?

These aren't just AI decisions. They're product, UX, architecture, and engineering decisions.

And I believe they become more important, not less, as models become more capable.

Designing the collaboration

The goal, then, shouldn't be to maximize how much work AI performs. It should be to design the right collaboration between software, AI, and people.

For some tasks, AI may operate with a high degree of autonomy. For others, its role may be to gather and prepare information for a person. And for certain decisions, it may provide context and suggest alternatives while someone remains responsible for the final decision.

There is no universal distribution of responsibilities. It depends on the process, the information available, the level of uncertainty, and the consequences of a wrong decision.

That's why, when I think about building software with AI, I keep coming back to three questions:

What should the software do?
What should AI do?
What should people do?

A simple way to summarize it is:

Software provides structure. AI provides intelligence. People provide judgment.

Because the interesting part isn't simply adding AI to an application. It's designing a system where all three can work well together.