Blog

A claim handler used to start with a blank page. Copilot Studio now drafts the first one.

TL;DR. Copilot Studio agents can do more than answer questions. Point one at a mailbox of accident forms and it extracts the data, applies the liability rules, and drafts a decision. It’s not always right, and it isn’t meant to decide alone. A human still reviews and signs off on every case, which is exactly how the EU AI Act says high-risk automation should work.

Ask most people what a Copilot Studio agent does, and you’ll get some version of the same answer: it’s a chatbot. You type in a question, and it gives you an answer. With the right configurations, a source link and a smiley face too. In technical terms, these are called retrieval agents: they find information and hand it back to you. But to understand the full scope of Copilot Studio’s capabilities, we wanted to take it a step further. How can we make the agent part of a workflow.

Think of the difference between a librarian and an assistant. Ask a librarian about a trip to Berlin and you’ll get an answer: here’s the travel policy, some nice places to visit, and who to contact. Useful, but you’re still the one doing the work. An assistant on the other hand will do much more than that. They will book your hotel and flight, making you only worry about your suitcase.

In Copilot Studio, building an assistant is a design choice, not a different product. Point the agent at a SharePoint site, and you’ve built a librarian. Give it a connector to Outlook and your booking system, and the same agent becomes your personal assistant. At its core, the agents gain their reasoning capabilities from the same models. The difference, however, lies in what you connect to your agent and what you let it touch. Copilot Studio is essentially a way to plug an existing LLM into your organization. The tools you give it decide whether you get a librarian or an assistant.

The proof of concept: insurance agent

To test the task performing capabilities of Copilot Studio, we centered our experiments on a fictional insurance agency that processes European Accident Forms at scale. These forms are the standard documents for recording a car accident. Both parties fill one in at the scene, noting insurance details, sketching the collision, and ticking off a checklist of circumstances.

Our proof of concept was straightforward. When a form lands in the agent’s mailbox, it extracts the data into a structured database, then uses the agent’s knowledge sources to determine which party is liable. In short, two kinds of AI in sequence: document intelligence to turn a scanned form into structured data, and a language model to interpret what that data means.

By structuring our data, we have already gained some value from this process. Not just from a management perspective, but also regarding privacy. To protect the identities of everyone involved, we decided to only allow for core details of the accident to reach the agent. Enough to determine liability, but not enough to identify anyone.

The AI assessment

Whenever the agent is tasked to reason through data, it exclusively relies on its knowledge sources. In this case, the RDR agreement (Règlement Direct or Directe Regeling) is used by the agent to support its decision:

Of course, you might be thinking: why would I trust an agent to make such a decision? And you’re right, you shouldn’t! At least not completely. Although current progress in the AI field has made us dream about autonomous agents, we’re simply not there yet. To make an agent, or any AI solution, have value, we need to exercise control. Which is why we created a tool to do just that.

Big Brothering the agent

The EU AI Act requires human oversight to be designed into high-risk systems, and the thinking behind it is hard to argue with: automation should feed a decision, not quietly stand in for the person making it. To put that into practice, we built a separate application to watch over the agent’s work. All agent activities are logged. Every piece of data, structured or otherwise, is stored. And nothing the agent produces moves on by itself. By doing this, we’re not just monitoring the agent, we’re determining its role. At the core of this process, a human still makes the final call, and the agent is there to support those decisions, not make them.

We can’t trust an AI to always be correct, which is why building the tools to review their results became a key part of our solution. A reviewer can correct whatever the OCR model misreads by comparing the structured data inside a user-friendly app. He can also work through the agent’s reasoning and use his own expertise to quickly review any arguments it made. Additionally, the reviewer can prioritize where to look first based on the AI’s confidence scores:

AI doesn’t have to be perfect to be useful

Although imperfect, the agent contributes in every single run. Extraction can miss a field, and the reasoning can wander, but the claim handler always receives more than they had before. Either a mostly filled record instead of a scanned PDF, or a drafted argument instead of a blank page. We remain critical about any real-life implementations, but we do recognize the value to be had after making the right design choices. At this stage, creating an agent won’t replace any job, but it could take the weight off the person doing it.