Answered

D365 ERP MCP tool in CoPilot Studio Inconsistent results

(0) Share

Report

Posted on by Sai Kiran Kolusu

Hi Team,

I'm using D365 ERP MCP in my copilot studio, I want to analyse my inventory data from D365, I've tried various combinations of instructions, prompts, but there was no consistency in the result.
Example prompt: Give me top 100 active items from inventory.
Every time it responds there are different items. By fine tuning the instructions I've got to a point where I was able to get 70% similar items across sessions. (Attaching pics of 2 analysis reports with the same prompt, instructions, highlighting the missing items)

I want to make it more accurate so that the business can take decisions using this Agent.
By doing some research, reverse engineering I got to know that this data inconsistency is due to the data retrieval that is done by MCP. In some cases there were time out issues as these operations are timing consuming.
Is there a way where I can get consistent data across sessions? or is this too much to ask from the server.

Thanks

Categories:

Calling actions from Copilot Studio

All responses (2)

Answers (1)

Sort by

Suggested answer

Sayali-MSFT Microsoft Employee on at

Like
a
(0)

Report
Copy link

Link copied!

Hello Sai Kiran Kolusu ,

This is a known challenge with AI-driven MCP scenarios. The agent does not always analyze a fixed dataset in a deterministic way. Large inventory queries can be affected by retrieval limits, pagination, timeouts, or variations in tool execution, resulting in different outputs for the same prompt.

For business-critical reporting, it's recommended to use a deterministic D365 query/API/report that returns the exact Top 100 items and let the agent summarize the results, rather than relying on the LLM to identify and rank items dynamically. This approach provides much more consistent and reliable results across sessions.
Verified answer

Sai Kiran Kolusu 12 on at

Like
a
(0)

Report
Copy link

Link copied!

I was able to achieve conistent results when I've used new harness and added skills to the agent where I defined which entities to be queried and in which order.