Power Platform Community Forum Thread Details

Our Compliance team is trying to create an AI chatbot from Copilot Studios that can answer compliance related questions. The knowledge we've given it is all stored in our company's Sharepoint. When asking questions, our AI chatbot works just fine, however, when we want to review responses in the backend (Analytics > Sessions), most of our chatbot's responses have been redacted. We can't seem to figure out how to turn that off. We really need to review its responses to verify its accuracy. Is there any idea on how to fix this?

We also looked at https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-transcript-controls?source=recommendations and the first note says, "Agent responses that use SharePoint as a knowledge source aren't included in conversation transcripts." I asked the page's "Learn" AI and it stated,

"If your custom AI agent uses SharePoint as a knowledge source, its responses will not be included in the conversation transcripts. This means you won’t be able to see those responses in the Analytic Sessions section of Copilot. The system specifically notes that “Agent responses that use SharePoint as a knowledge source aren’t included in conversation transcripts.”1

To work around this limitation, you might consider uploading documents directly to your agent instead of relying on SharePoint. This way, the responses can be included in the transcripts and thus visible in the analytics. However, be aware that if you choose to upload files, they must adhere to certain size limits; for instance, files can be up to 512 MB when uploaded directly, while SharePoint files have different limits based on licensing and features enabled.2

The reason that answers derived from SharePoint knowledge cannot be shown in transcripts is likely due to privacy and data management policies, which aim to control sensitive information and the size of the conversation transcript table in Dataverse. 1

Therefore, if you require visibility into the interactions, uploading documents directly may be the best approach."

Is this true? Can we not properly use our AI chatbot with it's knowlege stored in Sharepoint?

Categories:

Building Copilot Studio chatbots in Microsoft Teams

Bot analytics

General topics

Hi,
yes, it's true, and it's by design, not a setting you can switch off. Responses grounded in a SharePoint knowledge source are intentionally left out of conversation transcripts, so those redacted entries in Analytics > Sessions will stay redacted no matter what you toggle. And it's not just that one note. The same line shows up across the current docs (Add SharePoint as a knowledge source, the generative answers with SharePoint page, and the admin transcript controls page). Microsoft frames it as a privacy safeguard and a way to keep the Dataverse transcript table from blowing up. So the Learn AI's summary is basically right.

Good news is you're not stuck, and you can keep SharePoint as your knowledge source.

It really comes down to what you're after:
For verifying accuracy, which sounds like your case, you don't really need the production transcripts. Run your test questions in the test pane and review the answers right there.

You can also hit the three dots next to "Test your agent" and pick Save snapshot, which downloads a botcontent zip with a dialog.json showing exactly what the agent did that session. That's the cleanest way to spot-check responses without touching your knowledge setup.

For ongoing audit logging of live conversations, that's where the redaction genuinely bites. The documented workaround (what the Learn AI pointed to) is uploading the documents straight to the agent instead of using a SharePoint source. That does get responses into transcripts, but you lose live sync when the source docs change and the permission trimming that keeps users from seeing content they shouldn't, plus file size limits apply.

For compliance material that's usually a real tradeoff, routing the agent's telemetry to Azure Application Insights is another logging path worth a look.
So nothing's broken on your end, and SharePoint knowledge is fine to use. You just can't review those specific answers through the Sessions view.

For accuracy checks I'd start with the test pane and snapshot.