Agent doesn't Fall Back with SharePoint-Based Knowledge (it hallucinates more)

(0) Share

Report

Posted on by ND-02010700-0

Hi all,

I've noticed a difference in fallback behavior depending on the type of knowledge source I use.

Initially, I uploaded Word documents as knowledge sources, and my copilot would correctly fall back when it couldn’t find an answer. However, after switching to SharePoint-based knowledge (without semantic search, since the bot uses manual authentication), the fallback behavior has changed.

Now, when I ask a question that isn’t covered by the knowledge base, instead of falling back in 'fallback topic', the bot acknowledges that it doesn't have the answer but still attempts to generate a likely response based on other knowledge sources. This is frustrating, I would prefer that it simply fall back instead of making assumptions that make no sense. All my settings are correct either (that it is not allowed to answer with own knowledge etc...)

Is there any way to restore the proper fallback behavior when using SharePoint? Has anyone else experienced this?

Categories:

General topics

I have the same question (0)

All responses (3)

Answers (0)

Michael E. Gernaey 53,429 Super User 2025 Season 2 on at

Like (0)

Report

Hi @ND-02010700-0

No, if you have turned off all external sources aside from SharePoint, it should not do this.

Can you share your Agent Main Page Configuration please, not inside the knowledge sources but the main page.

It should fall back unless

-You have other sources and not enough proper instructions

-You didn't turn off for it to try to answer using default options.

I literally have an instruction that tells my bot NOT to answer anything unless it comes from my single

sharepoint Knowledge source. Now, there is a difference from uploading Word docs and using sharepoint document libraries, so you will need to be a bit more description in your Natural Language instructions.

By doing so I was able to insure fallback worked properly and my sharepoint libraries did too.

Don't forget it also has limitations on URL depth etc so not sure how deep your document libraries go etc etc

Was this reply helpful? Yes No
ND-02010700-0 24 on at

Like (0)

Report

Hi @Michael E. Gernaey,

Thank you for your reply! I will rewrite my instructions.

I do have a question about the URL depth. I remember seeing documentation on this at some point, but I can’t to find it anywhere and I thought it might no longer be relevant since I cannot find it. Do you happen to have a link to where this is documented?

And what do you mean with ‘It should fall back unless you have other sources’? You mean if it is mentioned in another source or doesn’t it work when other sources (although they do not contain the knowledge) are involved?

Was this reply helpful? Yes No
Suggested answer

Michael E. Gernaey 53,429 Super User 2025 Season 2 on at

Like (0)

Report

Hi @ND-02010700-0

Here are some details

Search public data or use a Bing Custom Search for generative answers nodes - Microsoft Copilot Studio | Microsoft Learn

Add a public website as a knowledge source - Microsoft Copilot Studio | Microsoft Learn

Also

And what do you mean with ‘It should fall back unless you have other sources’? You mean if it is mentioned in another source or doesn’t it work when other sources (although they do not contain the knowledge) are involved?

Fallback is for when nothing is found, regardless, but if you specify a knowledge source specifically it won't fallback as you told it to only do that source

Was this reply helpful? Yes No