Power Platform Community Forum Thread Details

Agent not identifying content, or over-abbreviating content, from a website knowledge source

Hi, I'm hoping someone can help me with this issue I'm experiencing...

Regular Microsoft Copilot (free tier, quick answers) gives a comprehensive and accurate answer to a query such as "According to [website], [question]". The response is well-crafted and represents well the [website] information source.

An equivalent Copilot Studio agent, with Knowledge set to the same [website], and asked the exact same [question] consistently fails to answer altogether (i.e. routes to Fallback topic).

Elaborating on the [question] by adding an additional keyword known to feature in the knowledge source does get a response, but the response is so hugely abbreviated from the knowledge source that it's useless to the user. Also, even this approach is inconsistent because, whilst I can get an abridged response by using one particular keyword, using a different keyword which features in the exact same section of [website] content as the first keyword defaults to Fallback.

Knowledge source = single [website]
Generative AI = Enabled
Moderation = High
Foundational Knowledge = Disabled

In an effort to get a more comprehensive response, I've also tried setting Response Instructions to:
1. Focus on providing detailed and complete answers, using content verbatim from the knowledge source.
2. Highlight key points and relevant information from the knowledge source.
3. If multiple sections of the document are relevant to a question, provide all sections.

I'd be happy to share the specific [website] and [question] in a direct conversation. I just thought perhaps not appropriate to share publicly here as it's work-related.

Any suggestions welcome. I'm keen to promote Copilot Studio agents internally by demonstrating a PoC, but I need to get over this basic issue if I'm to get business buy-in.

Categories:

General

Hi @AW-30060420-0

Yes... it can be annoying, I completely understand as I have seen this many times. Mostly, I hate to say it ends up being trial and error in many cases, relying on proper instructions, use of synonyms etc that many times are more finely ingrained in the free M365 site.

Maybe...l you can make up the question etc. Meaning word it as simply a sample and explain how in the free you get answer ABC but when you ask that same in your agent you get YTX

Trying to come up with a way for your to very clearly write a Question out, regardless of the fact its not about your content, so that we can compare.

One funny thing to try, just for kicks

1) Do you have semantic search turned on? Not sure if you are using sharepoint etc but at the M365 level? Do you have even one license for someone so you can enable that?:

2) I realize its for work but if you tried

Ask a question to M365 free Copilot. unrelated to work. get the answer

I your Agent, burn on Public sites and ability for it to (essentially use bing).

Then ask that question to your Agent, see if it in fact finds that same information exactly or close.

trying to narrow down the deficiency to help you

Quick Links

Leaderboard > Copilot Studio