Unanswered

Copilot Studio Agent not using knowledge sources when creating generative responses

(1) Share

Report

Posted on by Tommy_Euthyphro

I've created a couple of agents in copilot studio that, theoretically, look at document libraries and answer user questions by digesting the document library contents and providing specific information based on user input.

One agent looks at the knowledge sources and provides relevant information in return but ONLY when I use it. Everyone I have asked to test the agent has access to the agent and to the sharepoint site.

Another I have created doesn't use the knowledge sources at all, even when I use it. It just uses generative AI to create a generic response that is not relevant to my organisation.

The only difference I can see between the two agents and their knowledge sources are shown in the attachments - one shows one of the sources as 'Files' rather than a 'SharePoint' knowledge type.

Two questions

- How can I make the agent actually utilise the knowledge sources?

- Why does the agent only work for me and no one else?

I'm sure there's something I'm missing.....

Categories:

Building Copilot Studio chatbots in Microsoft Teams

I have the same question (1)

All responses (2)

Answers (0)

Sort by

Michael E. Gernaey 53,958 Moderator on at

Like (0)

Report
Copy link

Link copied!

Hi @Tommy_Euthyphro

So for the first question, you Shared the agent to them all and they all have licenses? Also validate in the Admin UI for Apps to make sure and definitely ensure they have licenses for it

For the second question, what type of files are there? I(as in extensions)

What Channel are these posted too?

Also, as a test, you can change your instructions to specifically tell the bot to ONLY answer with the Knowledge Sources or a specific one. Also I would turn off AI completely then run some tests and see if it either says a) I have no knowledge of that or b) it answers it correctly.

I tend to tune my instructions to go towards the specific knowledge source I want first.

Also, it would be good to understand your agent from the perspective of how it talks to them. If you have a topic that has Generate answers and set it to specific knowledge sources or triggers etc.

If you can verify the first few statements I gave you

Was this reply helpful? Yes No
Tommy_Euthyphro 17 on at

Like (0)

Report
Copy link

Link copied!

Thanks @Michael E. Gernaey for the quick response.

I've shared the agent with the team members and they're all licensed (incl. Copilot licenses - on that, do users have to have Copilot licenses to be able to use bots in Teams? I thought it was just the creation of the agents that required the Copilot license?)

The SharePoint knowledge sources point at the folder within which the files are stored. The files are .pdfs. I've tried adding the files individually as knowledge sources but still hit the same wall of it not looking at them correctly.

I've tested turning off AI orchestration, and it just says 'search knowledge sources', or it says that it can't find what it's looking for - so there's something not linking between the agent and it finding the source.

Currently, I have a topic that creates generative responses based on user input after a few intro questions that help the agent look in the right place. I've tested using flows to get the documents - it just limits what the agent can acutally do - I don't want the user to have to specifically name the document that they want to look at - I want to agent to do the time consuming part of finding the document and answering the question (which it does perfectly, but only when I use the agent... so I know it's possible).

Was this reply helpful? Yes No