SharePoint Online & Generative Answers w/ Search and Summarize Content

(0) Share

Report

Posted on by jhoffmann2

Hi All,

at the moment, a colleague and I are facing several "challenges" w/ Copilot Studio, expecially when it comes to authentication and SharePoint access. Though browsing several MSFT resources, we were not able to solve all of them. To be precise: Copilot should use several documents hosted at a SharePoint site and answer user qeustions on documents stored w/ SharePoint.

Given the case, Copilot / Graph / Bot Framework is able to access a SharePoint document, but the User asking Copilot isn't allowed to access that specific information (e.g. a document), which authorization applies in that case? Hopefully, the user will NOT see content forbidden to him/her. But how does it work technically?
So far, we understood, Copilot indexes documents and accesses the index to answer questions. In case, that's correct, when exactly does indexing take place in/with MS Graph? The time, SP connection is initiated / in the design phase? What about newly added documents? If Copilot instead analyses content in real-time, how's achieved this (from a technical point of view)?
According to the MSFT documentation: if a copilot is published on SharePoint, then it cannot access SP - what does data in Teams count for? Or SSON doesn't work? Or both?
We can add knowledge sources globally and also in a node. What's the difference? When it comes to SharePoint sites, the can be added also als "classical data source". What are the differences in that case? Are there any issues / restrictions w/ classic data sources?
An user is able to rate answers given by Copilot. Is there any effect of the user's rating to Copilot behaviour / learning? Or is it only for statistical purposes?
Which LLM is Copilot using at the moment? We found, using some modern LLMs, e.g. GPT-4, we get better results compared to the Copilot ones.
As there's no native SharePoint integration available at the moment (yes it will come in few weeks) and to achieve proper SharePoint access, we setup manual authentication w/ custom Copilots. at the moment, we're struggling "AADSTS50011" errors pointing to the rediredt URI. We tested both, https://europe.token.botframework.com/.auth/web/redirect and https://token.botframework.com/.auth/web/redirect. With no success 'til now.

Any ideas, hints helping us to get this Copilot up and running? We searched MSFT documentation and GitHib repositories available to us (we're Microsoft Solution Partner).

Categories:

Calling actions from Copilot Studio

I have the same question (0)

All responses (3)

Answers (0)

jhoffmann2 10 on at

Like (0)

Report

Update: in the meantime, we fixed the AADSTS550011 error. Unfortunately, we messed up redirect URIs ... too bad.

Any ideas though? Nothing? Some Copilot Studio specialist at Microsoft?

Was this reply helpful? Yes No
Suggested answer

Mahesh Chintha 158 on at

Like (1)

Report

1. If the documents are uploaded to SharePoint and configured an authentication and Entra ID application with delegated permissions, Users can access only the content they have access to in SharePoint sites. Users will not see the files/content they do not have access to. Internally it uses users permission to search the content with SharePoint.

2. Files in SharePoint are indexed by SharePoint search and Copilot accesses this search index. SharePoint search crawls and index the content within SharePoint sites continuously and incrementally based on the Search configuration. This search index is accessed using MS Graph API and Copilot Studio Generative Answers uses top 3 results from the search to generate answers.

3. I did not understand this question - But, Once a custom copilot is published to SharePoint, it can still access the SharePoint content and generate answers.

4. You can have multiple Generative Answers nodes in various topics. Knowledge sources added at globally is accessible to all Generative Answers nodes across topics to generate answers. Where as, you can configure knowledge source local to a node which is only accessible for the specific node configured.

5. CSAT rating given by users at the end of the conversation is for analytics in Copilot Studio. You can review these ratings and fine tune your Bot. Thumbs up/down given for Generative Answers is used by MS for an LLM model evaluation and learning.

6. As of March, Copilot Studio Generative Answers was using GPT 3.5 Turbo LLM. MS can confirm if they have updated it to GPT 4 or above

Was this reply helpful? Yes No
jhoffmann2 10 on at

Like (0)

Report

Many thanks Mahesh for sorting out our questions so far. That'll help, definitely!

Related to topic #3: my question refers to this MS Learn article: Configure single sign-on with Microsoft Entra ID - Microsoft Copilot Studio | Microsoft Learn.

Especially when I stumbled upon that information box:

Do you have any information here, too?

Appreciate your comment / answer. Regards, Jochen.

Was this reply helpful? Yes No