Skip to main content

Notifications

Copilot Studio - Calling Actions from Copilot S...
Suggested answer

SharePoint Online & Generative Answers w/ Search and Summarize Content

(0) ShareShare
ReportReport
Posted on by 10
Hi All,
at the moment, a colleague and I are facing several "challenges" w/ Copilot Studio, expecially when it comes to authentication and SharePoint access. Though browsing several MSFT resources, we were not able to solve all of them. To be precise: Copilot should use several documents hosted at a SharePoint site and answer user qeustions on documents stored w/ SharePoint.
  1.     Given the case, Copilot / Graph / Bot Framework is able to access a SharePoint document, but the User asking Copilot isn't allowed to access that specific information (e.g. a document), which authorization applies in that case? Hopefully, the user will NOT see content forbidden to him/her. But how does it work technically?
  2.     So far, we understood, Copilot indexes documents and accesses the index to answer questions. In case, that's correct, when exactly does indexing take place in/with MS Graph? The time, SP connection is initiated / in the design phase? What about newly added documents? If Copilot instead analyses content in real-time, how's achieved this (from a technical point of view)?
  3.     According to the MSFT documentation: if a copilot is published on SharePoint, then it cannot access SP - what does data in Teams count for? Or SSON doesn't work? Or both?
  4.     We can add knowledge sources globally and also in a node. What's the difference? When it comes to SharePoint sites, the can be added also als "classical data source". What are the differences in that case? Are there any issues / restrictions w/ classic data sources?
  5.     An user is able to rate answers given by Copilot. Is there any effect of the user's rating to Copilot behaviour / learning? Or is it only for statistical purposes?
  6.     Which LLM is Copilot using at the moment? We found, using some modern LLMs, e.g. GPT-4, we get better results compared to the Copilot ones.
  7.     As there's no native SharePoint integration available at the moment (yes it will come in few weeks) and to achieve proper SharePoint access, we setup manual authentication w/ custom Copilots. at the moment, we're struggling "AADSTS50011" errors pointing to the rediredt URI. We tested both, https://europe.token.botframework.com/.auth/web/redirect and https://token.botframework.com/.auth/web/redirect. With no success 'til now.
Any ideas, hints helping us to get this Copilot up and running? We searched MSFT documentation and GitHib repositories available to us (we're Microsoft Solution Partner). 
  • jhoffmann2 Profile Picture
    jhoffmann2 10 on at
    SharePoint Online & Generative Answers w/ Search and Summarize Content
    Many thanks Mahesh for sorting out our questions so far. That'll help, definitely! 
     
    Related to topic #3: my question refers to this MS Learn article: Configure single sign-on with Microsoft Entra ID - Microsoft Copilot Studio | Microsoft Learn
    Especially when I stumbled upon that information box: 
     
    Do you have any information here, too? 
     
     
    Appreciate your comment / answer. Regards, Jochen.
  • Suggested answer
    Mahesh Chintha Profile Picture
    Mahesh Chintha 158 on at
    SharePoint Online & Generative Answers w/ Search and Summarize Content
    1. If the documents are uploaded to SharePoint and configured an authentication and Entra ID application with delegated permissions, Users can access only the content they have access to in SharePoint sites. Users will not see the files/content they do not have access to. Internally it uses users permission to search the content with SharePoint.
     
    2. Files in SharePoint are indexed by SharePoint search and Copilot accesses this search index. SharePoint search crawls and index the content within SharePoint sites continuously and incrementally based on the Search configuration. This search index is accessed using MS Graph API and Copilot Studio Generative Answers uses top 3 results from the search to generate answers.
     
    3. I did not understand this question - But, Once a custom copilot is published to SharePoint, it can still access the SharePoint content and generate answers.
     
    4. You can have multiple Generative Answers nodes in various topics. Knowledge sources added at globally is accessible to all Generative Answers nodes across topics to generate answers. Where as, you can configure knowledge source local to a node which is only accessible for the specific node configured.
     
    5. CSAT rating given by users at the end of the conversation is for analytics in Copilot Studio. You can review these ratings and fine tune your Bot. Thumbs up/down given for Generative Answers is used by MS for an LLM model evaluation and learning.
     
    6. As of March, Copilot Studio Generative Answers was using GPT 3.5 Turbo LLM. MS can confirm if they have updated it to GPT 4 or above
     
     
  • jhoffmann2 Profile Picture
    jhoffmann2 10 on at
    SharePoint Online & Generative Answers w/ Search and Summarize Content
    Update: in the meantime, we fixed the AADSTS550011 error. Unfortunately, we messed up redirect URIs ... too bad. 
     
    Any ideas though? Nothing? Some Copilot Studio specialist at Microsoft? 

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Microsoft Kickstarter Events…

Register for Microsoft Kickstarter Events…

Announcing Our 2025 Season 1 Super Users!

A new season of Super Users has arrived, and we are so grateful for the daily…

Announcing Forum Attachment Improvements!

We're excited to announce that attachments for replies in forums and improved…

Leaderboard

#1
WarrenBelz Profile Picture

WarrenBelz 145,495

#2
RandyHayes Profile Picture

RandyHayes 76,287

#3
Pstork1 Profile Picture

Pstork1 64,822

Leaderboard

Featured topics