web
You’re offline. This is a read only version of the page.
close
Skip to main content

Announcements

News and Announcements icon
Community site session details

Community site session details

Session Id :
Power Platform Community / Forums / Copilot Studio / Generative Answers nod...
Copilot Studio
Suggested Answer

Generative Answers node failing to reference all uploaded Knowledge sources

(1) ShareShare
ReportReport
Posted on by 14
I made a "Create generative answer" action / node. I clicked on "Search only selected sources" and referenced the following 6 documents:
1. Specific.txt
2. Measurable.txt
3. Achievable.txt
4. Relevance.txt
5. Time-Bound.txt
6. SMART_Score.txt
 
 
however when I click on "Test" and go through the flow, I only see 2 knowledge sources referenced: SMART_Score.txt and Relevance.txt.
 
 
My question is: Why? My prompt for this action / node was this: """Run the SMART analysis on this target: 
​ strictly using the definitions and scoring rules from the knowledge base.
Action:
Score Specific using only written evidence strictly using the definitions and scoring rules in the document "Specific.txt" in your knowledge base
Score Measurable using only written evidence strictly using the definitions and scoring rules in the document "Measurable.txt" in your knowledge base
Score Achievable using only written evidence strictly using the definitions and scoring rules in the document "Achievable.txt" in your knowledge base
Score Relevance using only written evidence strictly using the definitions and scoring rules in the document "Relevance.txt" in your knowledge base
Score Time-Bound using only written evidence strictly using the definitions and scoring rules in the document "Time-Bound.txt" in your knowledge base
Calculate the average SMART Score at the top of the output adhering to the rules of "SMART_Score.txt"
Do not speak to the user. Do not say 'Based on the documents...' or 'Here is your score.'
Your output must be ONLY a raw data string in this format: S:[score], M:[score], A:[score], R:[score], T:[score], AVG:[average]"""

 
As you may see I specifically told the LLM to use all 6 .txt files, and not just the ones it sees fit. So what is going on here? Can someone explain? And how do I make sure that all my knowledge sources are used?
  • MichaelFP Profile Picture
    2,008 Moderator on at
    If you do not put anything on the instructions it shows all documents?

    The documents have the information provided? Because RAG is not to calculate values.
     
  • Suggested answer
    Isamu Ueda Profile Picture
    10 on at
    Hi,
    Naming files in the prompt does not control retrieval.
    The node ranks passages from the selected sources and the model writes from what came back, so the citations show what actually contributed.
    Two of six is normal, not the node ignoring your selection.

    Your six files all define scoring rules in similar language, so they compete for the same slots.

    Either put the five criteria in one document, so a single retrieval brings back the whole rubric, or call generative answers once per criterion, scoped to one file each.
    I drafted this with AI help.

    If this helped, please check the box Does this answer your question?
  • Suggested answer
    chiaraalina Profile Picture
    2,553 Super User 2026 Season 2 on at
     

    I think the main issue is how the Generative Answers node works.

    When you enable Search only selected sources and select several uploaded files, this limits the Generative Answers node to those sources.

    But it does not mean that the node will use content from every selected file.

    The Generative Answers node still uses relevance-based retrieval. It searches the selected sources and retrieves the content it considers most relevant for the current question.

    So even if you select six files and write in the instructions that all six should be used, this does not guarantee that content from all six files will actually be retrieved.

    This could be a possible solution:

    Instead of using one Generative Answers node with all six files, you can create six separate Custom Search (Perform a custom search node under Tools) steps.

     

    It means you stop asking one Generative Answers node to decide which files are relevant.

    With six separate Custom Search steps, each search is restricted to one specific file. So you force Copilot Studio to retrieve information for every required SMART dimension separately.

     

    I quickly tested it with three files and it seems to work:

    If you want to try you can add the other steps for the other files:

    kind: AdaptiveDialog
    modelDescription: Evaluate my SMART goal
    beginDialog:
      kind: OnRecognizedIntent
      id: main
      intent: {}
      actions:
        - kind: Question
          id: question_q5TLlo
          interruptionPolicy:
            allowInterruption: false
     
          variable: init:Topic.Goal
          prompt: What goal would you like me to evaluate using the SMART framework?
          entity: StringPrebuiltEntity
     
        - kind: SearchKnowledgeSources
          id: searchKnowledgeSources_a9E3Yw
          displayName: Custom search Specific
          fileSearchDataSource:
            searchFilesMode:
              kind: SearchSpecificFiles
              files:
                - cr38e_sdas.file.Specific.txt_PX-
     
          knowledgeSources:
            kind: SearchSpecificKnowledgeSources
     
          result: Topic.SpecificResults
          userInput: Retrieve the definition, evaluation criteria, scoring rules and relevant guidance for evaluating the Specific dimension of a SMART goal.
     
        - kind: SearchKnowledgeSources
          id: searchKnowledgeSources_XXak2r
          displayName: Custom search Measurable
          fileSearchDataSource:
            searchFilesMode:
              kind: SearchSpecificFiles
              files:
                - cr38e_sdas.file.Measurable.txt_mAR
     
          knowledgeSources:
            kind: SearchSpecificKnowledgeSources
     
          result: Topic.MeasurableResults
          userInput: Retrieve the definition, evaluation criteria, scoring rules and relevant guidance for evaluating the Measurable dimension of a SMART goal.
     
        - kind: SearchKnowledgeSources
          id: searchKnowledgeSources_4UUWd5
          displayName: Custom search Achievable
          fileSearchDataSource:
            searchFilesMode:
              kind: SearchSpecificFiles
              files:
                - cr38e_sdas.file.Achievable.txt_uPX
     
          knowledgeSources:
            kind: SearchSpecificKnowledgeSources
     
          result: Topic.AchievableResults
          userInput: Retrieve the definition, evaluation criteria, scoring rules and relevant guidance for evaluating the Achievable dimension of a SMART goal.
     
        - kind: SetVariable
          id: setVariable_ovtTNp
          displayName: Set variable value SpecificContext
          variable: Topic.SpecificContext
          value: |-
            =Concat(
                Topic.SpecificResults,
                Content & Char(10) & Char(10)
            )
     
        - kind: SetVariable
          id: setVariable_zcw5nj
          displayName: Set variable value MeasurableContext
          variable: Topic.MeasurableContext
          value: |-
            =Concat(
                Topic.MeasurableResults,
                Content & Char(10) & Char(10)
            )
     
        - kind: SetVariable
          id: setVariable_31P5oz
          displayName: Set variable value AchievableContext
          variable: Topic.AchievableContext
          value: |-
            =Concat(
                Topic.AchievableResults,
                Content & Char(10) & Char(10)
            )
     
        - kind: InvokeAIBuilderModelAction
          id: invokeAIBuilderModelAction_4JYxhX
          input:
            binding:
              AchievableGuidance: =Topic.AchievableContext
              Goal: =Topic.Goal
              MeasurableGuidance: =Topic.MeasurableContext
              SpecificGuidance: =Topic.SpecificContext
     
          output:
            binding:
              predictionOutput: Topic.SMARTResult
     
          aIModelId: 0458edc5-d697-45c0-b997-b7417a2f79af
     
        - kind: SetVariable
          id: setVariable_ftbJA7
          displayName: Calculate SMART average
          variable: Topic.AverageScore
          value: |-
            =Round(
                (
                    Topic.SMARTResult.structuredOutput.specificScore +
                    Topic.SMARTResult.structuredOutput.measurableScore +
                    Topic.SMARTResult.structuredOutput.achievableScore
                ) / 3,
                2
            )
     
        - kind: SendActivity
          id: sendActivity_SMARTResult
          activity: |-
            SMART Evaluation
     
            Specific: {Topic.SMARTResult.structuredOutput.specificScore}
            {Topic.SMARTResult.structuredOutput.specificReason}
     
            Measurable: {Topic.SMARTResult.structuredOutput.measurableScore}
            {Topic.SMARTResult.structuredOutput.measurableReason}
     
            Achievable: {Topic.SMARTResult.structuredOutput.achievableScore}
            {Topic.SMARTResult.structuredOutput.achievableReason}
     
            Average score: {Topic.AverageScore}
     
        - kind: EndDialog
          id: endDialog_SMARTEvaluation
          clearTopicQueue: true
     
    inputType: {}
    outputType: {}

    Hope it helps, and let me know if it worked, if you try this solution!

  • Suggested answer
    sannavajjala87 Profile Picture
    1,113 Super User 2026 Season 2 on at
    I agree with @chiaraalina The Generative Answers node uses retrieval-based grounding, so selecting six files only limits the search scope to those documents. It does not force the model to retrieve content from all six. If you need guaranteed use of each document, a better pattern is to perform separate searches against each file and then combine the results. That gives you deterministic retrieval instead of relying on the relevance ranking used by a single Generative Answers node.
     
    Thanks & Regards,
    Manoj Annavajjala

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Season of Sharing Community Challenge Winners!

Congratulations to our community stars!

Kudos to our 2025 Community Spotlight Honorees

Expanding mentorship, skilling, and AI innovation

Leaderboard > Copilot Studio

#1
Mohsin Ali Profile Picture

Mohsin Ali 356

#2
Valantis Profile Picture

Valantis 253 Super User 2026 Season 2

#3
11manish Profile Picture

11manish 179 Super User 2026 Season 2

Last 30 days Overall leaderboard