Power Platform Community Forum Thread Details

Copilot Studio Agent: Why Does It Start With the Right Answer and Then Switch Mid-Response?

Hi everyone,

I’ve built an agent in Copilot Studio that uses SharePoint and a website as its knowledge base. The agent uses generative orchestration for its flow.

Here’s the issue: Sometimes, when I ask a question, the agent starts answering correctly (I can see the right answer being streamed), but then mid-response it switches and gives a wrong or incomplete final answer. In the test panel, I notice this happens when conversational boosting kicks in.

From what I understand, conversational boosting is a fallback when orchestration fails. But I don’t know why orchestration fails in these cases, especially since it starts off correctly.

My questions:

Why does the agent begin with the correct answer and then switch mid-response?
What causes generative orchestration to fail and trigger conversational boosting?
How can I prevent this fallback and ensure the agent sticks to the orchestrated answer?

For context, here’s an example of my system prompt (company name anonymized as X and document types as Y and Z):

YAML

# PURPOSE
Your mission is to answer users’ questions about X using Y and Z documents.

# RESPONSE CONTRACT
- Tone: Professional, clear, and concise.

# RESPONSE FORMAT
1. Answer:
   - Provide a clear answer relevant to the question (do not write “Answer:” as a label).
2. Source:
   - Include excerpts that were used to generate the answer.
3. Disclaimer:
   - Always include:
     - *This response was generated by an AI assistant based solely on X’s official Y and Z documents. Please verify the information provided by reviewing the cited sources, as this content was generated using AI and may require human validation.*

# EXAMPLES TO SIMULATE
User: "Here I give the agent an example of a question"
Your answer: Here I give the agent an example of an answer
Source:
- "here I give an example of the text chunk"

Disclaimer...

Has anyone experienced this behavior? Any ideas on why orchestration fails and how to avoid the fallback?

Thanks!

This happens because generative orchestration begins answering, but if the orchestration pipeline fails (e.g., timeout, retrieval error, or ranking mismatch), Copilot Studio falls back to conversational boosting mid-stream. Boosting uses a simpler retrieval approach, so the final answer can differ from the initial streamed content.

Common causes of orchestration failure:

Latency or timeout in knowledge source calls (SharePoint/website).
Ranking conflicts when multiple sources return similar scores.
Prompt complexity or overly strict response contract causing orchestration to abort.
Dynamic chaining enabled with generative actions sometimes triggers fallback when orchestration can’t resolve all steps.

How to prevent fallback:

Optimize orchestration:

Reduce prompt complexity (avoid excessive examples or rigid formatting).
Ensure knowledge sources are healthy and fast (check SharePoint indexing and site connectivity).
Disable conversational boosting: In Settings → Generative AI, turn off Conversational boosting so the agent sticks to orchestration even if partial failure occurs.
Enable Enhanced search results for SharePoint and web sources to improve retrieval stability.
Test with dynamic chaining off if enabled; this reduces orchestration complexity.

If orchestration still fails intermittently, capture Conversation ID and raise a support ticket—Microsoft recommends this for orchestration fallback issues.

Quick Links