Inconsistent behavior when using GPT-5 Chat in test mode and published channel (Teams)

(2) Share

Report

Posted on by KW-18110402-0

Hi community,

I just want to ask about a strange behavior when I am using Copilot Studio with GPT-5-Chat.

When i test my copilot studio with debug (testing) mode with GPT-5-Chat, it work normally, but when I published to Teams channel, it always return "I am sorry" message that it couldn't return an answer. But when i switch back to GPT-4.1, the behavior is normal and consistent for both testing in debug mode and published in Teams channel. May I know if anyone encountered the same and how to fix it?

Categories:

Publish & channel management

General topics

All responses (5)

Answers (0)

Sort by

Suggested answer

11manish 4,802 Super User 2026 Season 2 on at

Like
a
(1)

Report
Copy link

Link copied!

This behavior happens because GPT-5 Chat is more strict and sensitive to permission, data access, and tool execution failures in the Teams

channel compared to test mode.

While test mode runs with maker-level access, Teams runs with end-user or bot identity, which often causes connectors, flows, or knowledge

sources to fail. GPT-5 then returns a fallback “I’m sorry” message instead of generating an incomplete answer.

The fix is to validate permissions, connections, DLP policies, and tool accessibility in the Teams channel, and isolate whether the issue is

caused by external dependencies.
Suggested answer

Haque 4,280 Super User 2026 Season 2 on at

Like
a
(2)

Report
Copy link

Link copied!
Hi @KW-18110402-0,

Ah mix of good news and bad news with GPT-5 and GPT 4.1: GPT-5 Chat is more advanced and precise but can be more sensitive to environment configuration, permissions, or runtime conditions in production (Teams) compared to debug mode. GPT-4.1 is more stable and consistent but less precise.

Here are some common causes for GPT-5 Chat Issues in Production

Permissions and Auth: The published bot may lack proper permissions or consent to access required resources or APIs.

Configuration Differences: Missing environment variables, connection references, or API keys in production.

Timeouts or Throttling: Longer processing times or throttling in production.

Licensing or Quota Limits: Usage limits or licensing restrictions in production.

Error Handling: Stricter fallback logic returning generic apology messages.

Where to Troubleshoot

Verify all connections and permissions for the agent and GPT-5 Chat in the production environment.

Check environment variables and secrets for completeness.

Review Power Platform and Azure AD logs for authentication or permission errors.

Test the published bot with minimal queries to isolate issues.

Monitor API usage and quotas for GPT-5 Chat.

Add detailed logging or telemetry to capture error details.

Implement fallback or retry logic to handle transient errors.

Trick for Balancing Precision and Stability

Use GPT-5 Chat for complex or precision-required queries.

Use GPT-4.1 as a fallback for general or conversational queries.

I am sure some clues I tried to give. If these clues help to resolve the issue brought you by here, please don't forget to check the box Does this answer your question? At the same time, I am pretty sure you have liked the response!
KW-18110402-0 54 on at

Like
a
(0)

Report
Copy link

Link copied!

just some more background on my chatbot, it is a simple RAG chatbot using SharePoint as knowledge source without calling any tool, all permissions work well with GPT-4.1 and license shouldn't be an issue if it works with GPT-4.1?
HB-09041311-0 47 on at

Like
a
(0)

Report
Copy link

Link copied!

Hi, thanks for sharing this issue.

We are currently experiencing very similar behavior with GPT-5 Chat where the agent works correctly in Copilot Studio test mode but gives 'Something went wrong' error after publishing to Teams.

Has anyone found a root cause or reliable resolution for this? It would be great if you could share any findings, Microsoft responses, workarounds, or configuration changes that helped resolve the issue.

Thanks in advance!
Suggested answer

M Bilal Khan 376 on at

Like
a
(0)

Report
Copy link

Link copied!
The fact that GPT-4.1 works consistently in both the test pane and Teams, while GPT-5 Chat works in testing but fails in Teams, is the key clue.

I would not immediately assume the prompt or agent configuration is wrong. This looks more like a model/channel-specific issue.

A few things I would check:

Create a minimal test topic/agent using GPT-5 Chat

Remove knowledge sources, tools, flows, and complex instructions temporarily and test something simple such as:

"What is 2 + 2?"

Publish that to Teams.

If GPT-5 still returns the generic "I'm sorry, I couldn't return an answer" message in Teams while the same minimal agent works in the test pane, you've isolated the issue to the published channel/model path.

Check the published version

Make sure the GPT-5 configuration was saved and the agent was republished after changing the model. Also start a new Teams conversation rather than continuing an older conversation, since conversation state can sometimes make testing misleading.

Compare the same prompt across channels

Test:

GPT-5 Chat → Copilot Studio test GPT-5 Chat → Teams GPT-4.1 → Copilot Studio test GPT-4.1 → Teams

If only GPT-5 + Teams fails, that is valuable evidence for a Microsoft support case.

Check whether tools/knowledge are involved

If the minimal GPT-5 agent works in Teams, add the knowledge sources and actions back one at a time. A model may behave differently once orchestration, grounding, or tool calls are involved.

I would also capture the Conversation ID / activity details and timestamp from a failed Teams interaction if available. The generic "I'm sorry" response isn't very diagnostic by itself; the backend telemetry is much more useful.

I wouldn't downgrade permanently to GPT-4.1 based only on this test, but I would use GPT-4.1 as the control case. Since it works with the same agent and Teams channel, that makes it a particularly useful comparison.

If GPT-5 consistently fails with even a brand-new minimal agent in Teams, while GPT-4.1 succeeds, I'd raise it with Microsoft as:

GPT-5 Chat succeeds in Copilot Studio test mode but fails consistently in the Teams published channel; GPT-4.1 succeeds in both channels.

That gives Microsoft a much cleaner reproduction than simply reporting that GPT-5 returns an error.