Need to implement Response Streaming via copilot studio agent

(1) Share

Report

Posted on by dibhalla Microsoft Employee

We have a requirement where, after receiving data from each skill, the information is sent to a summarizer LLM to enrich insights and establish correlations. This step can sometimes take more than 10 seconds, depending on the input size and output token generation. Therefore, we need to start streaming the response to the user through the CS agent as soon as the summarization begins, so the user is aware of the process currently in progress.

Categories:

Copilot Studio skills development

I have the same question (0)

All responses (2)

Answers (0)

Sort by

Suggested answer

Michael E. Gernaey 53,963 Moderator on at

Like (0)

Report
Copy link

Link copied!

Hi @dibhalla

This really depends on the type of bot implementation you are doing because a straight just "agent" won't specifically do this, even if you enable streaming in your LLM call.

If you use streamable options such as Direct Line you can support incremental responses. Like anything, think of it as chunking the data packets.

I don't know how/where you are embedding this agent but does it use the Bot SDK or is it just a straight CP agent, as the SDK also supports this as long as your front end is written to do so.

Can I ask what your front end is? As that is going to tell us what option(s) we can really attempt to make it work for what you want.

Was this reply helpful? Yes No
AG-19120441-0 Microsoft Employee on at

Like (1)

Report
Copy link

Link copied!

Hey Michael E. Gernaey

We are using Copilot Studio agent, not a Bot SDK implementation. We want to start showing the user incremental/streaming output in the Copilot Studio agent while the summarizer is still generating, so the user knows the process is in progress.

Since this is a straight Copilot Studio agent, could you please confirm what capabilities Copilot Studio currently provides for enabling streaming or progressive responses in this scenario?

If there are any supported patterns or limitations, it would help us plan accordingly.

Was this reply helpful? Yes No