Skip to main content

Notifications

Building Copilot Studio Chatbots in Microsoft T...
Unanswered

Streaming Text from Copilot Studio to Teams

Posted on by 3

Hi everyone,

 

I'm currently in the process of rolling out GenAI via Teams to my company.

 

I have released two apps so far:

  1. First App: A simple GPT-4 connection for 1-to-1 chat via Teams and Azure OpenAI. For this, I utilized Azure OpenAI Services, CoPilot Studio, and Power Automate to manage aspects like chat history and session-based responses.

  2. Second App: An Azure Prompt Flow Endpoint that utilizes the "Chat with your own data" service.

Both apps employ Power Automate to establish the connection with the LLM.

However, I'm facing two issues:

  1. The responses sometimes take more than 2 minutes, leading to a timeout.
  2. I would like to implement streaming text responses to enhance the user experience.

I have already found the KB article on setting up the Azure ML endpoint: https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-enable-streaming-mode?view=azureml-api-2

 

Setting this up is not a big issue, but how do I adapt the requirements for CoPilot Studio and Power Automate to handle the streamed texts?

 

Has anyone already done this, or is there another way to achieve it?

 

I look forward to your answers, and thank you very much!

Stefan

Categories:
  • jamesbio Profile Picture
    jamesbio 29 on at
    Re: Streaming Text from Copilot Studio to Teams

    Hi @HenryJammes, I went ahead and submitted using the link you shared here.

    Does it need to be reviewed and then updated by staff before showing:
    https://portal.productboard.com/fdeco3ykgkwvchtg4qbdowug/tabs/1-under-consideration

    How do I know if the idea is actually being reviewed? Once it is added, I would like to share with other colleagues and developers at other large companies so they can upvote.

  • ManikandanS Profile Picture
    ManikandanS 238 on at
    Re: Streaming Text from Copilot Studio to Teams

    @Stefan3000 - We are also having similar setup like you I assume, for each user question we have 1 Azure open AI call + Azure Search + 1 Azure open AI call again, please check these areas you should be able to get it.

  • HenryJammes Profile Picture
    HenryJammes on at
    Re: Streaming Text from Copilot Studio to Teams

    Hi @Stefan3000 

     

    Copilot Studio doesn't support streamed messages -- at least not in its current architecture.

    If this is an important requirement, may I ask that you submit a feature request detailing the business case, here: aka.ms/CopilotStudioFeatureRequest?

     

    Henry

  • Stefan3000 Profile Picture
    Stefan3000 3 on at
    Re: Streaming Text from Copilot Studio to Teams

    Hi @ManikandanS ,

     

    thanks for your reply.

    We have a RAG-powered Azure Endpoint in place, which I can't get under 10-15% timeout rate.

    I have found a way to handle long HTTP calls:

    https://sergeluca.wordpress.com/2018/02/01/pattern-for-making-long-async-http-calls-in-microsoft-flow/

     

    Unfortunatly the way which describes how to get the URI of the specific action doesn't work.

     

    Is their a new way to get the URL of the HTTP action, so that I can build a loop to check if the status is 200 or 202?

     

    Thank you!

    Does anybody know how to get the URL of a 

  • ManikandanS Profile Picture
    ManikandanS 238 on at
    Re: Streaming Text from Copilot Studio to Teams

    Hi @Stefan3000 ,

     

    We have also undergone the same situation and don't think we have a way to show streaming response using power automate and co pilot studio. What we did from our side is we checked the other layers of system like Azure Open AI, Azure Search to speed up the overall turnaround time, thus improved our state and now we managed to send response to user question without timeout issues 99% of time. 

Helpful resources

Quick Links

Exciting News for Copilot Studio Communi…

Get ready to experience a whole new level of engagement with the Copilot Studio…

Celebrating the May Super User of the…

LaurensM is an exceptional contributor to the Power Platform Community…

Check out the Copilot Studio Cookbook…

We are excited to announce our new Copilot Cookbook Gallery in the Community…

Leaderboard

#1
renatoromao Profile Picture

renatoromao 6,459

#2
Pstork1 Profile Picture

Pstork1 1,997

#3
Expiscornovus Profile Picture

Expiscornovus 1,708

Leaderboard

Featured topics

Liquid error: parsing "/forums/thread/details/?threadid=%27nvOpzp;%20AND%201=1%20OR%20(%3C%27%22%3EiKO))," - Too many )'s.