web
You’re offline. This is a read only version of the page.
close
Skip to main content
Community site session details

Community site session details

Session Id : w7Pl037MPxmWxWxlBR/Gid
Copilot Studio - General
Answered

Including conversation history in generative answers

Like (0) ShareShare
ReportReport
Posted on 17 Jan 2024 16:58:01 by

Hi all,

 

The support for generative answers is really great, but seems to have a major flaw: conversation history is not factored in. Take a simple example:

 

Me: How do I do x when y is true?

Answer: [ good answer ]

Me: What about when z is true?

Answer: [ totally incorrect answer because the initial question and response don't seem to be included: the follow-up question is used without any context ]

 

Because the conversational UI/UX so strongly implies that you're conducting a multi-turn conversation, the lack of context can lead to results that are both terrible and very confusing for the user. It feels like a fatal flaw in the implementation.

 

Is there some kind of workaround, to include conversation history somehow in fallback generative answers?

 

Please let me know.

 

Thank you!

 

Chris

 

Categories:
  • grjacob Profile Picture
    19 on 29 May 2024 at 09:07:49
    Re: Including conversation history in generative answers

    Conversation history is still working fine for us.

     

    Example scenario:

    1. "Define <some concept that the bot has knowledge of>."
    2. "Explain in more detail."
    3. "Summarise the main points."

     

    The second and third messages are checking whether the bot can refer back to previous turns. Tested this morning, all good. đꤖ

     

    Something that can disrupt the conversation is other 'topics'. You might accidentally trigger another topic and find it hard to get back to your gen AI conversation. If you trigger the 'Multiple topics matched' topic you may end up in the 'Fallback' topic (if you use default topic config) and it feels to the user like the AI just can't answer your question, when actually it might have been able to.

    I guess if we want to focus on generative AI conversations then maybe we should avoid using other topics, or keep their triggers conceptually narrow, to reduce the risk of triggering them when not needed?

  • ISD Development Profile Picture
    28 on 28 May 2024 at 17:52:42
    Re: Including conversation history in generative answers

    We seem to be having related issues still now 5/28/2024, is anybody still having issues with conversational history? any work arounds?

  • grjacob Profile Picture
    19 on 10 Apr 2024 at 12:09:33
    Re: Including conversation history in generative answers

    I am working on a project that is using Copilot Studio.

     

    In our experience, Copilot temporarily lost the ability to use conversation history around 6 to 8 weeks ago, roughly. It was answering every question from scratch. It had no knowledge of a 'previous answer'.

     

    Then, it regained conversation history around 2 weeks ago.

     

    Perhaps it was briefly disabled, while MS made some fixes? It was possible for the bot to go quite far off-topic before, but the functionality does seem a bit better since they re-enabled the conversation history.

    It would be nice to understand what was going on.

  • HenryJammes Profile Picture
    on 21 Feb 2024 at 13:32:23
    Re: Including conversation history in generative answers

    @Surlig1213 yes it should

  • OOO1337777 Profile Picture
    37 on 21 Feb 2024 at 02:39:56
    Re: Including conversation history in generative answers

    maybe maybe not... these things are highly buggy and not reliable at the moment...

     

    for example, the answer sent from a dynamically chained plugin is something different from a classic message sent to the user...  so it has to be tested what exactly is included in the conversation history sent to the Generative answers node

  • Surlig1213 Profile Picture
    18 on 20 Feb 2024 at 23:23:05
    Re: Including conversation history in generative answers

    @HenryJammes  If I use Custom data from Jira Service Management API(get the knowledge base) via Conversational boosting topic. Do the last 10 turns to ground follow-up questions still work?

  • HenryJammes Profile Picture
    on 22 Jan 2024 at 23:44:50
    Re: Including conversation history in generative answers

    Hi @chrislrobert, the behavior has indeed changed since then, and we do use the last 10 turns to ground follow up questions 🙂

    If you have examples to share with a public website data source (the easiest one to repro), feel free to share them in IM.

  • chrislrobert Profile Picture
    9 on 22 Jan 2024 at 13:26:57
    Re: Including conversation history in generative answers

    So Henry, I've continued to have inconsistent results here. And I found this post that explicitly says "Single versus multi-turn answers: The boost conversation capability in PVA does not remember context across multiple questions in the conversation. Each boosted conversation post will consider as context only the URL and the latest query, not on the entire conversation. Make sure to provide sufficient context in each query instead of assuming the feature knows about the prior turn."

     

    https://powerusers.microsoft.com/t5/General/Tips-amp-tricks-for-using-the-Power-Virtual-Agents-Boost/m-p/2045836

     

    That was from earlier last year, so maybe this behavior has changed? But like I said, in my own testing, results are quite inconsistent, with context sometimes seeming to be included and sometimes not.

     

    Thanks,

     

    Chris

  • Community Power Platform Member Profile Picture
    on 17 Jan 2024 at 17:41:23
    Re: Including conversation history in generative answers

    I just re-tested and you're right! I'm sorry. Is this something that recently changed? I'm using the same sequence I'd tested with three weeks ago, but perhaps there was something subtle that had reset the conversation in that earlier case.

     

    Thanks for the quick clarification!

     

    Chris

     

  • Verified answer
    HenryJammes Profile Picture
    on 17 Jan 2024 at 17:10:16
    Re: Including conversation history in generative answers

    That's odd. Generative answers actually keeps context of the last 10 turns.

    So, follow-up questions are interpreted in the context of previous questions and answers.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Announcing our 2025 Season 2 Super Users!

A new season of Super Users has arrived, and we are so grateful for…

Paul Stork – Community Spotlight

We are honored to recognize Paul Stork as our July 2025 Community…

Congratulations to the June Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > Copilot Studio

#1
Michael E. Gernaey Profile Picture

Michael E. Gernaey 169 Super User 2025 Season 2

#2
Romain The Low-Code Bearded Bear Profile Picture

Romain The Low-Code... 150

#3
sandeep_angara Profile Picture

sandeep_angara 75 Super User 2025 Season 2

Loading complete