Unanswered

Conversational boosting and Setting "Limit responses to your data content" to false doesn't seem to work

(0) Share

Report

Posted on by RoyBatty

Hi,

In AI Studio we setup an index for some of our internal documents to be used by the chatbot to be used as a support agent. That works great. We've got everything setup in the Conversation boosting topic data source connection properties and it's answering questions related to the documents.

The problem is that it doesn't answer questions from its own knowledge base unrelated to the documents. Let me show an example of of what I'm talking about. Here is Azure OpenAI Studio

There's a checkbox where you can "limit responses to your data content", and if unchecked it'll just answer any random question that it knows of from its knowledge base and that can be seen from the question I gave it.

I would expect PVA to work similarly in that if you "Limit responses to your data" to false in the Model data of connection properties of the data source of create generative answers in Conversational boost. But that's not the case. It won't answer questions like "Who was the 5th president of the United States?" But in the he playground and the website deployed from AI Studio it will answer that question.

Am I missing something here? I'm new to PVA and doing topics, so maybe I have to setup the fallback topic differently because I noticed it ends up there when "Track between Topics" is on and a question not related to the indexed documents is asked.

Thanks

Categories:

General topics

I have the same question (0)

All responses (2)

Answers (0)

Jameslew_MSFT on at

Like (0)

Report

PVA explicitly does work to ensure that the answer it generates seems to be present in the content provided to it by the data source. For example, in a public data source, PVA queries Bing, gets back results (blue links & snippets), has GPT generate a summary from there, and then validates that what the GPT generated seems to be present in the links returned from the search.

Have you tried adjusting moderation settings in PVA to low to see if you get any different result?

Was this reply helpful? Yes No
RoyBatty on at

Like (0)

Report

With the "Limit responses to your data content" property set to false, I would expect the behavior to be that if it couldn't generate an answer from the content, to fallback to whatever GPT can come up with in its internal knowledge base. That's how the website that is deployed from Azure OpenAI Studio works as can be seen in the first image of my above post. I believe we tried turning moderation all the way down and same results.

Was this reply helpful? Yes No