Unanswered

Issue with PVA Analytics

(0) Share

Report

Posted on by Community Power Pla...

Hi all,

We noticed that PVA Analytics capture testbot sessions in the analytics as well which is kind of weird. You could test it by entering a question in the Test bot window and it would appear as a session in the sessions excel file (downloaded via Sessions Tab in Analytics Menu). Is there a way to separate the actual sessions triggered via the demo website and the ones started from the test bot window (typically not needed as they are for testing purposes only).

I am thinking to get ChatTranscripts file from Power Admin Center to see if there's any indication in there.

Please share your views, suggestions, and any other similar findings with analytics

Thanks

Categories:

Bot analytics

I have the same question (0)

All responses (5)

Answers (0)

CU22081450-0 Most Valuable Professional on at

Like (0)

Report

Hi @Anonymous ,

I don't think so, look at this documentation here: View billed sessions - Power Virtual Agents | Microsoft Docs

If you are looking for a different way for reports, you can create your idea here: https://aka.ms/pva/community/ideas

Sessions that use the embedded test chat or that only leverage capabilities included with the Power Virtual Agents for Microsoft Teams plan don't count toward the billed sessions. The Billed Session report doesn't distinguish between the sessions included in the Power Virtual Agents for Microsoft Teams plan and those that are not. You might see an inflated number of billed sessions in the report.

Was this reply helpful? Yes No
Community Power Pla... on at

Like (0)

Report

I have read this before and its kinda surprising for me too, please cross-check and let me know if you are experiencing the same.
Thanks

Was this reply helpful? Yes No
CU22081450-0 Most Valuable Professional on at

Like (0)

Report

Hi @Anonymous ,

Yes, unfortunately, same behaviour here. 😞

Was this reply helpful? Yes No
mboninco on at

Like (0)

Report

Hello @Anonymous @renatoromao

Except for Billed Sessions, the analytics page includes sessions from PVA's test pane. The thinking is this would help users discover how analytics work and aide in the development for their bot. In most cases the sessions from the test pane should be very small in comparison to those from the deployed bot.

The conversation transcripts which can be downloaded from Dataverse contain a flag IsDesignMode that can be used to differentiate sessions that are from the test pane.
Mike Bonincontri
Senior Program Manager
Microsoft

Was this reply helpful? Yes No
Community Power Pla... on at

Like (0)

Report

Thanks @mboninco for your reply,
Yeah, I scanned the Chattranscripts file very carefully yesterday and found that channelId is null for test sessions and IsDesignMode is true. However, the reason for merging test sessions with the actual billed sessions is still not convincing. As a chatbot manager, one would be interested in the actual analytics, not the test ones.

Cheers
Kanwal

Was this reply helpful? Yes No