Answered

Can PowerAgents understand audio and video inputs for its natural language functions?

(0) Share

Report

Posted on by davidpauldoyle

I know that we can point PowerAgents to a Sharepoint folder to read and learn about information and then use it's AI Cognitive abilities (Open AI Engine) to respond to questions about that content in a conversational way.

Can I include audio and video files within that Sharepoint folder and will PowerAgents have the ability to process and understand those inputs to then answer questions? Or does it only understand text inputs?

I would love to be able to record video screen captures of using our platform, or at least audio recordings, describing and explaining how things work with our actual UI/UX and then have PowerAgents understand that information to answer user queries.

Categories:

Bot extensibility

I have the same question (0)

All responses (2)

Answers (1)

Sort by

Verified answer

remidyon Microsoft Employee on at

Like (2)

Report
Copy link

Link copied!

Hi @davidpauldoyle
right now the conversation boost can only use text document (word, pdf, sharepoint pages, etc.). Parsing videos and sound in real time and getting an answer would be a performance challenge depending on the size of the file.
However, you could automate the transcription of your audio files to text and store the transcript back in Sharepoint using power automate, see this article. That way PVA will be able to use that transcript in its answer.

Was this reply helpful? Yes No
davidpauldoyle 5 on at

Like (0)

Report
Copy link

Link copied!

Thank you! I will transcribe for now, but look forward to Azure Cognitive Search having that level of cognitive ability in the future. Will make it's understanding and awareness of our product that much better! Put in a good word to your dev team. 🙂 Thanks.

Was this reply helpful? Yes No