Skip to main content

Notifications

Power Platform Community / Forums / Bot Extensibility / Can PowerAgents unders...
Bot Extensibility
Answered

Can PowerAgents understand audio and video inputs for its natural language functions?

Posted on by 5

I know that we can point PowerAgents to a Sharepoint folder to read and learn about information and then use it's AI Cognitive abilities (Open AI Engine) to respond to questions about that content in a conversational way. 

 

Can I include audio and video files within that Sharepoint folder and will PowerAgents have the ability to process and understand those inputs to then answer questions? Or does it only understand text inputs?

 

I would love to be able to record video screen captures of using our platform, or at least audio recordings, describing and explaining how things work with our actual UI/UX and then have PowerAgents understand that information to answer user queries. 

Categories:
  • davidpauldoyle Profile Picture
    davidpauldoyle 5 on at
    Re: Can PowerAgents understand audio and video inputs for its natural language functions?

    Thank you! I will transcribe for now, but look forward to Azure Cognitive Search having that level of cognitive ability in the future. Will make it's understanding and awareness of our product that much better! Put in a good word to your dev team. 🙂 Thanks.

  • Verified answer
    remidyon Profile Picture
    remidyon on at
    Re: Can PowerAgents understand audio and video inputs for its natural language functions?

    Hi @davidpauldoyle 

    right now the conversation boost can only use text document (word, pdf, sharepoint pages, etc.). Parsing videos and sound in real time and getting an answer would be a performance challenge depending on the size of the file.

    However, you could automate the transcription of your audio files to text and store the transcript back in Sharepoint using power automate, see this article. That way PVA will be able to use that transcript in its answer.

Helpful resources

Quick Links

Exciting News for Copilot Studio Communi…

Get ready to experience a whole new level of engagement with the Copilot Studio…

Celebrating the May Super User of the…

LaurensM is an exceptional contributor to the Power Platform Community…

Check out the Copilot Studio Cookbook…

We are excited to announce our new Copilot Cookbook Gallery in the Community…

Leaderboard

#1
renatoromao Profile Picture

renatoromao 6,459

#2
Pstork1 Profile Picture

Pstork1 1,997

#3
Expiscornovus Profile Picture

Expiscornovus 1,708

Leaderboard

Featured topics