Microsoft Copilot Studio Agent Image generation with annotation after reading the original image

(1) Share

Report

Posted on by AD-01081101-0

Hi All,

We have a requirement of Microsoft Copilot Studio Agent Image generation with annotation after reading the original image.

User will upload an image in copilot studio agent prompt along with the user prompt.

Copilot Studio agent need to read the image, analyze the image and need to generate the image as an agent response output with annotation.

Please suggest if this is technically feasible to achieve this requirement by building a Microsoft Copilot Studio custom Agent.

Regards.

Avinash.

Categories:

Bot extensibility

I have the same question (0)

All responses (3)

Answers (0)

Sort by

Suggested answer

AP-26031104-0 Microsoft Employee on at

Like (1)

Report
Copy link

Link copied!
Yes, this requirement is technically feasible, but it cannot be achieved using Microsoft Copilot Studio alone. It will require integration with Azure AI services and a custom backend for image processing.

At a high level, the approach would be:

Use Microsoft Copilot Studio to handle user interaction and file input

Pass the uploaded image to Azure AI Vision or Azure OpenAI Service for image analysis (object detection, OCR, etc.)

Implement a custom API (e.g., Azure Function) to generate annotated images (drawing bounding boxes, labels, highlights)

Store the processed image (e.g., in Blob Storage) and return the image URL back to Copilot Studio for display to the user

Key point: Copilot Studio can orchestrate the flow but does not natively support image annotation or generation. That logic must be handled externally.

Please consider factors like response latency, channel support for file uploads, and accuracy of image detection while designing the solution.

Image Generator Capability for Declarative Agents for Microsoft 365 Copilot | Microsoft Learn

Was this reply helpful? Yes No
AP-26031104-0 Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

Just following up on the answer I shared recently. Could you please let me know if it resolved your issue, or if you need any further clarification?

Was this reply helpful? Yes No
Suggested answer

AD-01081101-0 66 on at

Like (0)

Report
Copy link

Link copied!

@AP-26031104-0 Thank you for your quick suggestion and reply.

Was this reply helpful? Yes No