Copilot Studio Kit - Agent Configuration Settings

(2) Share

Report

Posted on by RS-04061405-0

Hey Community,

I am trying to register my agent in Copilot Kit for Evaluation & Observability. When I try to create a Generative Agent created in Copilot Studio, I have to add a bunch of details like Dataverse URL, AgentID etc. As I am not an IT Infra guy, I am finding it hard to understand what settings to apply for Direct Line. The agent is configured currently to authenticate using "Authenticate with Microsoft" option.

I was able to fetch dataverse URL as well as Agent iD but I don't know what options to select for direct line and user authentication too.

Thanks in advance for helping

Categories:

General topics

I have the same question (0)

All responses (2)

Answers (0)

Suggested answer

stampcoin 5,058 Super User 2025 Season 2 on at

Like (0)

Report

@RS-04061405-0 Hi,

Those Articles will give you more details regarding this.

Configure web and Direct Line channel security

About Direct Line

Was this reply helpful? Yes No
psimolin Microsoft Employee on at

Like (0)

Report

Hello,

At this time, Copilot Studio Kit does not support "Authenticate with Microsoft"-option. We are investigating how to add support for this. No authentication and Manual authentication with Entra ID v2 with SSO work.

I recommend creating a new (temporary) "hello world" type of agent with no authentication and configure that in the Copilot Studio Kit agent configuration. Create a test set, one simple "helllo world" Response Match test case and start a test run just to make sure that everything works as expected. i.e. the Kit is able to reach your agent and is able to get responses back.

Minimum setup only needs the Base configuration and Direct Line settings. Everything else can be left disabled. If you enable "Channel security" from "Direct Line Settings", you can copy paste the secret from your Copilot Studio Agent "Settings" -> "Security" -> "Web channel security" and pick either secret 1 or secret 2.

Here are more details on the settings:

Power-CAT-Copilot-Studio-Kit/CONFIGURE_COPILOTS.md at main · microsoft/Power-CAT-Copilot-Studio-Kit

User Authentication section -> if your agent requires authentication (only manual authentication entra id v2 + sso supported for now)

Results Enrichment -> this is required for generative answers

Conversation Transcript Enrichment -> this is required for topic match

Generative AI testing -> this is required for generative answers

Best regards,

Petri Simolin

PM of Copilot Studio Kit

Was this reply helpful? Yes No