Power Platform Community Forum Thread Details

I'm testing a Copilot Studio Agent I created and not understanding why the same input I use for testing returns a different response so often. At a high level, I have a custom topic that asks a question then uses a GenAI node to evaluate the response.

Here is one of my questions where I'm storing the response in a variable activity_purpose:

What is the goal of this process and why is it being done?

Please describe what the process does (like collecting, analyzing, or reporting data), and explain the goal or benefit of doing it.

Example: “This process analyzes customer feedback to improve product quality and customer satisfaction.”

--------

Here's my GenAI node prompt (I'm using Classic Data, Allow the AI to use its own general knowledge (preview):

Evaluate the following response for clarity and specificity: activity_purpose

A high-quality response must meet ALL of the following criteria:

Clearly describe what the process does (e.g., collects, analyzes, aggregates, or reports data).
Clearly explain why the process is being done (e.g., to support KPIs, improve services, meet compliance).
Be written in complete sentences.

Evaluation Rules:

If the response meets all three criteria, respond with exactly: Looks good
If the response is vague, generic, incomplete, or missing any part, respond with exactly: Needs clarification

Additional Instructions:

Do not assume or infer meaning beyond what is explicitly stated.
Do not include any extra text, punctuation, or follow-up questions.
Ignore spelling or grammar issues unless they affect clarity or meaning.

Examples of valid responses:

“This process analyzes customer feedback to improve product quality and customer satisfaction.”
“We collect and report usage data to support monthly KPI tracking.”
“The process aggregates employee survey results to identify trends and improve engagement.”

Examples of invalid responses:

“To improve things.”
“Analyzes data.”
“To meet compliance.”
“It helps the business.”
“We use it for reporting.”

If I answer the question with "to improve things", sometimes the AI node correctly returns Looks good but other times it's incorrectly accepting the answer as valid.

I've tried numerous things in my GenAI prompt but can not produce a consistent result 10 out of 10 times. What am I doing wrong?

Categories:

General topics