Power Platform Community Forum Thread Details

I recently used the new Evaluation feature in Copilot Studio for my Catering Service Assistant, and it was a real eye-opener.

My initial test run scored 60%, which seemed okay at first but the detailed results told a different story.

The bot performed well on basic support questions (order tracking, cancellations)

But failed on important scenarios like:
- Allergen information
- Vegan/vegetarian options
- Same-day catering
- Corporate packages

This helped me realize that my bot was handling general queries but lacked depth in business-specific knowledge.

The Evaluation feature helped me move from manual, biased testing to a more structured and data-driven approach.

Instead of guessing what works, I now have clear insights into:

Highly recommend using Evaluation early - it really highlights gaps you might otherwise miss.

Would love to hear how others are using it!

Categories:

How Evaluation Helped Me Improve My Catering Service Assistant