creating a questioning strucure that produces great answers

(2) Share

Report

Posted on by CU13051508-0

Hi, I’m creating an agent that searches multiple documents. Before searching, I want it to ask clarifying questions to understand the user’s specific context and intent.

Our documentation is highly context-dependent. The same question may have different answers depending on the process, department, or situation.

Has anyone developed an instruction structure that reliably guides users through clarifying questions before the agent searches its knowledge sources? If so, would you be willing to share the structure or approach you used?

Thanks,

Brad

Categories:

Building Copilot Studio chatbots in Microsoft Teams

General topics

Copilot Studio skills development

Topic creation & management

All responses (7)

Answers (0)

Sort by

Suggested answer

11manish 4,802 Super User 2026 Season 2 on at

Like
a
(0)

Report
Copy link

Link copied!

use Clarify → Capture context → Validate context → Retrieve → Answer, rather than Question → Search → Hope the model figures out the context.

For your scenario, this will also make the agent considerably easier to test because you can create test cases such as same question + different department = different expected answer.
Suggested answer

Haque 4,280 Super User 2026 Season 2 on at

Like
a
(0)

Report
Copy link

Link copied!
Hi @CU13051508-0,

That's the beauty of the agent that asks the questions to narrow down the context so that intent can be significanlty precise to make the answers. We can follow some approaches where we can decorate the agent that will ask to clarify before searching multiple documents - more specifically when documentation or knowledge source is highly context-dependent.

Step-1: Lets make intellegent the agent by defining contextual dimensions - meaning, if the agent can identify the contextual factors that influence the answers, like as Process Type, Department, Situation, User role and even location would be useful.

Step-2: Nice to have a goal oriented topic flow - initial conversation flow can be simple and narrow done the context. Like greets the user, based on the contextual dimension identified clarify the targeted questions. Finally, collects user responses to narrow down the context. To set an example:

"Which department are you referring to?"

"Is this related to process A or process B?"

"Can you specify the situation or scenario?"

Step-3: Decorating context variable will be advantageous. Let's store the user's answers to these clarifying questions in context variables or memeory slots within the agent.

Step-4: As we have proposed to decorate context varaible, what we can do now is invoke knowledge source conditionally meaning we can configure knowledge source or documents searches to be triggered only after the relevant context varaibles are set. This can be done by using topic conditions that check for context variables before invoking document searce and setting knowledge sources to "Only when referenced by topics" and referencing them in the context-aware topics.

Step-5: Construct query dynamically - let's use the collected context to dynamically tailor the search query or filter the documents, ensuring the retrieval is relevant to the user’s specific context. Dynamic chaining is one area that can help.

Step-6: If the initial search results are ambiguous or insufficient, design the agent to ask follow-up clarifying questions to refine the context further. Build an iterative evaluation framework in four stages.

So the general structure should be like this:

Welcome Topic: "Hi! To help you better, I need to ask a few questions."

Clarification Topic

Ask: "Which department is this about?"

Store answer in context.department

Ask: "Which process are you referring to?"

Store answer in context.process

Ask: "What situation or scenario applies?"

Store answer in context.scenario

Search Invocation Topic

Condition: Only trigger if all the context variables are set. Use these variables to filter or scope the document search

Answer Delivery: Present the retrieved, context-aware answer to the user

References:

Design a question node - ask a question.

Disambiguate customer intent

Design effective language understanding

I am sure some clues I tried to give. If these clues help to resolve the issue brought you by here, please don't forget to check the box Does this answer your question? At the same time, I am pretty sure you have liked the response!
Suggested answer

chiaraalina 2,553 Super User 2026 Season 2 on at

Like
a
(1)

Report
Copy link

Link copied!
Hi @CU13051508-0

I would use a topic with inputs for the important context: Department, Process or Situation.

The reason is that with generative orchestration, the agent can try to get these values from the user's message first.

For example, if the user asks:

"I work in Finance. How do I get a purchase approved?"

the agent can already identify Finance (Department) and does not need to ask for the department again

But if the user only says:

"How do I get a purchase approved?"

the department is missing. The agent can then ask a clarifying question before continuing.

Another example:

"I work in Finance and want to buy software for CHF 3000. Who approves it?"

In this case, the agent can identify all three inputs directly:

Department = Finance

Purchase Type = Software

Amount = CHF 3,000

This avoids building a long fixed sequence of "Ask a question" nodes and makes the conversation more natural because the agent can reuse information the user has already provided.

Here is an example of the implementation:

kind: AdaptiveDialog

inputs:

- kind: AutomaticTaskInput

propertyName: Department

description: The department that the purchase relates to, for example Finance, HR, or IT. Use the department already mentioned by the user if available.

shouldPromptUser: true

- kind: AutomaticTaskInput

propertyName: PurchaseType

description: The type of purchase the user is asking about, for example Software, Consulting, Equipment, or another purchase category. Use the purchase type already mentioned by the user if available.

shouldPromptUser: true

- kind: AutomaticTaskInput

propertyName: Amount

description: The purchase amount in CHF. Use the amount already mentioned by the user if available.

shouldPromptUser: true

modelDescription: |

Use this topic when the user asks about purchase approvals,

purchase approval requirements, approval thresholds,

or who needs to approve a purchase.

The correct answer may depend on the department,

purchase type, and purchase amount.

Use information already provided by the user whenever possible.

Ask only for missing information that is necessary to determine

the correct approval requirements.

beginDialog:

kind: OnRecognizedIntent

id: main

intent: {}

actions:

- kind: SearchAndSummarizeContent

id: npDxmT

userInput: "=Concatenate(\"Find the applicable purchase approval requirements. Department: \", Topic.Department, \". Purchase type: \", Topic.PurchaseType, \". Amount: \", Text(Topic.Amount), \" CHF.\")"

inputType:

properties:

Amount:

displayName: Amount

description: The purchase amount in CHF. Use the amount already mentioned by the user if available.

type: Number

Department:

displayName: Department

description: The department that the purchase relates to, for example Finance, HR, or IT. Use the department already mentioned by the user if available.

type: String

PurchaseType:

displayName: Purchase Type

description: The type of purchase, for example Software, Consulting, Equipment, or another purchase category. Use the purchase type already mentioned by the user if available.

type: String

outputType: {}
Suggested answer

sannavajjala87 1,113 Super User 2026 Season 2 on at

Like
a
(0)

Report
Copy link

Link copied!

I agree with @chiaraalina Using topic inputs for key context elements such as Department, Process, or Scenario is a great approach because the agent can automatically extract values already provided by the user and ask follow-up questions only when information is missing. This keeps the conversation more natural, reduces unnecessary prompts, and helps ensure knowledge retrieval is performed with the right context. For highly context-dependent documentation, combining targeted inputs with context-aware retrieval usually produces more reliable answers than a fixed question flow.

Thanks & Regards,

Manoj Annavajjala

www.powerplatformengineer.com
Suggested answer

Valantis 7,545 Super User 2026 Season 2 on at

Like
a
(0)

Report
Copy link

Link copied!

Hi @CU13051508-0,

AutomaticTaskInput with shouldPromptUser: true is a real node type, the orchestrator fills it from context already in the conversation and only prompts when a value is genuinely missing, exactly the ask-only-if-missing behavior described. Same pattern shows up identically in Microsoft's own reference material.

https://github.com/microsoft/mcscatblog/blob/main/_posts/2025-09-20-copilot-studio-child-connected-agents-inputs-outputs.md

SearchAndSummarizeContent is also confirmed as the correct node kind specifically for RAG style retrieval with query control, the right choice for searching documents with a constructed context aware query rather than a general knowledge answer node.

Worth knowing this makes the input driven approach the one natively aligned with generative orchestration, the orchestrator checks for already provided values before asking, so there's no need to hand build conditional logic to skip questions the user already answered, that's built into AutomaticTaskInput directly.

Best regards,

Valantis ✅ If this helped solve your issue, please Accept as Solution so others can find it quickly.

❤️ If it didn’t fully solve it but was still useful, please click “Yes” on “Was this reply helpful?” or leave a Like :).

🏷️ For follow-ups @Valantis.

📝 https://valantisond365.com/ 💼 LinkedIn ▶️ YouTube
Suggested answer

M Bilal Khan 376 on at

Like
a
(0)

Report
Copy link

Link copied!
Hi @CU13051508-0,

Yes, I’ve found it works better to treat clarification as a small decision flow rather than relying on one general instruction like “ask questions before searching.”

A structure that can work well is:

1. Identify the intent
First determine what the user is actually trying to accomplish. If the request is already specific enough, skip clarification.

2. Identify the context
Ask only for the context that can change the answer, for example:

Which department/process?

Which system or workflow?

Which scenario or use case?

Is the user asking about policy, procedure, or troubleshooting?

3. Check whether enough information is available
The agent should internally determine whether the collected context is sufficient to perform a meaningful search.

4. Search with the context included
Instead of searching only the original question, construct the search query using the user's original request + the clarification answers.

For example:

User: “How do I submit this request?”

Agent:

“Which process are you referring to?”

“Which department handles the request?”

Then the knowledge search becomes something closer to:

“How to submit [request] for [process] in [department]”

rather than simply searching “how do I submit this request?”

5. Avoid unnecessary questioning
I would also explicitly instruct the agent not to ask questions when the user's original request already contains enough context. Otherwise users can get frustrated with a long questionnaire.

One instruction pattern I've used conceptually is:

“Before searching the knowledge sources, determine whether the user's request is sufficiently specific. If important contextual information is missing and could change the answer, ask concise clarification questions. Ask only one or two questions at a time. Do not ask for information that is not required to answer the request. Once sufficient context is collected, use both the original request and the clarified context when performing the knowledge search.”

The key is to make the clarification criteria explicit. Don't just tell the agent to “ask questions”; tell it what information changes the answer and when it should stop asking questions.

That usually produces much more consistent results, especially when the same terminology exists across multiple departments or processes.
Valantis 7,545 Super User 2026 Season 2 on at

Like
a
(0)

Report
Copy link

Link copied!

Hi @CU13051508-0,

Just wanted to check in and see if everything is working now. If you still need any help, feel free to let me know.

Also, if the issue is resolved, it would be great if you could mark the answer as solved so others with the same question can find it easily.

Thanks and have a great day!