Copilot Studio Generative Answers Returning Invalid URLs

(0) Share

Report

Posted on by ML-03030915-0

I am building an agent in Copilot Studio that retrieves the latest top 10 news articles from the web based on the industry selected by the user and have configured Generative Answers to provide URLs as sources for the responses it generates. However, during testing, the URLs provided in the answers are either broken, non-clickable, or invalid.

I’ve ensured that:
•The generative answers feature is enabled.
•The copilot has access to external sources.
•No specific filtering is blocking valid links.

Despite this, the responses still contain URLs that don’t lead to the correct articles. Has anyone encountered this issue before? Are there any settings or workarounds to ensure the URLs provided are valid and functional?

Any help would be appreciated!

Categories:

General topics

I have the same question (0)

All responses (3)

Answers (0)

Sort by

Suggested answer

Artur Stepniak 1,539 Moderator on at

Like
a
(0)

Report
Copy link

Link copied!

Hello,

it's hallucinating, that's why you receive incorrect responses. Are you sure that the resources available are made public? I assume yes. The only way to overcome this is to use RAG technique: instead of querying the website directly via knowledge source, you should gather your data in tabular format (on a SP list/Dataverse table/somewhere else) and query it directly, and let the chatbot only present the outcome via generative answers. Otherwise it won't be that efficient, especially when you have a big dataset with many articles. It won't be that precise and it'll mix it up.

In case of any other questions, let me know. If the answer helped you, mark it, so that others can benefit from it.

Best regards,

Artur Stepniak
Interested in GenAI? Visit my site!

Was this reply helpful? Yes No
ML-03030915-0 2 on at

Like
a
(0)

Report
Copy link

Link copied!

Hi Artur,

Thanks for your response! To clarify, I am building an agent in Copilot Studio that retrieves the latest top 10 news articles from the web based on the industry selected by the user. The goal is for the copilot to search for real-time news and provide the article titles along with valid source URLs.

I understand that generative answers can sometimes hallucinate, but in this case, I was expecting it to return real URLs from actual news sources. Do you think implementing RAG with a structured data source is the best approach for this use case? Or is there a way to ensure that the URLs retrieved are valid and clickable when using generative answers?

I appreciate your insights!

Was this reply helpful? Yes No
HL-20072017-0 2 on at

Like
a
(0)

Report
Copy link

Link copied!

I have a similar experience and question, except that my knowledge sources are documents stored in Sharepoint sites. When I just use the agent without any topics, it returns the link correctly. However, once I put a generative answer node within a topic, the urls are wrong about 75% of the time.

Is this something I can configure within the topic to fix or is this a known bug/limitation?

I got around this by only using topics when I'm sending a fixed message, but it would still be nice to get a generative answer limited to a few knowledge sources when a particular topic is referenced and still get correct links.

The other avenue I was considering pursuing would be nested agents instead of topics, but I'm trying the less work option first.

Was this reply helpful? Yes No