Unanswered

Extremely inconsistent and slow refresh of dataflows

(0) Share

Report

Posted on by Community Power Pla...

Hi there!

Is anyone else experiencing slow and inconsistent refresh of dataflows to an CDS entity?

I have created two dataflows connected to CSV text files hosted in Azure file storage. The queries consist of 1-3000 rows and 5-9 transformation steps.
The dataflows are set to update a new CDS entity, and are having an automatic refresh setting of 1 min.

However, it's very inconsistent and slow.

To demonstrate this I have two examples:
#1
Here it somehow stopped 7/18, 8:37 AM refreshing by itself? Where I had to manual press refresh to start the refreshing again
1,5-2 min per refresh also seems quite slow, when thinking of the relative simple query

#2
In this example the duration of the latest refresh are 288 hours +
Where it normally takes around 10-15 seconds

I would love to hear if anyone have any advises or tips? Because it seems like dataflows are not a reliable solution..

Best regards
MrAjay

Categories:

Microsoft Dataverse with Power Apps

I have the same question (0)

All responses (6)

Answers (1)

SteveYao on at

Like (0)

Report

In case, the refresh is stuck with in Progress, you need to open a MS support ticket to cancel it from backend.

As you have no option to cancel from UI.

Was this reply helpful? Yes No
Verified answer

EricRegnier 8,720 Most Valuable Professional on at

Like (1)

Report

Hi @Anonymous,
Although Dataflows is a great low code tool to import and sync data, there are some limitations and one of them is getting access to detailed logs and detailed error messages. Once of the slowness issue commonly face is actually with authentication and MFA where the Dataflow is waiting someone to approve the login. Suggest to use a non-MFA account if you aren't already. I would also raise a Microsoft Support ticket at https://admin.powerplatform.microsoft.com/support so they can investigate the issue from their side and hopefully provide you with more details. Good luck...

Was this reply helpful? Yes No
Community Power Pla... on at

Like (0)

Report

Thank you for your answer @EricRegnier
I do not use MFA for the Dataflow, as I use a SAS token in the connection string.
I will try raising a support ticket.

Was this reply helpful? Yes No
Hapa1 49 on at

Like (0)

Report

@EricRegnier You mention Dataflow's limited "access to detailed logs and detailed error messages". Do you know of any current workaround to at least see whether my Dataflows have succeeded or failed without checking the refresh history of each? My org has 8 dataflows set to refresh daily and it is getting tedious checking their history daily.

Was this reply helpful? Yes No
EricRegnier 8,720 Most Valuable Professional on at

Like (2)

Report

As for as I know, the only place that you can see is in the Power Platform Admin Center, https://admin.powerplatform.microsoft.com. Click on Data Integration and there's a chart of all the Dataflows (aka Project Runs). Here's the Microsoft documentation about it: https://docs.microsoft.com/en-us/power-platform/admin/data-integrator-error-management
Hope this helps...

Was this reply helpful? Yes No
Hapa1 49 on at

Like (0)

Report

Thanks @EricRegnier Unfortunately we are on government cloud so this feature is not yet available to us.

Was this reply helpful? Yes No