Unanswered

Dataflow Engine V2 - What exactly changed?

(0) Share

Report

Posted on by Charles Seguin

401

I noticed an important difference today while creating a new dataflow, a new dataflow type called Standard V2. In the past, my new dataflows were always created as V1, but today (without doing anything differently, I think) my new dataflow was a V2.

Does anybody know what the changes are between V1 and V2?

I suspect there are changes in relation to the way the dataflow engine interprets dates and choices, because my "old way" of doing things now results in failed loads.

Categories:

Microsoft Dataverse with Power Apps

I have the same question (0)

All responses (5)

Answers (0)

Charles Seguin 401 on at

Like (2)

Report
Here a few tips for those having issues with the Standard V2 dataflow engine:
When working with dates, ensure that the date format in your dataflow is exactly the same as the date format in Dataverse. In particular, look out for slashes and dashes. If your date data is stored with dashes in dataverse (2020-02-06), make sure your dates are also structured with dashes in your dataflow. The Standard V1 dataflow engine was more "permissive" to this effect, accepting slashes instead of dashes. This no longer seems to be the case with Standard V2.
The Standard V2 dataflow engine doesn't like importing null data in choices. This wasn't a problem with the Standard V1 engine, but it is now. My recommended solution is the following: create a new choice called "None" (or anything else you like). Take note of the numerical value associated with your newly created choice. In your dataflow, replace all null fields with that numerical value.
If you disagree with my findings, please comment below, I have been known to be mistaken from time to time... 😉Nevertheless, the solutions above worked fine for me. If you have any other tips relating to the Standard V2 engine, please comment below!

Was this reply helpful? Yes No
KSven 182 on at

Like (1)

Report

Hi @CharlesS ,

in addition to your findings I've recognized with Standard V2 Dataflows there are no "integration keys" available anymore after first run of dataflow. With changing of alternate key incl. lookup columns ypou can't set up the right integration columns anymore cause there's no entry to your table anymore...
Till now theres nothing better with Standard V2 engine for me...:/

BR,

Sven

Was this reply helpful? Yes No
KSven 182 on at

Like (0)

Report

Hi @CharlesS ,

just ran into the same issue with sensitve dateformat for standardv2 dataflows.
I've an alternate key for a table with 2 columns:
- crefb_year (text type)
- crefb_validfrom (date type store as yyyy-mm-dd)

Have tried to set up the dataflow but I'm not able to "format" the date column matching for the following odata request (behind the dataflow action):

It MUST be like this
When I set up the "crefb_validfrom" column as string with Power query I will get the following error protocol:
"/crefb_marketforecasts(crefb_validfrom='2021-01-01',crefb_year='2021')""
--> So there are some apostrophe before and after my date value.

So I've tried to set up the date column as date column with Power query. Here I will get the following error protocol:
"/crefb_marketforecasts(crefb_validfrom=1/1/2021,crefb_year='2021')""
--> So there are no apostrophe anymore, but not the "yyyy-mm-dd" format.

My question:
How to set up date column in Power Query to ensure "yyyy-mm-dd" value without apostrophe?
Already have tried the english (sweden) locale format, but of course it will stay as "d/m/yyyy" format.

BR,
Sven

Was this reply helpful? Yes No
Charles Seguin 401 on at

Like (0)

Report

Hello @KSven ,

Sorry for my late reply. I think the Microsoft team are making backend changes. I just posted a new thread this morning. I have a dataflow used to import data into Date Only fields. It has been working fine for several weeks. This morning, surprise, the dataflow is failing:
New Dataflow v2 problem with Date Only imports - Power Platform Community (microsoft.com)

I trust Microsoft will look into these date/time issues and ensure any fixes are fully functional, whatever locale they are using.

To be continued...

Keep 'em flowin'

Charles

Was this reply helpful? Yes No
KSven 182 on at

Like (0)

Report

Hi @CharlesS ,

no problem, already have created MS Support Ticket some days before. Haven't received some more information about this bug, but I have 2 more Tickets about reference links via Datflows. So I guess its a Main Problem with Standard V2 Dataflow (especially regarding to the integration key setting with standard v1 I guess they tried to set it automatically with v2, but failing till now 😄 )

BR,
Sven

Was this reply helpful? Yes No