DataFlow does not bring all records to the final table in Dataverse

(0) Share

Report

Posted on by calerof

255

Hi,

I'm creating a DataFlow appending 3 tables from different sources:

Excel file in OneDrive for Business
SharePoint List
Imported table from SQL Server

After several attempts I got it working but when checking the actual data, I see some records missing, but in the DataFlow they seem to be correct.

Here's an screen-shot of one item with more than 15 rows in the DataFlow.

Here is a filter of the table in Dataverse for the same item, it only has 8 rows.

Any help would be extremely appreciated.

Thanks,

Fernando

Categories:

Microsoft Dataverse with Power Apps

I have the same question (0)

All responses (10)

Answers (1)

mahoneypat 1,720 on at

Like (0)

Report

Just to clarify. You are using a Dataflow to combine 3 sources which is working correctly, but when you view the Dataverse table in Excel rows are missing?

Regards,
Pat

Was this reply helpful? Yes No
calerof 255 on at

Like (0)

Report

Hi @mahoneypat
The DataFlow creates an appended table Cardex. When I look at the data in the DataFlow everything looks fine, but when I look at the actual Cardex table it's incomplete. No, I don't use Flow.

Was this reply helpful? Yes No
EricRegnier 8,720 Most Valuable Professional on at

Like (0)

Report

Hi @calerof,

Just to clarify, the 3 sources import into 3 different tables, correct? If so, which one is causing issues? Have you checked in advanced find without any filters to confirm that records are missing? Also, can you check that there are no errors/warnings after a dataflow run? To check go to make.powerapps.com --> Data --> Dataflows, click the little warning icon next to the Last Refresh column:

Cheers

Was this reply helpful? Yes No
calerof 255 on at

Like (0)

Report

Hi @EricRegnier ,
The Dataflow image above shows the original tables and the new appended one is called "cardex". In this new table within the Dataflow all records appear to be complete.
I didn't find the Advanced Find, this is a Canvas app, but I checked for example one record of date 13-feb-2021, which appears in the Dataflow image above with DocNum 1100 and Qty of 10 two times, and in the final appended table it does not exist.
Also, there are no errors nor warnings in the Dataflow refresh:
Still I don't know what's going on.
Thanks for the help.
Regards,
Fernando

Was this reply helpful? Yes No
Drew Poggemann 9,287 Most Valuable Professional on at

Like (0)

Report

Hi @calerof,

Quick question, have you verified that the records showing in your dataflow were ever created in the Dataverse? The 15 records you show have DocNum values where in your screenshot from the 8 rows in the dataverse only 5 of the 8 DocNum values match and if you look at these 5, none of the Fecha or CodigoTransaccion values match between the two. I am guessing none of these records have been written to the cardex table.

Am I looking at something incorrectly?

Thanks,

Drew

Was this reply helpful? Yes No
calerof 255 on at

Like (0)

Report

Hi @dpoggemann ,
Thank you for your reply. Here are two screen shots of the complete Dataflow, all records from the appended table "Cardex" are in there:
Fernando

Was this reply helpful? Yes No
Drew Poggemann 9,287 Most Valuable Professional on at

Like (0)

Report

Hi @calerof,

Sorry, I am a bit confused here. You mentioned the records are not making it to the dataverse but in your last communication you are identifying they are "all there"? Are you comfortable the data is in the dataverse and it is just a challenge with viewing the records? If so, could this be simply a difference between running the dataflow as a user with admin rights (or elevated rights at BU or Oganization level) vs you querying dataverse as your own user that may have a role where you can view only your one records in the Cardex table? Can you verify this from a security perspective with user roles assigned?

Sorry if still off base, trying to understand exactly what the challenge is here.

Thanks,

Drew

Was this reply helpful? Yes No
Verified answer

Charles Seguin 401 on at

Like (1)

Report

I'm thinking the table you extracted from Dataverse shows that the missing records are actually not in Dataverse. You can always check with a view as well to be sure they are not there.

If they are indeed missing, here is what I would do:
1. Filter your dataflow to include only the 15 rows you are trying to import.
2. Once the refresh is complete, check the refresh history (see@EricRegnier 's reply) and open the log to validate how many upserts you got. If someone else in your organization is running that dataflow, ask him or her to do the same. If the upserts number is not the same, then @dpoggemann idea about user rights to data would need to be investigated further.
3. What are you using as a primary key? Is there any way the data could be overwriting itself?

Was this reply helpful? Yes No
calerof 255 on at

Like (0)

Report

Hi @dpoggemann ,
The Dataflow records are complete, all the records in the appended table in Dataverse are included in the Dataflow, but not all the records in the Dataflow are included in the appended table.
I am System Administrator in the environment.
Thanks,
Fernando

Was this reply helpful? Yes No
calerof 255 on at

Like (0)

Report

Hi @CharlesS ,
You're exactly right, your No. 3, I was using a "Folio" consecutive number in the SharePoint List which was not unique, it was used to request items and to return them back with the same "Folio". I changed it to the "ID" and now all the records are complete in the appended table in Dataverse.
Thanks so much!
Happy Dataverse! #PowerAddicts
Fernando

Was this reply helpful? Yes No