Azure Synapse Link for Dataverse: Malformed sync files

(0) Share

Report

Posted on by MKennerley

Has anyone had issues with Azure Synapse Link for Dataverse creating malformed <entity>-model.json files??

This has happened three times against a number of different entities.

We are using the Azure Synapse Link for Dataverse as part of the Microsoft solution to sync D365 data to a SQL database (after D365 DES was removed).

This time there is a problem with the Competitor entity.

There is no error shown in Azure Synapse Link for Dataverse, it thinks that the sync was successful.

But the Synapse Pipeline that ingests the sync file fails as the file is malformed.

When you look at the generated file "competitor-model.json" that fails it only contains the schema for 2 fields on the Competitor entity. All the other fields are missing from the file. A working file will contain all the fields on the Competitor table.

This scenario of the file only containing two fields from the synced entity has been the same for all the other failures we have seen.

Wondering if anyone else has had this issue and if so have they found a reason why it fails.
We know we can remove and add the entity back into the Azure Synapse Link for Dataverse and it will start working correctly but we really need to know what is causing this as its happened 3 times ove the last 40 days and has effected 22 different entities so far.

I wish Microsoft would bring back DES. It was a much simpler, quicker, more robust and massively cheaper solution that using Azure Synapse.

Thanks,

Mark

Categories:

Microsoft Dataverse with Power Apps

I have the same question (0)

All responses (4)

Answers (1)

Ade_25 4 on at

Like (0)

Report

Hi,

We struggled with the same issue, and raised an issue with Tech support - but no resolution and no one seems to be able to explain what the issue was.
Hope Microsoft will address this soonest.

Thanks
Ade

Was this reply helpful? Yes No
Verified answer

MKennerley 11 on at

Like (0)

Report

Ade,
I should have replied to this post back in September when the Microsoft Dynamics 365 team fixed it for us.

This is what the MS engineering team said the problem was:

We got an update from our dev team that they have investigated the issue from backend, and they observed that during creation of new incremental folder tricklefeed service was only partially copied. As a result, model.json of many entities were not copied into new folder. So, when data sync happened for these entities, it was assumed that some columns were missing, and we tried to recreate model.json by resyncing metadata which led to incomplete model.json. The issue which caused not all files to be copied in new incremental folder is resolved now and they should not see this anymore.

Once they had done the above fix I had to remove the effected entities from the Azure Syanapse Link for dataverse. Let it do a sync and then add the entities back in.

Touch wood. We haven't had an issue since.

Mark

Was this reply helpful? Yes No
Ade_25 4 on at

Like (0)

Report

Hi thanks for the reply, did they mention why the "new incremental folder tricklefeed service was only partially copied"?
At the moment the CDC tricklefeed service seems to have reset (on one of our connections) - as the update count has gone back to nothing or less than what it used to be.

Thanks
Ade

Was this reply helpful? Yes No
MKennerley 11 on at

Like (0)

Report

Ade,

That quote above was all we got from MS. TBH it had taken 9 months of back and forth with MS departments passing the buck between them that when it was fixed I didn't have any more time (physically or mentally) to delve any deeper.

Mark

Was this reply helpful? Yes No