Power Platform Community Forum Thread Details

I am running into an issue where Azure Synapse Link for Dataverse is failing to trigger the Apache Spark pool for Delta/Parquet conversion. Can it be due to our cross-cloud architecture?

Our Environment:

Power Apps / Dataverse Environment: Hosted in US Government Cloud (GCC).
Azure Synapse Workspace & ADLS Gen2 Account: Hosted in Azure Commercial.

The Problem:
When creating a brand new Synapse Link from the Power Apps Maker Portal, the wizard successfully allows me to check "Use Spark pool for processing" and I can see and select my optimized Spark pool (Apache Spark 3.5, Auto-scale enabled from 3 to 14 Small nodes).

The link saves without errors. The initial data sync works and I can see csv files landing in the storage account and I can query the data using contact_partitioned in the Serverless SQL pool but contact table never gets generated.

However, no Spark application ever spawns or registers in the Synapse Monitor tab. The Delta Lake parquet files are never built.

Any insights or workarounds from anyone who has faced such scenario where parquet files are never built and Spark pool is never started would be highly appreciated.

Categories:

Microsoft Dataverse with Power Apps

GCC, GCCH, DoD - Federal App Makers (FAM)

Hi @HK-21062016-0,

Yes, your cross-cloud architecture (GCC Dataverse → Commercial Azure) is the root cause of this behavior.

What you’re seeing is a known platform limitation. The difference between CSV working and Delta/Parquet failing comes down to how the backend processing works:

Why CSV works
The Dataverse service exports data directly to Azure Data Lake Storage (ADLS Gen2) as CSV files using storage APIs. This data-plane operation can cross GCC → Commercial boundaries as long as permissions are configured correctly.

Why Delta/Parquet fails (and appears to “do nothing”)
Delta generation depends on a Microsoft-managed orchestration service that:

Connects to your Synapse workspace
Submits jobs to your Spark pool

This is a control-plane operation, and in GCC scenarios, these services operate within strict compliance and boundary constraints.

Because GCC and Commercial Azure are isolated environments, this orchestration is not supported across the boundary. As a result:

No Spark job is ever submitted
Nothing appears in Synapse Monitor
No Delta/Parquet files are generated

This matches the documented support matrix, where:

CSV ingestion is supported cross-cloud
Delta (Spark-based processing) is not supported for GCC → Commercial [github.com]

Your options
1. Align cloud boundaries (recommended)
Move Synapse + ADLS into Azure Government (or move Dataverse to Commercial) so everything runs within the same cloud boundary.

2. Process CSVs manually
Keep the current setup and use Synapse Spark / Databricks to convert CSV + incremental updates into Delta format yourself.

3. Evaluate Fabric Link (if available)
Microsoft Fabric may provide an alternative path for Delta-based analytics without relying on Synapse Spark, depending on GCC availability.

✅ If one of the responses here solved your issue, please mark it as Accepted so others facing the same problem can benefit as well.
👍 If this or any other reply here helped you, feel free to give it a Like. It helps others and is always appreciated.

Sunil Kumar Pashikanti, Moderator
Blog: https://sunilpashikanti.com/posts/