Hi Ian,
I just did a proof of concept on this very thing, this very morning!
Found that manually creating the Site Settings did the trick as the wizard creates and insists on some settings which left me with invalid sign in errors. I haven't yet been able to figure out why and what knock on I have for excluding those settings, for now.
Rough steps:
Create the App Registration in Azure (named CustomerAzureAD in my case)
Follow instructions in the Msft doc but select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and make sure under Implicit grant, you have selected the ID tokens check box.
Then in Portal Management Studio manually create your site settings:
Authentication/OpenIdConnect/OpenId_1/AllowContactMappingWithEmail = true
Authentication/OpenIdConnect/OpenId_1/Authority = https://login.windows.net/common/ (documentation said (the newer!) https://login.microsoftonline.com/common but it would not work for me!)
Authentication/OpenIdConnect/CustomerAzureAD/Caption = (whatever you want to call your button)
Authentication/OpenIdConnect/CustomerAzureAD/ClientId = the Application ID from Azure
Authentication/OpenIdConnect/CustomerAzureAD/ExternalLogoutEnabled = true
Authentication/OpenIdConnect/CustomerAzureAD/IssuerFilter = https://sts.windows.net/*/
Authentication/OpenIdConnect/CustomerAzureAD/RedirectUri = yourportal.powerappsportals.com/signin-openid_1
Authentication/OpenIdConnect/CustomerAzureAD/ValidateIssuer = false
Additonally I've configured:
Authentication/Registration/Enabled = true
Authentication/Registration/InvitationEnabled = true
Authentication/Registration/OpenRegistrationEnabled = false
You'll see that even though you have manually created the Identity Provider it does appear in the listed Providers in Authentication Settings for your Portal.

Let me know if you figure anything else out with this, that I haven't already 😀
Good luck!