Hi @devost
To note. It might not be your power pages application allowing guest users, or it can be teams or any other app settings allowing "Guest users"
To solve
1. Please remove the guest account from Azure AD and give it a try.
2. Remove the contacts from the Portal management
3. Apply the Guest user restrictive settings for external collaborations
4. Azure AD allow only a single tenant.
5. Restart the portal
Hope it helps.
------------
If you like this post, give it a Thumbs up. Where it solved your request, Mark it as a Solution to enable other users to find it.