To delete the portal and purge the website definition you'd have to do 2 steps
1. From make.powerapps.com delete the portal. This option is available on the portal record in the recent apps list
2. Go to portal management app, navigate to Websites and then delete the website record. This will purge all the metadata for this portal.
Now from make.powerapps.com create your portal again.