Hi skulla,
Yes I have encountered this issue last night.
The situation at my side is I logged in PowerApp web portal with two different accounts, the first time I tried to manage connections in PowerApp Studio, it then redirected me to the web portal, but didn't auto sign out the other account in the new Browser Session.
I suspect this should be related with Browser cache.
After switching between Browers (Internet Explorer 11, Microsoft Edge and Chrome) back and forth, it then allows me to delete the connections under web portal, with only Chrome opens.
Please verify the connection settings again, make sure the parameters are all configured rightly before you delete it.. When deleting connections with SQL server(two of them with the same name), I also encountered the issue that it will do nothing after click delete under Connection page, wait for a while, click the delete in the popped out window, it then showing with deleting and deleted the connection. And now the delete works under all browsers.

Regards