Hi Richard,
For me, jquery is showing as 3.6.0. And it is upgraded. 😀 But keep in mind that you also need to update the dependent libraries for JQuery 3.6.0. I am not sure about the risk of taking it to production. So please play around in your dev environment.
Here are the steps for you.
Pre: Download Jquery minified version in your local
Log in to portal management.
1. Click Settings > Advanced Settings
2. In the Dynamics 365 Settings > Customise the system

3. Click Web resources > Filter the name which starts with j > you can find jquery
4. Replace the current jquery with your recent downloaded one.

Note: You may need to unblock your JS extension in Dynamics 365 if needed
We are done now.
In Portal studio,> Refresh the page > Sync configuration and browse the website. Press Ctrl + F5
In the console, try the following

As mentioned above, you may need to upgrade and perform the above steps for the following dependent library of jquery

I can't think of any other alternative way. Unless the Portal engineering team decided to upgrade jquery and bootstrap.
Hope it helps.
------------
If you like this post, give it a Thumbs up. Where it solved your request, Mark it as a Solution to enable other users to find it.