It's really unclear to me what this Extensibility feature is. There are three main products I've seen:
1. Copilot Studio to create your own bots
2. Copilot Studio to extend your bots into the native copilot search (i think??)
3. AI studio to do...not sure.
When should I be using what?
I believe I understand your question and have the perfect video to answer it. Please watch this video and let me know if this answered your question as I assumed:
https://www.youtube.com/watch?v=X8Qc9-0mPFE
This image may also help (click on it to zoom in):