@summitb
That one you reference is about what security role(s) the App itself will use, such as in the linked Dataverse Data Source, etc. and it is not about about who specifically inside these roles that the app will be shared with - those are two separate things. The app may use the, say the Salesperson role and have this checkbox checked in this part. However, if the app was not shared with a single person through the Share feature on that App, then not even a single Salesperson will see that app. That roles thing is for the app scope permissions, not about which users see the app, that one is separate.
To check who it is shared with is in separate list from the roles one. The roles one is the permissions of the app - not about who in the role will get the app shared with them - they are actually not the same thing.
By the way, System Administrator is not very good example to use to illustrate above, that is why we picked another role even as an example. Because System Administrator could do almost whatever they want, and they may be able to even all the apps including those not shared with them, if their view selector in make.powerapps.com -> Apps is set to "Org Apps" instead of "My Apps" for example, and potentially in some other ways as well. This could cause even further confusion then. So try instead with another role that is reduced like Salesperson, or perhaps even just create a custom role, for best results about checking the above.
Check about if the above helps. If not let us know if this one is still not clear to you, and we might check and see if we could provide you with some more details.
