I have exactly the same problem.
I have Site Visibility on Private and already shared with 3 people, which were showing in the list.
Because I had to add another 15 people, - filled in their email-addresses and then clicked "Share".
after this the error-notification came and the list does not get shown anymore...
Already tried to work around this by temporary changing to "Public" and switching back to "Private" but the error does not disappear and the list does not get shown again...