Setting the overall height of a flexible gallery doesn't seem to have any straight forward resolve. There should just be an Autoheight option for the gallery so it can be set inside scrolling screens and other galleries.
My workaround for this, until that occurs, was to add a hidden label in the gallery, behind the label with the auto-height text appearing, ensuring it is the same width and padding, text size, etc.. I called this label 'NoRows', and added to its Text field:
(Len(TheLabelwithAutoHeightText.Text)/75)*1.2
My understanding is that this is counting the text, dividing it by the number of characters (in my case approx. 75), and multiplying it by the line-height.
Then changing the gallery height to the below"
Sum(Gallery.AllItems,NoRows) * 30+10
where (30) is essentially the padding around the label. This gave a much closer result, with some detail pages just short or long a bit, but the large differences are gone. Adding a bit more to the end (+10) helped ensure the scroll would disappear.