At some point Microsoft added the Horizontal Layout to the radio buttons. I don't know when.
Using the Layout selector of the control is the easiest method to do this.
However, I needed additional functionality, so I made my own Horizontal Radio button as a custom component.
I did this by building a custom component with a horizontal gallery to hold the options and an output property to indicate which is selected. Using the Circle and the "selected Flag" to fill or not the circle. this is very flexible and customizable.
I set the Default of the Gallery to "First(Self.AllItems.Value)" but you could use any valid logic to set it.
My default Items List is: ["None", "Supplies", "Printables", "Kits"]
Here is a screenshot of the compnent:

My OnSelect for the gallery is: Set(varSelectedItem, ThisItem.Value);
And I use varSelectedItem in the Output Property.
I set the Fill Property of the Circle to: If(ThisItem.IsSelected, RGBA(56, 96, 178, 1), RGBA(0,0,0,0))
I tried using varSelectedItem for the Fill, but that didn't work for the initial default selection.