>Do you want to remove the custom property within the Component?
Yes, within the Component definition, rather than a specific component instance. In the PowerApps studio there is a '+' to add a custom property, but no corresponding '-' to remove it; or better still an option within the custom property options '...' ellipsis menu of a custom property to remove it.
>Do you want to remove the custom property from the Master Component without affecting other instance which use the custom property?
It is understood that removing a custom property from the Master Component / Component definition would necessarily remove it from component instances, but there appears to be no way to even do that. So to achieve that, the developer has to delete the entire component and laboriously recreate it without the unwanted custom property.