Hi @R3dKap,
Could you please share a bit more about your scenario?
I have made a test and the issue is confirmed on my side.
There are some things to keep in your mind when you customize forms:
"The properties for the SharePointIntegration control may not be available in OnStart or OnVisible, and those events execute only once while the list is loaded. You can use OnNew, OnView, or OnEdit formulas to run logic before the form is shown to the user every time"
More details about the things that you need to keep in mind when customizing forms, please check the following article:
https://docs.microsoft.com/en-us/powerapps/maker/canvas-apps/sharepoint-form-integration#customize-the-default-form
In other words, the OnStart or OnVisible property could only be fired once when you edited your list (Just when you edit your list firstly), when you clicked the "Cancel" button, it just hide the custom list form rather than close the custom list form.
As an alternative solution, you could consider take a try to initialize/set a global variable within the OnEdit property of the SharePointIntegration control:
Set OnEdit property of the SharePointIntegration control to following formula:
EditForm(SharePointForm1);Set(VariableA,false)

Set the OnSelect property of the Button control to following formula:
Set(VariableA,true)
Best regards,
Kris