The easiest way to do that would be to try and run the Attach to running Excel action first and handle its error. If it fails, then run the Launch Excel action. If it succeeds, then continue without launching Excel.
An alternative could be using the If process action and checking if the Excel process is running. If it is, you could assume that the document is already open and use Attach to running Excel. But this approach is risky, because you could have an existing Excel process, but a different document, which would still lead to Attach to running Excel failing. There is no way to check the exact document that is open solely based on the process, apart from using some custom scripting in VBscript, PowerShell or CMD actions.
If the Excel instance is visible, you could also use the If window action and check if an Excel window with the specific file name is open, since the file name will be in the window title. Then, if it is open, you can use the Attach to running Excel to attach to it. But if the instance is not visible, that won't work.
-------------------------------------------------------------------------
If I have answered your question, please mark it as the preferred solution. If you like my response, please give it a Thumbs Up.
I also provide paid consultancy and development services using Power Automate. If you're interested, DM me and we can discuss it.