@Matt22365 wrote:
I know the key column and value tells this flow which row to update, however what do I put in here when I want it to update multiple rows?
Presuming an example Excel Table in an Excel file like this:

And presuming an example Flow somewhat like this:

Then a solution is that inside the Apply to Each in the Update a row Action, you would just click on the Key Value field, and then use the Dynamic content to select the value of the column you want that matches the Key Column selected above it - that makes the Key Value set to the one that will be for that row. Example below:

In example above, since Excel File has columns A, B, C, and D, and since Key Column is A, I select the Dynamic Content for the Column named A. If the Key Column and the Dynamic content selected column match as in the above, then it should achieve the result you want.
When doing this, the test run will then change all of the rows, as in the example below:

Check if it helps @Matt22365 .