yes there are multiple options to do this:
1. legacy data import--go to advanced settings--data management--data import and import your data.
2. Immersive Excel: go to a view for the desired entity, click export to excel--export to Excel online. this opens an in-line excel online file and you can paste your data into there and save.
3. the CDS get data button--go to make.powerapps.com, click data, go to entity, click get data button and select Excel as source.
there is also a new Excel add-in that allows you to save your data to CDS directly from Excel. See https://powerapps.microsoft.com/en-us/blog/cds-for-apps-excel-importexport/.
My favorite is #1--it give you the most flexibility to map lookup fields and it allows you to import multiple files at the same time if you put them in a zip file. Doesn't support xlsx, but does csv, xls, or xml spreadsheet format.