Web Scrape to SharePoint List

(1) Share

Report

Posted on by dazmatic

Hi Everyone,

I'm hoping the community might be able to provide some support with a project I'm working on.

In essence, I am using a SharePoint list with each row being a user profile and the columns holding date information for qualifications and am using the 'Get Items' function to get the list.

The source web data is the EUSR website. I have already a list of names and numbers using the profile list and I have worked out how to scrape the qualifications and date data iterating through the profile list.

Currently, my flow scrapes the website and puts the results into the EUSRResults variable. For each of the resulting outputs from the scrape, I have figured out how to get the qualification name and the date, CurrentQualName and CurrentWorkingDate.

This I originally thought would be the tricky part, scraping the data but I was so wrong.

The issue I am facing is then using that data, relating it back to the SharePoint List profile and updating the dates in the profile. This issue takes several forms:

1. Because the EUSR information is dynamic there is no consistency to it therefore I cannot use the 'Update Item' function as if the variable doesn't exist during that iteration, the update fails and there's no way to populate the Update Item function dynamically.

2. Relating the fields - for example in the list, one of the dates is 'GL/6 Update' but during the get items, this field is called 'GL_0020x_6Update' as it is called in the list - therefore there's no easy way to match and update.

3. Had thought about updating individually via http request but that seems even more difficult.

At this point, I'm frustrated because I've got the data I need, but can't physically get it to the place where I need it.

Any help would be greatly appreciated!

Categories:

Power Automate Desktop

I have the same question (0)

All responses (2)

Answers (0)

Suggested answer

Agnius Bartninkas Most Valuable Professional on at

Like (0)

Report

If I understand it correctly, the actions in your screenshot are inside another loop that iterates through the response from Get items?

If that's the case, you do have your item in a variable - it's a part of the %CurrentItem% variable. You should have an Id in there somewhere that can be used to target the specific item you need, and you can use that in Update item.

I believe the tricky part here is combining all of the relevant data from %EUSRResults% into a single item. If I understand it correctly, you probably need to push everything from these results into the same item. So, you should not be using Update item inside the loop where you iterate through %EUSRResults%. You should probably keep appending those values to some string variable inside the loop, and then, when the loop ends, use Update item to send it all to your SharePoint list item.

As for column names, there's not much you can do about it. You'll need to pick the names up from SharePoint. They're assigned automatically and special characters get replaced. You can either get those names via your Get items response, or if you open the list in a browser and export it to CSV with schema, the schema will show you the names of the columns.

Was this reply helpful? Yes No
dazmatic 6 on at

Like (0)

Report

You are correct, initially Get Items is used to get the list of profiles and current data and associated fields.

The issue I'm finding with this is unlike Power Automate Cloud, PAD only pulls through fields that are populated - For example I have a profile with an unpopulated field Gl/6 Update, during the Get Items, the profile is pulled through but because the GL/6 Update is blank, the field it not pulled through therefore I've got no starting point for field names to update.

I have already tried as you have suggested, appending those values to a datatable with pertaining columns which are added dynamically based on the data scraped giving me easily accessible data in the format of %EUSRScrape[0]['GL/6 Update']% to use with an Update Item. However, Update Item fields are not dynamic and if the web scrape does not include a GL/6 Update item, then the Update Item fails because because it doesn't exist.

Was this reply helpful? Yes No