GET Restful API - Loop through offset requests

(1) Share

Report

Posted on by TesDA

611

I have a Restful API request using the HTTP Step:

Method - Get

Authentication - Raw

Value - API key

It picks up everything on the first call correctly, but I dont know how to create the loop so that it continues until there are no more records.

Below is the JSON returned from the initial request (from the first set of records). How do I create a variable and loop through so that it uses the "next" in the JSON until "next" returns null?

{"count":7816,
"next":"https://blackedoutname.endpointname.com/api/v1/something/?_limit=4000&_offset=4000&include_custom_fields=False&jurisdiction=ABCDEF&status=Active",
"previous":null,
"results":[
{
"id":"9172391872397a",
"jurisdiction":{
"id":"kajhdkjhasdkjhb",
"name":"ABCDEF",
"slug":"ABCDEFno"},
"organization_person":{
"id":"kahdkjahdh",
"custom_fields":{},
"person":{
"id":"kajshdkhd",
"username":"65432","first_name":"Nate",
"last_name":"Smith",
"email":"Nate.Smith@something.com"}},
"is_primary":false,
"status":"active",
"in_compliance":false,
"start_date":"1995-01-01",
"end_date":"1996-12-31",
"extended_date":"1997-12-31",
"earned":5765.0,
"applied":5740.0,
"overall_deficit":1000.0,
"overall_applied":5740.0,
"carry_over":0.0,
"license_id":"",
"latest_annual_period":{
"id":"kasdkjahsdh",
"in_compliance":false,
"start_date":"1997-01-01",
"end_date":"1998-12-31",
"extended_date":"1999-12-31",
"earned":0.0,
"applied":0.0,
"overall_applied":0,
"overall_deficit":1000.0,
"carry_over":0.0}}]}

Categories:

Building flows

I have the same question (0)

All responses (6)

Answers (1)

TesDA 611 on at

Like (0)

Report

Additional information added to the summary

Was this reply helpful? Yes No
yeamans 83 on at

Like (0)

Report

'm having a similar issue. I have 100s of records with a system, however I can only call 100 at a time. I need to figure out how to loop through the pages.

I do get presented with s string "record": 857

Would i need to create a variable based upon dividing the total records by 100 then rounding up the next whole number?

Was this reply helpful? Yes No
Verified answer

TesDA 611 on at

Like (1)

Report

I finally figured this one out. In the returned JSON is a next, with the next http statement for the next download of json. I created a variable to pick up the "next" from the parsed JSON and then did a do until loop until next = null:

Step 1 - Initialize Variable

"next" is initially set to the initial http address.

Step 2 - Do Until loop

I set the count to 4 (it was defaulted to 60), since I know the API should only need to loop 2-3 times.

Was this reply helpful? Yes No
Automate123 128 on at

Like (0)

Report

Hi @TesDA

I have a similar issue when trying to call Jiras REST API. I am only getting back a maximum of 100 results. I am trying to apply your logic into my current flow.

Could you share what is inside the append to array variable connector please?
Thanks

Was this reply helpful? Yes No
yeamans 83 on at

Like (1)

Report
@Automate123,

Here's what I ended up doing to get it to work.

Larger Screen Shots:

Condition expression is:
length(body('Get_Items')?['items'])

Let me know if you want help. Happy to jump on a screen share to walk through it.

Was this reply helpful? Yes No
Automate123 128 on at

Like (0)

Report

Hey @yeamans

Apologies for the late reply I went out and just got back.
Thank you very much for sharing your solution. From first glance it is beginning to make more sense as to how I can achieve this.

I am going to try and apply the logic you used into my flow now.

In regards to your offer of a screen share that would be much appreciated, If I struggle to get it working I will send you a DM to try and set that up with you.

Thanks again, I have been trying to find a solution for weeks now

Was this reply helpful? Yes No