With the announcement of JSONata and Variables support in Step Functions, this just got a whole lot easier.

Here is my short blog on looping paginated results with Step Functions:

You can do things like loop through a set of paginated results, appending the results to an existing array.

"Assign": {
  "results": "{% $append($results, $states.result.Items) %}"
}

You can also keep track of an index as a Step Function variable now and increment it with JSONata rather than States.MathAdd