Scripting REST APIs Without Fragile curl Loops

Tue, 30 Jun 2026 00:00:00 +0000

Somewhere in your infrastructure there is a shell script with a loop like this:

url="https://api.example.com/items?page=1"
while [ -n "$url" ]; do
 resp=$(curl -s "$url")
 echo "$resp" | jq -r '.items[].id'
 url=$(echo "$resp" | jq -r '.links.next // empty')
done

It works, mostly. It has also quietly accumulated a list of failure modes that nobody will notice until a bad day:

A 500 response exits with status 0, so the error page flows into jq, which prints nothing, and the script “succeeds” with empty output.
There is no retry, so one transient network blip fails the whole job — or worse, half of it.
There is no timeout, so a hung connection hangs the cron job behind it.
There is no page bound, so a pagination bug upstream turns the loop into an accidental load test.
If next ever points somewhere unexpected — another host, an attacker- influenced URL in a response body — the loop follows it without a thought.

None of these are exotic. They are the standard tax on hand-rolling HTTP plumbing in shell, paid one incident at a time. This post is about paying it once, in the tool, instead.

Automation on Restish

Scripting REST APIs Without Fragile curl Loops