Scripting REST APIs Without Fragile curl Loops

Somewhere in your infrastructure there is a shell script with a loop like this:

url="https://api.example.com/items?page=1"
while [ -n "$url" ]; do
  resp=$(curl -s "$url")
  echo "$resp" | jq -r '.items[].id'
  url=$(echo "$resp" | jq -r '.links.next // empty')
done

It works, mostly. It has also quietly accumulated a list of failure modes that nobody will notice until a bad day:

• A 500 response exits with status 0, so the error page flows into jq, which prints nothing, and the script “succeeds” with empty output.
• There is no retry, so one transient network blip fails the whole job — or worse, half of it.
• There is no timeout, so a hung connection hangs the cron job behind it.
• There is no page bound, so a pagination bug upstream turns the loop into an accidental load test.
• If next ever points somewhere unexpected — another host, an attacker- influenced URL in a response body — the loop follows it without a thought.

None of these are exotic. They are the standard tax on hand-rolling HTTP plumbing in shell, paid one incident at a time. This post is about paying it once, in the tool, instead.

Restish is a CLI for REST-ish HTTP APIs, and one of its design goals is being boring in scripts: response data on stdout, diagnostics on stderr, exit codes that mean something, and bounded loops by default. Here is what that looks like for each failure mode above.

The Loop, Replaced #

The whole script above is one command:

Restish recognizes the collection’s next links and follows them automatically, the -f filter selects one field from each item, and -o lines prints one scalar per line for the next program in the pipe. The pagination is bounded (25 pages by default, configurable with --rsh-max-pages), and next-page URLs must stay on the same origin — scheme, hostname, and effective port. A link that wanders off-origin stops the loop with a warning instead of being followed.

When a script needs the whole logical collection at once — to count it, sort it, or deduplicate it — collect first, then filter:

And when you want to bound the work explicitly, say so:

restish api.rest.sh/images --rsh-no-paginate     # exactly one page
restish api.rest.sh/images --rsh-max-pages 3     # at most three pages
restish api.rest.sh/images --rsh-max-items 100   # at most 100 items

Exit Codes That Mean Something #

A script’s first question about an API call is “did it work?”, and the answer should not require parsing anything. Restish maps outcomes to exit codes:

So ordinary shell control flow just works:

if ! restish -S api.rest.sh/status/204; then
  echo "health check failed" >&2
  exit 1
fi

-S suppresses output for the cases where the exit code is the whole answer.

And when the reaction depends on whose fault it was, the 4 versus 5 split is already there — no body parsing required:

restish -S api.rest.sh/status/204
case $? in
  0) ;;                                            # healthy
  4) echo "client bug: fix the request" >&2 ;;
  5) echo "server error: retry later" >&2 ;;
  *) echo "transport or usage failure" >&2 ;;
esac

HTTP error statuses still write the response body to stdout before exiting non-zero, so you can log what the API actually said. And when the script handles HTTP status itself and wants the error body as data — a structured problem response, say — keep the body and force a zero exit:

stdout Is for Data, stderr Is for Commentary #

Restish keeps the streams disciplined: selected response data goes to stdout; progress, warnings, verbose request traces, and pagination notices go to stderr. A pipeline never has to strain diagnostics out of its data, and -v debugging does not corrupt the output a downstream step consumes.

Output formats make the data side explicit instead of terminal-shaped:

restish api.rest.sh/images -o json            # one complete JSON document
restish api.rest.sh/images -o ndjson          # one JSON record per line
restish api.rest.sh/images -f body.self -o lines   # one scalar per line

json suits a single document handed to one consumer, ndjson suits record streams processed line by line, and lines suits scalar values feeding xargs, sort, or a while read loop. There is no guessing about prettification either: redirecting an unfiltered response writes the body bytes unchanged, and any -f filter or explicit -o format renders structured output for the next program.

Retries and Timeouts Without a Wrapper #

The retry-with-backoff wrapper function pasted between shell scripts can retire. Bound the time, state the retries:

restish 'api.rest.sh/slow?delay=2s' --rsh-timeout 3s
restish 'api.rest.sh/flaky?failures=1&key=my-job' --rsh-retry 2

One detail matters more than it looks: automatic retries apply to GET and HEAD by default, not to writes. Replaying a POST because the first attempt timed out is how scripts double-charge customers. When a non-idempotent endpoint genuinely tolerates replay, opting in is explicit: --rsh-retry-unsafe.

Putting It Together #

A realistic CI step — check that every image resource an API lists is actually reachable:

#!/usr/bin/env bash
set -euo pipefail

restish api.rest.sh/images -f body.self -o lines --rsh-max-items 50 |
while read -r path; do
  restish -S "api.rest.sh$path" --rsh-timeout 10s --rsh-retry 2 ||
    { echo "unreachable: $path" >&2; exit 1; }
done

Every fragile part of the opening loop is now someone else’s tested code: pagination is automatic and bounded, transient failures retry, hangs time out, HTTP errors become exit codes, and stdout carries nothing but data.

This works the same against your own APIs — and if an API publishes OpenAPI, you can connect it and write the script against generated commands with profiles and auth handled:

restish api connect example api.rest.sh
restish example list-images -f body.self -o lines

Try It #

Install Restish:

brew install restish
restish --version

Or with Go:

go install github.com/rest-sh/restish/v2/cmd/restish@latest
restish --help

Then run the replacement for the opening loop:

restish api.rest.sh/images -f body.self -o lines

Useful next stops:

• Scripting and Automation is the durable reference for exit codes, streams, and the stable script flags.
• Pagination and Links covers limits, collect mode, and APIs that paginate without next links.
• Retries and Caching goes deeper on retry behavior.
• Output explains the format model and redirect semantics.

The fragile parts of API scripts were never the interesting parts. Move them into the tool, and the script that is left is the part you actually meant to write.