Use a retry wrapper for flaky steps

A download or a deploy call fails now and then for reasons outside your control, and rerunning the whole job takes ten minutes.

retry() {
  local max=$1 delay=$2 n=1; shift 2
  until "$@"; do
    if (( n >= max )); then echo "giving up after $n attempts" >&2; return 1; fi
    echo "attempt $n failed, retrying in ${delay}s"
    sleep "$delay"; delay=$(( delay * 2 )); n=$(( n + 1 ))
  done
}
echo 0 > count
flaky() { n=$(( $(cat count) + 1 )); echo $n > count; [ "$n" -ge 3 ]; }
retry 5 1 flaky && echo "succeeded after $(cat count) calls"
echo ---
retry 2 1 false; echo "status: $?"

attempt 1 failed, retrying in 1s
attempt 2 failed, retrying in 2s
succeeded after 3 calls
---
attempt 1 failed, retrying in 1s
giving up after 2 attempts
status: 1

until "$@" runs the command and repeats while it fails, so the function retries any command with its arguments. The counter stops it after max attempts, and the wait doubles each time to give a struggling service room. When it gives up it returns 1, which is what makes the CI step fail honestly instead of passing after a hidden error. The loop condition is described in the Bash manual.

Retry only operations that are safe to repeat, such as a download or a read. A deploy or a payment that failed after doing its work could run twice. Add a random jitter to the delay when many jobs hit the same service at once. GitHub Actions has no built-in retry for a run step, so the loop belongs in the script.