How to Test Install Scripts Before Publishing Them

An install script is the first thing a new user runs, and a failure there ends the relationship. It is also the script most affected by the machine it runs on: the distribution, the shell, the user, the tools present. Testing it on your own laptop proves almost nothing. This post builds a test matrix with containers, covering the cases that break real installers. It uses ideas from testing on multiple distros, idempotency tests and the curl pipe bash risk. Examples use Docker Engine 27.

A matrix of clean machines

#!/usr/bin/env bash
set -uo pipefail
fail=0
for image in debian:13 ubuntu:24.04 fedora:41 alpine:3.21; do
  for user in root nobody; do
    echo "== $image as $user"
    if docker run --rm --user "$user" -v "$PWD":/src:ro -e HOME=/tmp "$image" \
         sh -c 'cp /src/install.sh /tmp/ && sh /tmp/install.sh && /tmp/bin/mytool --version'; then
      echo "ok"
    else
      echo "FAILED: $image as $user"; fail=1
    fi
  done
done
exit $fail

Each combination is a fresh filesystem. Alpine uses BusyBox and ash with musl, which finds Bash-isms and GNU-only flags; Fedora has no apt; the unprivileged user finds scripts that assume sudo or write to /usr/local. Run the script with sh, not bash, if the shebang says #!/bin/sh, since that is what users with that line will get. Many minimal images lack curl and ca-certificates; the script must say so clearly or install them.

Serve it like production

mkdir -p /tmp/site/releases
cp install.sh /tmp/site/
tar -czf /tmp/site/releases/mytool-1.4.0-linux-amd64.tar.gz -C dist mytool
( cd /tmp/site && python3 -m http.server 8099 & echo $! > /tmp/server.pid )
docker run --rm --network host debian:13 sh -c \
  'apt-get update -qq && apt-get install -y -qq curl ca-certificates >/dev/null && curl -fsSL http://127.0.0.1:8099/install.sh | MYTOOL_BASE=http://127.0.0.1:8099 sh'
kill "$(cat /tmp/server.pid)"

Make the download base URL overridable by an environment variable, here MYTOOL_BASE, so the same script can be checked against a local server. This tests the whole curl | sh path, including the case where the script is read from a pipe, which breaks scripts that read from stdin or call exec in the wrong way. A script run through a pipe cannot prompt the user on stdin, so interactive questions need /dev/tty or a flag.

Checks after the install

• The binary runs and reports the expected version: mytool --version.
• The checksum or signature verification actually ran, and fails on a tampered archive: serve a corrupted file and expect a non-zero exit.
• The PATH advice is correct for the user's shell and the install directory.
• No leftovers in the temporary directory; the script cleans up with a trap.
• Re-running the script upgrades or reports "already installed", not an error.

Failure paths

docker run --rm --network none debian:13 sh -c 'sh /src/install.sh; echo "exit=$?"'
docker run --rm -v "$PWD":/src:ro debian:13 sh -c 'PATH=/nonexistent /bin/sh /src/install.sh; echo "exit=$?"'

No network should produce a clear message and a non-zero exit, not a half-written file. A missing tool, an unsupported architecture (uname -m returning something unexpected) and an unwritable target directory each deserve a test and an error message that tells the user what to do. These are the cases real users hit and the author never sees.

Uninstall

sh install.sh --uninstall
find /tmp/bin /tmp/.config/mytool 2>/dev/null

If you offer an uninstall, test it: after it runs, the files should be gone and a second uninstall should not fail. Installers that cannot be undone make people reluctant to try them.

Where this does not apply

Containers do not reproduce desktop integration, systemd units that need a real init, or macOS and Windows behavior. Test macOS in a CI job on a macOS runner, and the shellcheck and dash -n checks catch syntax problems early; see ShellCheck in CI. Review the script as a user would: it runs with their privileges and deserves the same scrutiny as any download you verify.

Choosing the machines to test on

An installer meets a wide variety of systems, and the bugs live in the differences: the shell, the package manager, the versions of core tools, the init system, the architecture. Test on a small matrix that covers the families you claim to support: a current Debian and Ubuntu, a Fedora or Rocky, and Alpine if you support it, since Alpine uses BusyBox and musl and breaks scripts that assume GNU tools. Containers make this cheap: docker run --rm -v "$PWD":/work -w /work image sh install.sh runs the installer on each image in seconds. Add an older release of each family, since many users run versions several years behind, and test on both architectures if you ship for both. Containers lack an init system and some kernel features, so a script that installs a service needs a virtual machine to test starting it, and the container test covers everything up to that point. Keep the list of supported systems in the readme and make the test matrix match it exactly.

The test should run the installer the way users will: as the user they will use, with the permissions they will have, from the path they will have, and through the same download path.

Serving the script like production

Many installers are fetched with curl and piped into a shell, which has a known failure mode: if the connection drops partway, the shell executes a truncated script, which may stop in the middle of a command. Defend against it by wrapping the whole body in a function and calling the function on the last line, so a truncated download defines nothing useful and runs nothing. Test it by serving the script from a local web server and cutting the transfer short. Test the checksum or signature check as well: serve a modified file and confirm the installer refuses it. Serve it over HTTPS in the test if production does, with a local certificate authority, because certificate handling is one of the places that differ between systems. Run the test against the real published URL occasionally, since a redirect, a content type or a cache can change what arrives. Avoid recommending the pipe to a shell when a download followed by an inspection is possible.

Checks after the install and the failure paths

After the installer finishes, verify the result as a user would: the command is on the PATH in a fresh shell, it prints the expected version, the files have the expected owners and modes, a service is enabled and running if one should be, and the program does one real piece of work. Then test the ways the install can fail. Run it without root, without curl, without network access, on an unsupported architecture, on a system where the program is already installed, and with a full disk, and read each message. A good installer says what is wrong and what to do about it, and exits with a nonzero status. A bad one prints a stack of errors from a command that was never going to work. Test that a second run upgrades or does nothing rather than breaking the existing setup, as covered in the idempotency checks.

Finally, test the removal. List the files on the system before the install and after the uninstall and compare: anything left behind is a bug, apart from user data you decided to keep and documented. An installer that cannot be undone is a liability for everyone who tries it once.

See docker run and the POSIX shell language.