How to Fuzz Command Line Tools With Simple Shell Loops

Most command line tools are checked with the inputs their authors thought of. The inputs that cause failures are the ones nobody thought of: an empty argument, a path of 5,000 characters, a file of random bytes. A shell loop that supplies those costs ten lines and finds real bugs in parsers, converters and anything that reads files. It extends the ideas in property tests in shell. The examples assume Bash 5.2 and a tool under test called mytool.

A first fuzz loop

#!/usr/bin/env bash
mkdir -p crashes
for i in $(seq 1 500); do
  head -c $(( RANDOM % 4096 )) /dev/urandom > in.bin
  timeout 5 ./mytool in.bin > /dev/null 2> err.txt
  rc=$?
  if (( rc > 127 )); then
    cp in.bin "crashes/crash-rc$rc-$i.bin"
    echo "run $i: exit $rc"
  fi
done
echo "crashes: $(ls crashes | wc -l)"

run 87: exit 139
run 311: exit 124
crashes: 2

Exit codes above 128 mean death by a signal (139 is a segmentation fault, 134 an abort), and 124 is timeout's code for a hang; see decoding exit codes. Ordinary non-zero statuses (1, 2) are the tool rejecting bad input, which is the correct behaviour. Keep each failing input; a crash you cannot replay is a story, not a bug report. The output shown is an example of the report format.

Edge-case arguments and structured inputs

long=$(printf 'A%.0s' $(seq 1 100000))
for arg in "" " " "-" "--" "-0" "$long" "$(printf '\xff\xfe')" "../../etc/passwd" '$(id)' '`id`' "a b" "*" "~" "9999999999999999999"; do
  timeout 5 ./mytool "$arg" >/dev/null 2>&1
  rc=$?
  (( rc > 127 )) && echo "arg=[${arg:0:20}] rc=$rc"
done
for f in empty.txt onlynewline.txt nul.bin crlf.txt utf16.txt; do timeout 5 ./mytool "$f" >/dev/null 2>&1; echo "$f -> $?"; done

A table of awkward values covers the usual weak spots: empty strings, a lone dash that looks like an option, extremely long arguments, invalid UTF-8, path traversal strings, shell metacharacters (these should be treated as data), integer overflow and special files. For inputs with structure, mutate valid samples instead of generating pure noise: flip a few bytes of a real file with dd or python3, because random data fails at the first header check and never reaches deeper code.

Mutating real inputs

mutate() {
  python3 - "$1" "$2" <<'EOF'
import random, sys
data = bytearray(open(sys.argv[1], "rb").read())
for _ in range(random.randint(1, 8)):
    if data:
        data[random.randrange(len(data))] = random.randrange(256)
open(sys.argv[2], "wb").write(data)
EOF
}
for i in $(seq 1 300); do mutate sample.png mutated.png; timeout 5 ./mytool mutated.png >/dev/null 2>&1 || true; done

Starting from a valid file keeps most of the structure intact so that the parser goes further before it hits the damage. Add the interesting values directly (0, 255, 0x7f, 0x80) for the length and offset fields. Seed the generator (random.seed(i)) when you want a run to be reproducible.

Contain the blast radius

docker run --rm -v "$PWD":/w -w /w --network none --memory 512m --pids-limit 100 --read-only --tmpfs /tmp debian:13-slim ./fuzz.sh
ulimit -v 1048576 -f 10240

A tool that writes files, deletes things or opens network connections should never be fuzzed on your real machine. Run it in a container with no network, a memory limit and a read-only file system, as in testing destructive scripts in a sandbox. Set ulimit for file size and memory, and watch the disk during the run.

When to graduate to a real fuzzer

afl-fuzz -i seeds/ -o findings/ -- ./mytool_instrumented @@

Random loops find shallow bugs. For parsers written in C, C++, Rust or Go, coverage-guided fuzzers (AFL++, libFuzzer, Go's built-in go test -fuzz, Rust's cargo-fuzz) track which code paths each input reaches and evolve inputs to find new ones. They need the tool built with instrumentation. For Bash scripts, the bugs are usually unquoted variables and unsafe handling of names, which odd-argument tables find well.

What kinds of bugs a shell loop finds

Random input finds a specific family of defects, and knowing the family helps you judge the result. Crashes from unchecked lengths, such as a buffer sized for typical names meeting a very long one. Hangs from loops that never terminate on malformed input, where a parser waits for a closing marker that never comes. Memory blow-ups from a length field that claims four gigabytes. Mishandled encodings, where a byte sequence that is not valid UTF-8 reaches code that assumes it is. Each produces a visible symptom: a signal, a timeout or a resource limit, and a shell can see all three without any special instrumentation.

What it does not find is logic errors. A converter that silently drops a column returns exit code zero on every input, and no amount of noise reveals it. For those, compare against a reference. If a trusted tool does the same job, run both on the same inputs and diff the outputs, which is differential testing and is surprisingly effective with only a few lines of shell. Any disagreement is a bug in one of the two, and finding out which is the interesting part.

Making failures reproducible and small

A crash found after four hundred random runs is useless until you can repeat it. Save the exact input to a file named after the run, and record the command line and the tool version beside it. Seed the generator and print the seed at the start, so a rerun with the same seed produces the same sequence. Once you have a failing file, shrink it: remove half the bytes and see whether it still fails, keep the smaller version if it does, and repeat. A thirty-byte input that crashes the program is a bug report; a four-kilobyte one is a puzzle.

Automate the shrinking when failures are common. A short loop that tries deleting each chunk of the input and keeps the deletion when the exit code stays the same will reduce most failures in under a minute. Dedicated tools such as afl-tmin do the same with coverage information, and the idea is identical.

Classifying what you find

Not every non-zero exit is a defect. Define up front which codes are acceptable rejections, usually 1 and 2, and which are failures: death by signal, timeouts, and exit codes the tool documents as internal errors. Treat any output on standard error that contains words like "panic", "assertion" or "segmentation" as a failure even when the exit code looks normal, since some runtimes catch the fault and exit with 1. Group crashes by the last lines of the error output so that a thousand repeats of one bug show up as one entry, not a thousand.

Fitting it into regular testing

Run a short fuzz pass, a few hundred inputs with a fixed seed, in continuous integration so regressions in input handling fail the build. Run a long one, hours with a new seed each time, on a schedule outside working hours, and keep the corpus of interesting inputs in the repository. Every crash that was fixed becomes a permanent regression test: add the minimised input to the test directory with a comment naming the bug. Over a few months the directory becomes a record of every way the tool has been broken, and a cheap guard against breaking it the same way again.

Where this does not apply

Fuzzing finds crashes and hangs; it does not tell whether the tool's answers are right. Pair it with example-based tests. A tool that must reject hostile input securely (a setuid helper, a network parser) deserves proper security review beyond shell loops. See the AFL++ project and the Go fuzzing documentation.