How to Use Test Driven Development for Shell Scripts

Shell scripts have a reputation for being untested because they start as five-line helpers and grow into programs nobody dares to change. TDD works in shell if the script is shaped for it, and the habit costs less than the first hour of debugging a broken deploy script. This post builds a small function from a failing test to a passing one, with the structure that makes it possible. It uses the setup from the Bats tutorial and testing Bash functions. Examples use Bash 5.2 and Bats 1.11.

Shape the script for testing

#!/usr/bin/env bash
set -euo pipefail

slugify() {
  :   # not implemented yet
}

main() {
  slugify "${1:?usage: slug.sh TEXT}"
}

if [[ ${BASH_SOURCE[0]} == "$0" ]]; then
  main "$@"
fi

The guard at the bottom runs main only when the file is executed, not when it is sourced. A test can then source slug.sh and call slugify directly, with no process started. This single pattern is the difference between a testable script and an untestable one. Everything below depends on it.

Red: write the failing test

setup() {
  source "$BATS_TEST_DIRNAME/../slug.sh"
}

@test "lowercases and joins words with dashes" {
  run slugify "Hello World"
  [ "$status" -eq 0 ]
  [ "$output" = "hello-world" ]
}

bats tests/slug.bats

 ✗ lowercases and joins words with dashes
   (in test file tests/slug.bats, line 7)
     `[ "$output" = "hello-world" ]' failed

1 test, 1 failure

Watch it fail, and for the right reason: the output is empty, not a syntax error or a missing file. A test that has never failed has not been shown to be able to.

Green: the smallest code that passes

slugify() {
  local s="${1,,}"
  printf '%s\n' "${s// /-}"
}

Lower-case the text and replace spaces with dashes, nothing more. The temptation is to write the whole final function now; resist it, because each behavior should have a test that demanded it. Run the tests: one passes.

Next behavior, next test

@test "drops punctuation" {
  run slugify "Hello, World!"
  [ "$output" = "hello-world" ]
}

@test "collapses repeated separators" {
  run slugify "a  --  b"
  [ "$output" = "a-b" ]
}

@test "rejects empty input" {
  run slugify ""
  [ "$status" -eq 1 ]
}

Each new test fails against the current function, which drives the next change:

slugify() {
  local s="${1,,}"
  [[ -n $s ]] || { echo "empty input" >&2; return 1; }
  s="${s//[^a-z0-9]/-}"          # anything else becomes a dash
  while [[ $s == *--* ]]; do s="${s//--/-}"; done
  s="${s#-}"; s="${s%-}"         # trim edge dashes
  printf '%s\n' "$s"
}

All four pass. The tests now document the behavior, and the string operations come from parameter expansion with no external process. Note that this version is Bash 4 or later, because of ${1,,}.

Refactor with the net underneath

With passing tests, restructure freely: replace the loop with sed -E 's/-+/-/g', rename variables, split into helpers. Run the tests after each small change; if one fails you know what caused it. That safety is what TDD buys.

Test the command line too

@test "script prints the slug" {
  run "$BATS_TEST_DIRNAME/../slug.sh" "Hello World"
  [ "$status" -eq 0 ]
  [ "$output" = "hello-world" ]
}

@test "script fails without an argument" {
  run "$BATS_TEST_DIRNAME/../slug.sh"
  [ "$status" -ne 0 ]
}

A couple of tests at the process level check that the wiring in main works. Most of the tests stay at the function level, where they are fast.

Where this does not apply

TDD adds little to a throwaway one-liner. Code that is mostly external commands, such as an installer, is better tested as an integration with fakes (mocking commands) or in a container. Do not test internals such as variable names; test what a caller observes. For the next level, writing testable shell scripts goes further.

Designing the script so tests can reach it

Test-driven development works on shell scripts if the script is structured for it. Put the behavior into functions, and have the file do nothing when it is loaded by another script: end it with a guard such as if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then main "$@"; fi, so a test can source the file, call a function and look at the result without running the whole program. Pass inputs as arguments and return outputs on standard output, so functions can be checked with run and checked with assert_output, and keep the effects, such as writing files and calling other programs, at the edge. A script written as one long top-to-bottom sequence has no seams, and the first step of TDD on an old script is often to cut it into functions without changing behavior, protected by one coarse test that runs the whole thing on a sample input. Once there are seams, the cycle works the same as in any language.

Source the script in setup with a path relative to the test file, using BATS_TEST_DIRNAME, so tests run from any directory.

The cycle in small steps

Start with the smallest behavior you can state, write a test that describes it and watch it fail, and read the failure message, because a test that fails for the wrong reason, such as a missing file, proves nothing. Then write the minimum code that makes it pass, even if it is silly, such as returning a constant. Add the next case, which forces the code to become less silly, and repeat. After each green step, tidy the code with the tests as a safety net: rename, extract a function, remove duplication, run the tests again. The rhythm keeps changes small and the feedback quick, and the tests that result describe the behavior in terms a reader can follow. Keep each test fast, a fraction of a second, so running the whole suite after every change is a habit and not a chore.

Test the interface too. A few tests that run the script as a command check usage errors, exit codes and the help text, which users depend on, and for these the exit status of 2 for a usage error is a convention worth keeping.

A worked example, step by step

Suppose you are writing a script that renames files to lowercase. The first test states one behavior: given Report.TXT in a temporary directory, running the function leaves report.txt. It fails because the function does not exist. You create the function with the minimum body, run the test, and watch it pass. The second test adds a file that is already lowercase and checks it is untouched, which passes at once, so you note that the test adds protection and not new code. The third test creates a collision, two names that differ only by case, and the test says the script must refuse and leave both files alone. That test fails, you add the check, and it passes. The fourth covers a name with a space. Each step is a few lines, takes seconds, and leaves a test that describes a decision. After the last green test, refactor: extract the collision check into its own function, rename a variable, run everything again.

Notice what the tests did for you. They found the collision case before a user lost a file, they pinned down the behavior with spaces, and they now protect every later change. That is the payoff of the method, and it is the same on a ten-line script as on a large program.

Keep the discipline when it is tempting to skip. The urge to write the code first is strongest for small scripts, and those are the scripts that grow into things people depend on. A test written in the first five minutes costs almost nothing, and one added after a failure costs an afternoon.

See the bats-core documentation and Bash variables, BASH_SOURCE.