One Big Script vs Many Small Scripts Compared Side by Side

Every script that grows asks the question sooner or later: keep adding to this file, or break it up? The answer is not "always split", as some people say, nor "keep it together so it is easy to ship". Each shape has real advantages, and the decision depends on what the script does and who works on it. This post compares them on the points that matter and gives a rule for choosing. It connects structuring a large shell project and splitting into modules.

Side by side

• Deployment. One file is copied anywhere and runs; there are no paths to resolve. Many files need an install step or a known layout, and the entry point must find its siblings.
• Shared state. In one script, functions share variables freely. Between scripts, state travels through arguments, environment variables, files or stdout, which is clearer and more limited.
• Testing. Small scripts and functions can be tested alone. A big script needs a source guard (see TDD for shell) before its functions can be tested at all.
• Readability. Scrolling a 1,500-line file to find where a variable is set is hard. A directory of well-named files is a table of contents.
• Startup cost. One process start for one script; each sub-script is a new process, a few milliseconds each. Irrelevant for a deploy, relevant in a loop called thousands of times.
• Failure. A bug in a small script harms that step. In a big one, one unquoted variable can affect everything after it.
• Change. Small scripts with narrow interfaces can be changed without reading the others, and merge conflicts shrink with the file size.

A rule of thumb

Keep one file while it is short enough to read in one go, say under 150 lines, and does one job. Split when any of these is true:

• It does more than one distinct job (fetch, transform, report).
• Part of it is wanted by another script.
• You cannot test a part without running the whole thing.
• Different people or release cycles own different parts.

The numbers are a guide, not a law. A 400-line installer that runs straight through, once, may be better as one file than as eight; a 60-line script that does three unrelated things should be three.

How to split without making a mess

#!/usr/bin/env bash
set -euo pipefail
here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
. "$here/lib/common.sh"

"$here/steps/10-fetch"   "$@"
"$here/steps/20-build"
"$here/steps/30-migrate"
"$here/steps/40-restart"
log "deploy finished"

deploy
lib/common.sh        # log, die, retry: sourced, never run
steps/10-fetch       # each step is executable and does one thing
steps/20-build
steps/30-migrate
steps/40-restart
tests/

The entry point is thin: it finds its own directory, sources the shared library, and runs the steps in order. Because of set -e, a failing step stops the sequence. Each step is a script that can be run and tested alone: steps/30-migrate on a staging database without the rest. The library holds the functions that more than one step uses, and is sourced, not executed. Numbering the steps shows their order and allows a loop over the directory if you prefer: for s in "$here"/steps/*; do "$s"; done.

Contracts between scripts

Once there are several, their interfaces matter: what arguments each takes, what it prints, which exit codes it uses and which environment variables it reads. Write these at the top of each file in two or three lines, and keep them stable. Pass configuration through exported environment variables set in one place, not through globals set by sourcing in unpredictable order. Return data on stdout and logs on stderr, as described in the Unix way of small tools.

Where this does not apply

If the scripts are distributed to users who run a single file (an installer, a one-off tool), bundling into one file at release time is practical: develop as modules and concatenate in the build. If the logic is growing past what the shell handles well, splitting the shell is the wrong fix; consider a language with real modules, as in signs you should rewrite it. Do not split just to reach a line count: files with no clear purpose are worse than one honest big file.

What changes when a script becomes two

Splitting a script creates interfaces, and each interface is a place for misunderstanding. One script calling another passes arguments, environment variables and standard input, and receives output, an exit status and side effects on the file system. All of those are now a contract, and a change on one side that the other does not expect breaks the pair. Inside one script, a function that changes its arguments is a local matter. Between two scripts it is a coordinated change, possibly across repositories and owners. The costs of splitting are therefore real: more files to deploy and keep in step, version skew between a caller and a callee, a process started for each call, harder debugging because a failure is in another file with its own output, and the loss of shared variables. The benefits are real too, and the decision is whether the benefits are worth the contract. Many scripts that were split early are kept together again by people who found the interfaces were harder to maintain than the code had been.

Know the difference between running a script and sourcing it. A sourced file shares the caller's variables and shell options, which is convenient and risky, and an executed one runs in a separate process with only what you pass.

Signals that a split is worth it

Several signals point to splitting. A part of the script is reused by other scripts and copied between them, so it deserves one maintained home. A part has a different lifecycle, changing monthly while the rest is stable, or has a different owner. A part needs a different permission, such as one step that must run as root while the rest must not. A part can be checked on its own with clear inputs and outputs and cannot be checked while buried in a long script. A part has a natural meaning as a command someone might run alone, such as a function that lists stale branches. In contrast, splitting a script because it is long, with no reuse or boundary, spreads the same tangle across several files. Length is a prompt to look for structure, not an argument for division by itself. A single file of five hundred lines with clear functions is better than ten files of fifty with unclear dependencies between them.

Split along the seams of the problem, not the line count: configuration, input handling, the core operation and reporting often separate cleanly, and each can be a function before it is a file.

Designing the contract between scripts

When you do split, write the contract down. For each script, state the arguments and options, the environment variables it reads, what it reads from standard input, what it writes to standard output and standard error, its exit codes, and any files it creates or changes. Keep the output format stable and machine-readable. Fail loudly on a missing or invalid argument, with a usage message. Give the callee no hidden dependency on the caller's working directory or environment: pass paths explicitly, and resolve a script's own location with dirname of its path, not the current directory, so it works however it is called. Test each script alone against its contract, and test the combination with a small end-to-end case. Version the interface when it changes, and keep the old behavior for a while where other scripts depend on it. If you find that two scripts must always change together, they are one unit pretending to be two, and merging them removes the contract that was costing you.

See the Google Shell Style Guide on when to use shell and how to organize it, and the Bash source builtin.