Why Your Script Works in Bash but Fails in cron or systemd

When the same script works by hand and fails on a schedule, the script is rarely the problem. The environment is. This post lists what differs, in the order that finds the cause fastest, and it connects to the full diagnosis in cron jobs that do not run. The examples assume a Debian-style cron on Linux and a systemd unit from systemd 255.

See the environment cron gives you

* * * * * env > /tmp/cron-env.txt 2>&1

HOME=/home/deploy
LOGNAME=deploy
PATH=/usr/bin:/bin
SHELL=/bin/sh
PWD=/home/deploy

Run this for one minute, read the file and delete the entry. Compare it with env in your terminal. The differences, usually PATH and a handful of exported variables, explain most failures. The same technique works for systemd by adding ExecStart=/usr/bin/env in a throwaway unit.

PATH and absolute paths

#!/usr/bin/env bash
export PATH=/usr/local/bin:/usr/bin:/bin
cd "$(dirname "$0")" || exit 1
/usr/local/bin/backup-tool --target /srv/data

Set PATH at the top of the script, or use absolute paths to every command that is not in /usr/bin or /bin. Tools installed with nvm, pyenv, rbenv or Homebrew live in per-user directories that only an interactive shell adds. The cd line removes the dependence on the starting directory, which is the home directory under cron and / under systemd.

Shell and profile

Cron runs the command with /bin/sh, which on Debian is dash. A crontab line using [[ ]] or arrays fails even though the script has a Bash shebang; the shebang only applies when the script is executed as a file. Put SHELL=/bin/bash at the top of the crontab or call the script directly. Cron also never reads ~/.bashrc or ~/.profile, so a variable defined there is empty. Define needed variables inside the script, load a dedicated env file with set -a; . /etc/myapp.env; set +a, or use EnvironmentFile= in a systemd unit. See interactive versus non-interactive shells for what each mode reads.

No terminal, no user

Commands that ask for input, check [ -t 0 ], call sudo with a password prompt, or use colours and a progress bar behave differently without a TTY. Use sudo -n to fail fast instead of waiting, and add --no-color or --quiet flags where they exist. Also check the user: a job in root's crontab sees different files and permissions than your login. SSH agents and keyring sockets are normally absent, so a script that pushes over SSH needs a deploy key referenced with -i and IdentitiesOnly in the SSH config.

Capture the failure

*/5 * * * * /srv/bin/job.sh >> /var/log/job.log 2>&1

Without a redirect, output goes to local mail, which is often not configured, and the error disappears. Appending both streams to a log gives you the message you need. For systemd, read journalctl -u job.service. The crontab manual page documents the variables cron sets, and the systemd.exec reference covers the unit environment.

The differences, in order of how often they matter

Four things separate a cron job from your terminal, and they account for nearly every case. The first is the environment: variables like PATH are shorter or missing, and anything set in a profile file is absent. The second is the working directory: your terminal starts where you are, cron starts in the home directory, and systemd starts in the root. A script that reads ./config.yml works from the project folder and fails everywhere else. The third is the shell: cron uses /bin/sh by default, which may not be Bash. The fourth is the lack of a terminal, which changes how programs behave when they check for one.

A fifth difference appears with timing. A job at three in the morning runs when other jobs run, so locks, disk and network may be busy in ways your afternoon test never sees. A script that passes by hand and fails at night may be colliding with the backup that starts at the same time. Staggering schedules by a few minutes and logging the start and end of each job makes this visible.

Making the script independent of where it runs

The reliable fix is for the script to set up its own world. Begin with strict mode, then set the path, then change to a known directory, then load any configuration by absolute path. Print the user, the directory and the path to the log in the first lines, which costs nothing and answers the first questions of any later investigation. Use full paths for files the script reads and writes, derived from one variable at the top so that moving the installation changes one line.

Avoid relying on the current user's dotfiles. If the script needs a credential, read it from a file with strict permissions at a documented location, not from an environment variable that happens to be exported in your shell. If it needs a tool's configuration, pass the configuration path as a flag. Scripts that depend on nothing implicit run the same in cron, in a timer, in a container and in CI, which is exactly the property you want.

Reproducing the failure on demand

Do not wait for the schedule. Run the script as cron would: env -i with a minimal set of variables, from the home directory, with input from /dev/null and output redirected, as in cd ~ && env -i HOME="$HOME" PATH=/usr/bin:/bin SHELL=/bin/sh /bin/sh -c '/srv/bin/job.sh' < /dev/null. If it fails, you have the bug in front of you and can fix it in minutes. If it passes, the cause is timing or permissions, and the next step is to compare the user and the clock with the real job.

Interactions that only appear on a schedule

Overlapping runs are the classic. A job that takes longer than its interval starts a second copy while the first is still working, and the two corrupt each other's files or exhaust a connection pool. A lock with flock -n makes the second copy exit at once. Time zones and daylight saving cause jobs to run twice or not at all on two nights a year. Percent signs in a crontab line are newlines to cron, so a date +%F in the command must be escaped as \%F. Mail and logging matter as well: a job that prints a lot of output into a mailbox nobody reads fills the disk with queued messages. Direct the output to a file and rotate it.

Where this does not apply

If the job does not start at all, the cause is the schedule or the daemon, not the environment; check the cron log first. Percent signs in a crontab command are treated as newlines and need escaping, a separate trap from the environment problems here.