Wait for a File to Appear in a Bash Script
Wait for a file with an until loop on test -e, and have the writer create it under a temporary name and rename it, so the reader never sees half of it.
One process produces a file and another waits for it, and the reader must not start on a partial file.
( echo partial > out.txt; sleep 1; echo rest >> out.txt ) &
until [ -e out.txt ]; do sleep 0.1; done
echo "direct write, reader saw: $(tr '\n' ' ' < out.txt)"
wait
rm out.txt
( echo whole > out.tmp; sleep 1; echo rest >> out.tmp; mv out.tmp out.txt ) &
until [ -e out.txt ]; do sleep 0.1; done
echo "write then mv, reader saw: $(tr '\n' ' ' < out.txt)"
waitdirect write, reader saw: partial
write then mv, reader saw: whole restuntil [ -e file ] ends the moment the name exists, and a writer that creates the file and then writes into it makes that moment too early. The first part of the run shows it: the reader saw only partial. When the writer builds the content under out.tmp and finishes with mv, the final name appears only when the content is complete, and the reader saw whole rest. A rename within one file system is atomic, as rename in the specification requires.
The temporary file must be on the same file system as the target, or mv falls back to a copy and loses the guarantee. If you do not control the writer, wait until the size stops growing, or look for a separate marker file. Polling every 0.1 seconds is fine for a short wait and wasteful for a long one.