Skip to content
shellhacks.dev
Performance

Avoid cat in a pipe

Replace cat file | cmd with cmd < file. The measured cost of the extra process is small per call but adds up in a loop of thousands of calls.

1 min read
Summarize in AI

cat file | wc -l is everywhere, and it starts one more process than the redirection does.

Shell
printf 'line\n%.0s' {1..1000} > f.txt
TIMEFORMAT='%3R seconds'
echo "cat pipe:"
time (for i in {1..2000}; do cat f.txt | wc -l >/dev/null; done)
echo "redirect:"
time (for i in {1..2000}; do wc -l < f.txt >/dev/null; done)
Text
cat pipe:
1.860 seconds
redirect:
0.856 seconds

In the pipe version the shell forks for cat and for wc, and wires them together with a pipe. The redirection opens the file in the shell itself and runs only wc. The shell grammar is in the pipelines section of the manual. The numbers are for 2,000 iterations in a Debian 12 container on an arm64 laptop, so they show the ratio and not what your machine does.

For one call the difference is a millisecond or less and not worth a style fight. The pipe is also clearer when the first command is a real producer. A file name that starts with a dash or holds an unquoted space is a bigger risk than the extra process, so quote it in both forms.

Share

More snippets

Performance

Count lines in a huge file fast

Count the lines of a big file with wc -l, which is the fastest of the usual tools. It counts newlines, so a last line without a newline is not counted.

2 min read
Performance

Find syscalls that take time

Show how long each system call takes with strace -T. The figure in angle brackets is the time spent inside the call, so a slow write or sleep stands out.

1 min read
Performance

Trace file opens of a command

Find which files a command opens with strace -e trace=openat, and add -e status=failed to see the config files it looked for and did not find.

2 min read

Get new posts by email

One email when a new post is published. No spam.

Unsubscribe in one click. No spam, ever.