How to Use awk Patterns Ranges and BEGIN and END Blocks

The general shape of awk is more important than any one-liner: for every input line, test each pattern in order, and run the action of those that match. Learning that model makes programs readable, and explains surprises such as lines printed twice. This post covers the pattern types and the two special blocks. It continues fields and printf. Examples use GNU awk 5.2 and a small log.

The sample

2026-11-03 09:00:01 INFO start
2026-11-03 09:00:05 WARN slow query 1200ms
2026-11-03 09:01:10 ERROR upstream timeout
--- begin trace ---
frame 1
frame 2
--- end trace ---
2026-11-03 09:02:00 INFO stop

Patterns

awk '/ERROR/' app.log
awk '$3 == "WARN" || $3 == "ERROR" { print $3, $4 }' app.log
awk 'NR == 1 || /stop/' app.log
awk '$5 + 0 > 1000' app.log

2026-11-03 09:01:10 ERROR upstream timeout
WARN slow
ERROR upstream
2026-11-03 09:00:01 INFO start
2026-11-03 09:02:00 INFO stop

A regular expression between slashes matches against the whole line. A comparison tests fields, and patterns combine with &&, || and !. A pattern with no action prints the line. The last command in the block, $5 + 0 > 1000, would match a field like 1200ms only if awk could read it as a number; awk converts the leading digits, so "1200ms" + 0 is 1200, which is a handy parsing trick.

Ranges

awk '/begin trace/,/end trace/' app.log
awk '/begin trace/{f=1; next} /end trace/{f=0} f' app.log

--- begin trace ---
frame 1
frame 2
--- end trace ---
frame 1
frame 2

A range starts at the first line matching the first pattern and stops at the next line matching the second, inclusive. If the end never comes, it runs to the end of the file. The flag form on the second line excludes the markers, because the rule with next skips the begin line and the flag is turned off before the final test. Use the flag form when you need the boundaries excluded or different logic inside.

BEGIN and END

awk 'BEGIN { print "level count" }
     $3 ~ /^(INFO|WARN|ERROR)$/ { n[$3]++ }
     END { for (l in n) print l, n[l] }' app.log

level count
INFO 2
WARN 1
ERROR 1

BEGIN runs before any input is read, so it is the place to set FS, OFS and print a header; a program with only a BEGIN block never reads input at all. END runs after the last line, where totals are printed, and $0 still holds the last line in most implementations.

next and exit

awk '/^---/ { next } NR > 20 { exit } { print NR": "$0 }' app.log

next stops processing the current line and goes on to the next, skipping all remaining rules, which keeps the rules independent. exit ends input processing and runs the END block, so a total is still printed. An exit 3 inside END sets the exit status, which a shell script can test.

Overlapping rules

awk '/ERROR/ { print "E: " $0 } /upstream/ { print "U: " $0 }' app.log

A line that matches both rules prints twice, once for each. This is not a bug, it is the model. If you want an either-or, end the first action with next, or write the rules as an if / else inside one action.

Where this does not apply

Range patterns do not nest and misbehave if both markers can appear on one line, since the end pattern is checked on the start line only in some implementations (gawk tests it on the same line). For multi-line records, set RS to a blank (paragraph mode) rather than building ranges. For counting with arrays see associative arrays, and for two input files, NR and FNR.

How a program is organized

An awk program is a list of pattern and action pairs. For each input record, awk tests the patterns in order and runs the action of every one that matches, so several rules can fire on the same line, and the order of rules matters when they change the record or set variables. A pattern with no action prints the line, and an action with no pattern runs for every line. Patterns can be regular expressions in slashes, comparisons, boolean combinations of those with &&, || and !, and ranges. BEGIN and END are special: the first runs before any input is read and the second after the last record. A program made only of BEGIN rules does not read input at all, which is how awk is used as a calculator or to generate text. This structure explains why awk code is short, and why accidents such as two rules printing the same line are easy to make.

Use $0 ~ /re/ or $3 ~ /re/ to match a regular expression against the whole line or one field, and remember that a bare /re/ as a pattern tests the entire line.

Range patterns and their quirks

A range pattern /start/, /end/ turns on at the first line that matches the start and turns off after the first line that matches the end, and both end lines are included. Several details cause surprises. If the start and end match on the same line, the range covers just that line, and does not look for a later end. If the end never appears, the range runs to the end of the file. A range can fire again later in the input, each time the start pattern matches. Ranges can use line numbers, such as NR==5, NR==10. To exclude the boundary lines, use a flag variable instead: set it on the start line without printing, clear it on the end line, and print while it is set. That gives you control over each edge, and handles nested markers if you use a counter in place of a flag.

Keep the logic readable. A flag-based version takes three short rules, and anyone can follow it, while a clever range with offsets saves a line and costs the next reader some thought.

Controlling flow with next and exit

next stops processing the current record and moves on to the next one, skipping the remaining rules, which is how you handle special lines such as comments and blank lines at the top of the program: /^#/ { next }. exit stops reading input, though the END rules still run, and it takes an optional status. Because END runs after an exit, a program that exits early on an error can still print a summary, and a script that wants to skip END must use a flag. In END, $0 and the field variables keep the last record in most implementations, which is handy for printing the last line, though older ones cleared them. Use exit 1 in a rule that detects bad input, so a calling script can test the status with the shell's $?.

Set variables from the command line with -v name=value, which assigns before BEGIN, or with name=value between file names, which takes effect when awk reaches that argument. The two timings differ, and the second is useful for processing several files with different settings.

Common mistakes with patterns and ranges

Three mistakes recur. First, a range whose end pattern also matches the start line, which collapses the range to one line when you meant a block. Second, a range with a start pattern that matches again inside the block, which restarts nothing but is easy to misread, and the output looks right until a second block appears. Third, two rules that both print, producing duplicate lines, or a rule without next that lets the line fall through to a generic rule. When output looks doubled, add next to the specific rule. Use the debug approach of printing the rule number in front of each output line while developing, and remove it afterwards. Short programs are easy to get right, and a long awk program with many overlapping rules is usually better as a script with comments and a name.

See the gawk patterns and actions and POSIX awk patterns.