How to Use Regex in awk Matching and match Function

Awk's regular expression support is built in and always on, so no flag is needed: any pattern between slashes is a test, and the operators and functions below give you the rest. Knowing its flavor, ERE with a few rough edges, saves time. This post covers matching, extracting and replacing, for readers who already know fields and printf. Examples use GNU awk 5.2; notes say where mawk and macOS awk differ.

Match and negate

awk '/^2026-11/' app.log
awk '$3 ~ /^(WARN|ERROR)$/' app.log
awk '$5 !~ /ms$/ { print $1, $5 }' app.log

A bare regex tests the whole line. ~ tests a specific field or expression, and !~ is the negation. Anchors matter: without ^...$, $3 ~ /ERROR/ would also match ERRORS. Case-insensitive matching is available in gawk with IGNORECASE = 1 in a BEGIN block, or portably with tolower($0) ~ /error/.

Extract with match

echo 'user id=4821 name=ada' | awk '{
  if (match($0, /id=[0-9]+/)) {
    print RSTART, RLENGTH, substr($0, RSTART + 3, RLENGTH - 3)
  }
}'

6 7 4821

match returns the 1-based position of the leftmost longest match, or zero, and sets RSTART and RLENGTH. Here the match is id=4821, seven characters at position 6, and substr skips the three-character prefix. Awk has no lookbehind to drop the prefix, so the offset arithmetic is the portable method. In gawk only, a third argument captures groups: match($0, /id=([0-9]+)/, m) fills m[1]. mawk and macOS awk reject that form.

Replace with sub and gsub

echo 'a.b.c' | awk '{ gsub(/\./, "-"); print }'
echo 'cost: 45' | awk '{ sub(/[0-9]+/, "[&]"); print }'
echo 'a.b.c' | awk '{ n = gsub(/\./, "/"); print n, $0 }'

a-b-c
cost: [45]
2 a/b/c

sub replaces the first match and gsub all of them, in $0 unless a target is given as a third argument. Both return the number of replacements. In the replacement string, & is the matched text and \\& a literal ampersand. Changing $0 re-splits the fields, while changing one field rebuilds $0 with OFS.

Dynamic regular expressions

awk -v host="web1.example.org" '$1 ~ host' hosts.txt
awk -v pat='^a\\.b$' '$1 ~ pat' list.txt

A string used as a regex is converted at run time. Escapes get processed twice, once as a string and once as a regex, so a literal dot needs "\\.". And in the first example the dots in the hostname match any character, which is usually not what you want. For an exact literal match use index($1, host) or $1 == host. With -v, awk also processes escape sequences in the value, so passing patterns through ENVIRON is safer.

Interval expressions and classes

Use POSIX classes: [[:digit:]], [[:alpha:]], [[:space:]]. There is no \d or \w in standard awk, though gawk accepts \s, \S and \w. Interval expressions like {2,4} are on by default in gawk and in recent mawk releases, but an older mawk or awk reads the braces literally. Test with echo aaa | awk '/a{3}/' on your system.

Where this does not apply

Regexes in awk have no lookaround or backreferences within the pattern, and no lazy quantifiers. If you need them, use perl or Python; see the Python re module. Comparing numbers or strings is not a regex job: $3 > 100 is a comparison, and quoting it as a pattern would be a mistake. The wider differences are laid out in regex across tools.

What awk's regular expressions are, and which ones

Awk uses extended regular expressions with POSIX bracket classes, and the exact feature set depends on the implementation. Interval expressions such as {3} and {2,5} are supported by gawk and by the one true awk used on macOS, and older releases of mawk, the default on some Debian-based systems, treated the braces as literal characters, which silently changed the meaning of a pattern. Recent mawk versions support them, and the safe move on a machine you do not control is to check with a one-line test or to write the repetition out. Backreferences inside the pattern are not part of POSIX awk. Lookahead and lazy quantifiers do not exist, as in other extended-regex tools. Shorthand classes like \d and \w are gawk extensions at best, and \s and \S work in gawk, so write [[:digit:]] and [[:space:]] for portable code.

Case-insensitive matching differs too: gawk has the IGNORECASE variable, other implementations do not, and the portable way is to lowercase the text first with tolower and match against a lowercase pattern.

Dynamic patterns and the double escaping trap

A regular expression can be a string stored in a variable, and then it is parsed twice: once when awk reads the string literal, which turns each backslash pair into a single backslash, and once when the engine reads the resulting pattern. So a literal dot in a dynamic pattern needs two backslashes in the string, as in "[0-9]+\\.[0-9]+", whereas in a slash-delimited regular expression one is enough. Forgetting this is the cause of patterns that work as literals and fail when moved into a variable or passed with -v. The command-line assignment form also processes escape sequences, which is another reason to read values from ENVIRON when they hold backslashes. When a variable holds a plain string and you want to match it literally, escape the special characters first or use index(), which does a plain substring search with no regular expression at all.

The anchors and the dot behave as expected, with one point to remember: awk's dot matches any character including a newline when a record contains one, which happens only with a custom record separator.

Match positions and substitution details

The match(string, regex) function returns the position of the first match, or zero, and sets RSTART and RLENGTH, which you use with substr to extract the matched text. Only gawk lets you pass a third argument, an array that receives the captured groups, so portable extraction of a group needs a second step, such as matching a wider pattern and trimming the surrounding text with sub. In sub and gsub, an ampersand in the replacement means the matched text and a literal ampersand needs escaping, which doubles up again inside a string, so "\\&" produces a literal one. Both functions modify the target in place and return the number of replacements, which makes the return value a handy test of whether anything changed. They default to $0, and changing it re-splits the fields.

When a pattern misbehaves, test it with echo 'sample' | awk '/pattern/' before putting it in a larger program, and test again with a value that should not match.

A small extraction example and its limits

To pull a duration such as 350ms out of a log line, a program can call match($0, /[0-9]+ms/) and then take substr($0, RSTART, RLENGTH - 2) to get the number. Wrapping it in an if keeps lines without a match from printing a stale value, since RSTART and RLENGTH are only meaningful after a successful match. Sum the values in an array or a counter, and print the total in END. This approach is portable across awk implementations, and it is readable. When the extraction needs several groups, gawk's array argument to match is far more convenient, and a program that uses it should say so, since it fails with a syntax error on other implementations. For anything beyond a couple of fields, structured logs or a different tool will serve better.

See the gawk regular expressions chapter and POSIX awk regular expressions.