How to Use awk User Defined Functions and Math the Safe Way

Awk programs stay readable up to a point and then turn into a wall of conditions. User-defined functions are the way to break them up, and they work differently from other languages in one respect: the only way to get a local variable is to name it as an extra parameter. Misunderstanding that causes the classic awk bug, a value leaking between calls. The syntax extends fields and printf, and the examples assume gawk 5.2 and POSIX behaviour where noted.

Define and call

awk '
function max(a, b) {
  return a > b ? a : b
}
function trim(s) {
  sub(/^[ \t]+/, "", s)
  sub(/[ \t]+$/, "", s)
  return s
}
{ print max($1, $2), "[" trim($3) "]" }
' <<'EOF'
3 9   alpha
12 5  beta
EOF

9 [alpha]
12 [beta]

Functions can be defined anywhere in the program, before or after use. The call must not have a space between the name and the parenthesis. trim modifies its own copy of s because scalars are passed by value, then returns it. Note that max compares fields as numbers only if both look numeric; a field like 12 compared with 9 as strings would give the wrong answer.

Locals are extra parameters

awk '
function total(arr,    i, sum) {
  for (i in arr) sum += arr[i]
  return sum
}
{ nums[NR] = $1 }
END { print total(nums) }
' <<'EOF'
10
20
30
EOF

60

The spaces before i and sum are a convention that marks them as locals. Callers pass only one argument, so the rest start empty in each call. If you forget to declare i or sum, they become globals and the second call starts with the first call's sum. The array arr is passed by reference, so a function can fill an array for its caller.

Math and rounding

awk 'BEGIN {
  print int(7/2), 7 % 3, 2^10, sqrt(144)
  printf "%.2f %.0f %d\n", 2.675, 2.5, 3.99
  print 0.1 + 0.2 == 0.3
  srand(42); print int(rand()*100)
}'

3 1 1024 12
2.67 2 3
0
60

Awk stores numbers as doubles, so 0.1 + 0.2 == 0.3 is false, and %.2f of 2.675 prints 2.67 because the stored value is slightly below. printf %.0f rounds half to even in common implementations, and %d truncates. Do not use awk for money without scaling to integer cents. The sample random value will differ between implementations and seeds; srand() with no argument seeds from the clock.

String versus number

echo "10 9" | awk '{ print ($1 > $2), ($1"" > $2""), ($1+0 > $2+0) }'

1 0 1

Fields from input are "strnum": they compare as numbers if they look numeric. Concatenating with "" forces a string comparison, where "10" < "9". Adding +0 forces a number. Constants and computed values follow different rules from fields, which is why a surprising comparison is best settled by forcing the type. The related sorting traps are in sorting by column.

Recursion and libraries

awk 'function fact(n) { return n <= 1 ? 1 : n * fact(n - 1) } BEGIN { print fact(10) }'
gawk -i /usr/share/awk/join.awk 'BEGIN { a[1]="x"; a[2]="y"; print join(a, 1, 2, "-") }'

Recursion works. With gawk, @include "file.awk" or -i loads a library of functions, which keeps a shared helper in one place. POSIX awk has only -f file, which you can repeat. The gawk extras are covered in gawk features beyond POSIX.

Why awk's function model feels odd

Awk was designed in the 1970s for short programs, and its function rules reflect that. There is no block scope: a variable is global unless it appears in the parameter list. Scalars are copied into the function, arrays are passed by reference, and a function that is called with fewer arguments than it declares gets the rest as uninitialised locals. The last rule is the one the extra-parameter convention uses, and it is why experienced awk programmers separate the real parameters from the locals with extra spaces. The language gives no warning if you forget, so the discipline has to come from the code layout.

The practical consequence is a rule you can check by eye: every variable assigned inside a function must either be a parameter or be intentionally global. Reading a function and listing its assigned names against its parameter list takes thirty seconds and finds the usual bug. Gawk helps when run with --lint, which warns about a function that uses a variable it did not declare, and it is worth running once on any program that has grown past a few functions.

Returning more than one value

A function returns one scalar, so anything richer needs a side channel. The common techniques are writing into an array that the caller passed in, setting a global the caller reads, and encoding several values in one string with a separator and splitting it afterwards. The array method is the cleanest: function parse(line, out) fills out["key"] and out["value"], and returns 1 on success or 0 on failure. The caller keeps its own array, so there is no hidden global, and clearing it with delete out at the start of the function avoids stale entries from the previous call.

Gawk 4.0 and later allow arrays of arrays, with a[i][j] syntax, which removes the need for composite keys joined with SUBSEP. If the program has to run on a POSIX awk such as the one on macOS or BusyBox, stay with the single-level array and a joined key, and say in a comment that this is deliberate.

Organising a program that has grown

Place helper functions at the top, the main rules in the middle and the END block last, so the file reads like a script from general to specific. Give each function a one-line comment saying what it takes and returns, since awk has no types to document that. Keep functions small enough that the whole body is visible at once; when one needs a second screen, split it, because every extra line is another chance for a global to leak.

Test functions in isolation by writing a BEGIN block that calls them with known inputs and prints the result next to the expected value. A dozen lines at the bottom of the file, run with a flag that selects test mode, turn an awk program into something that can be checked after each edit, and they cost nothing in the normal path. Store such a file under version control with a name that says what it does, and the same code that processed a file last year can process it again.

Performance notes

Function calls in awk are not free, and in a program that processes millions of lines a helper called on each one adds measurable time. Gawk and mawk differ here; mawk is typically faster on plain loops and field splitting, and gawk adds features. Measure with a representative file before inlining anything, and prefer clarity unless a profile shows the call is the bottleneck. Recursive functions can hit stack limits in some implementations at depths of a few thousand, which matters for tree walks over deep data, so use a loop when the depth is unbounded.

Where this does not apply

A large awk program with many functions is a sign to switch to Python or Go: awk has no module system, weak error handling and no debugger beyond printing. Keep it for filters that fit on a screen. See the gawk manual on user-defined functions and the numeric functions section.