How to Call Go From Shell Scripts and Share Data Easily

Shell is good at gluing programs together and bad at parsing, retrying and concurrency. Go is the opposite. The combination works when each side keeps to what it does well: a small Go program does the hard part, and the script calls it like any other command. The interface is what makes it pleasant, and it comes from the conventions in replacing shell scripts with Go. Examples use Go 1.24 and Bash 5.2.

Build once, call the binary

mkdir -p bin
CGO_ENABLED=0 go build -o bin/urlcheck ./cmd/urlcheck
./bin/urlcheck https://example.org

A static binary has no runtime dependencies, so you can copy it to any Linux server of the same architecture. go run compiles to a temporary directory on every invocation; the build cache makes that fast after the first time, but it still needs the toolchain and adds startup latency. Use go run while developing and the binary for production scripts. Put the build in a Makefile target so it stays reproducible; see Makefile automation.

A tool with a shell-friendly interface

package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"os"
	"time"
)

type result struct {
	URL    string `json:"url"`
	Status int    `json:"status"`
	Millis int64  `json:"ms"`
}

func main() {
	if len(os.Args) < 2 {
		fmt.Fprintln(os.Stderr, "usage: urlcheck URL...")
		os.Exit(2)
	}
	client := http.Client{Timeout: 10 * time.Second}
	failed := false
	for _, u := range os.Args[1:] {
		start := time.Now()
		resp, err := client.Get(u)
		if err != nil {
			fmt.Fprintln(os.Stderr, "urlcheck:", err)
			failed = true
			continue
		}
		resp.Body.Close()
		out, _ := json.Marshal(result{u, resp.StatusCode, time.Since(start).Milliseconds()})
		fmt.Println(string(out))
		if resp.StatusCode >= 400 {
			failed = true
		}
	}
	if failed {
		os.Exit(1)
	}
}

One JSON object per line (JSON Lines) is easy for the shell to consume with jq and easy for humans to scan. Errors go to stderr, so a script can discard or capture them separately.

Calling it from Bash

if out=$(./bin/urlcheck https://example.org https://example.org/missing 2>errors.log); then
  echo "all healthy"
else
  echo "problems found" >&2
fi
printf '%s\n' "$out" | jq -r 'select(.status >= 400) | "\(.url) \(.status)"'

The exit status drives the if, and stdout carries the data. Arguments are passed as separate words, so quote them in the script. For large inputs, read stdin in the Go program instead of building a long command line: cat urls.txt | ./bin/urlcheck - with a small branch on os.Args[1] == "-". The jq patterns apply directly.

Exit codes

Pick a few and document them: 0 success, 1 the check failed (a normal outcome), 2 usage or internal error. Scripts can then tell "the site is down" from "the tool was misused". Never call os.Exit inside a function that has deferred cleanup unless you accept that the deferred calls are skipped.

Where this does not apply

If the whole job is a few commands joined by pipes, a Go program only adds a build step. Cross-platform distribution needs a build per target (GOOS, GOARCH); see cross compiling. When the Go program is called thousands of times in a loop, the process start cost adds up: pass many items per call instead. For the other direction, running shell commands from Go, see os/exec.

Deciding where the logic should live

A common reason to reach for Go is a shell script that outgrew its welcome: quoting bugs, slow loops over thousands of items, parsing that depends on text formats. The useful split is to keep Bash for what it is good at, starting programs, wiring pipes and handling files, and move the parts with logic and data structures into a binary that the script calls. The binary does one job well, reads arguments and standard input, writes results to standard output and diagnostics to standard error, and exits with a meaningful status. Called from Bash it behaves like any other command, so the script keeps its familiar shape, and the hard parts become testable code with proper types and unit tests. Do the move gradually: replace the one function that hurts, keep the rest.

The cost is a build and a distribution step. Someone has to compile the binary for each platform, and the script has to find it. That is real, though Go makes it easier than most languages, since a single static binary has no runtime to install.

Designing the interface for scripts

Treat the command line as a public interface. Use flags with clear names through the standard flag package or a library, and print a usage message on error. Write results in a form scripts parse easily: one item per line for lists, tab-separated fields for tables, JSON for structured data that jq will consume. Offer a null-separated output option for names that may hold spaces, in the same spirit as find -print0. Keep standard output for data only, since a progress message there breaks the next command in the pipeline. Check whether the output is a terminal with term.IsTerminal before printing colors or progress bars, and turn them off when it is piped. Support reading from standard input when no file is given, which lets the tool sit in the middle of a pipeline.

Document the exit codes. Pick a small set, such as 0 for success, 1 for a handled failure with a message, and 2 for a usage error, and keep to them, since a calling script will branch on them.

Calling it from the script

In Bash, capture the output with command substitution and check the status immediately, or use the command directly in an if. With set -e the script stops on a nonzero status, so use || true or an explicit test where a nonzero status is expected. Pass user data as arguments in quotes, never by building a command string with eval. For long-running tools, let output stream instead of capturing it, so the user sees progress. When the tool reads a large file, give it the file name and let it read directly, since piping everything through the shell costs copies. Put the binary on the PATH or refer to it by an absolute path set at the top of the script, and check once that it exists with command -v so a missing binary gives a clear message.

Version the interface. If the output format changes, a script built for the old one breaks, so add a flag for the new format or change the name, and note the change in a changelog.

Keeping the pair maintainable

Treat the Go program and the script as one product. Keep both in the same repository, build the binary in the same step that tests the script, and test them together with a small end-to-end case that runs the script against a sample input. A Makefile target that builds for the platforms you use and copies the binaries to a known directory keeps the script's assumptions simple. Version the binary and have it print its version with a flag, so the script can check that it has a compatible one and fail with a clear message. When the script and the tool disagree about a format, the version check turns a confusing failure into a one-line error.

Document how a new person builds and runs both, in a few lines at the top of the readme, and keep a note of why the tool exists in the first place. Six months later, nobody remembers which part of the shell script was too slow or too error-prone, and the reason decides whether the tool is still worth keeping.

See the go build documentation and the JSON Lines format.