How to Use curl to Download Files and Test URLs Step by Step

curl is the default tool for talking to URLs from a terminal, from downloading an installer to poking a REST endpoint. Its defaults suit interactive use, not scripts: it prints bodies to the terminal, treats an HTTP 404 as success, and waits forever. The examples use curl 8.9 on Linux; related errors are collected in curl failed to connect, and scripting patterns in curl in scripts.

Download a file

curl -fsSL -o go.tar.gz https://go.dev/dl/go1.24.0.linux-amd64.tar.gz
curl -fSLO https://example.com/files/report.pdf

The flag group -fsSL is the one to remember: -f returns an error exit code for HTTP 400 and above instead of saving the error page, -s turns off the progress meter, -S brings back error messages, and -L follows redirects. -o picks the output name and -O keeps the remote one. Add -C - to resume a partial download.

Inspect headers and status

curl -sI https://example.com | head -5
curl -s -o /dev/null -w '%{http_code} %{time_total}s\n' https://example.com

HTTP/2 200
content-type: text/html
200 0.214s

-I sends a HEAD request, which some servers answer differently from GET. -w prints a format after the transfer; %{http_code}, %{time_total}, %{size_download} and %{url_effective} are the commonly used variables. Sending the body to /dev/null keeps the output to the line you want. Add -v to see the request and response headers, including the TLS handshake, when diagnosing.

Send data

curl -s -H 'Authorization: Bearer '"$TOKEN" https://api.example.com/v1/me
curl -s --json '{"name":"alice"}' https://api.example.com/v1/users
curl -s -X POST -d 'a=1&b=2' https://example.com/form
curl -s -F '[email protected]' https://example.com/upload

-d sends a form-encoded body and switches the method to POST on its own, so -X POST is redundant. --json, available from curl 7.82, sets the content type and accept headers and sends the data. -F builds a multipart upload from a file. A token in an argument is visible to other users in the process list, as covered in shell script secrets.

Be safe in scripts

curl -fsS --connect-timeout 5 --max-time 30 --retry 3 --retry-delay 2 \
  -o out.json https://api.example.com/v1/data || echo "request failed"

Timeouts keep a stalled server from hanging a cron job, and --retry repeats on transient errors such as timeouts and 5xx statuses. --retry does not retry on a connection refused unless --retry-connrefused is given. Read about rate limits in retries and rate limits with curl.

Reading the verbose output

When a request misbehaves, curl -v is the first diagnostic. Lines starting with an asterisk are curl's own commentary about name resolution, connection and TLS. Lines starting with a greater-than sign are the request headers it sent, and lines with a less-than sign are the response headers it received. Reading them in order shows where time and errors arise. A long pause after the connection line points at the server or a firewall dropping the handshake. A TLS line naming a protocol version and cipher confirms which was negotiated. A redirect with a location header shows why the body you got was not the one you expected. The --trace-ascii - option goes further and prints the bytes of the bodies as well, which is useful for a malformed JSON request.

The timing variables in -w split the total into phases. %{time_namelookup}, %{time_connect}, %{time_appconnect}, %{time_starttransfer} and %{time_total} are cumulative, so subtract one from the next to find which stage is slow: DNS, TCP, TLS or the server's own work before the first byte.

Redirects, cookies and authentication

By default curl stops at a redirect, which is why -L is part of the usual download flags. It then follows up to fifty hops, and --max-redirs lowers that. On a redirect to another host, curl drops the Authorization header unless --location-trusted is given, which is the safe default and the reason an authenticated download sometimes fails after a hop to a CDN. Cookies are not stored unless asked: -c jar.txt writes them and -b jar.txt sends them, so a login followed by a protected request can be done in two commands that share the jar. Basic authentication with -u user prompts for the password, and putting both on the command line exposes it in the process list, so use a ~/.netrc file with mode 600 and --netrc, or read a header from a file with -H @file.

Config files and reuse

Options that never change can live in ~/.curlrc, one per line, and curl reads it on each run. That is convenient for a corporate proxy or a default --connect-timeout, and a hazard for scripts, because the same command behaves differently on a machine with another file. Pass -q as the first argument in scripts to ignore it, or use --config with an explicit file. The same applies to environment variables: HTTPS_PROXY, NO_PROXY and CURL_CA_BUNDLE all change behaviour silently, and -v shows when a proxy is in use.

Parsing the response

For JSON, pipe the body into jq and let it do the extraction instead of using grep or sed. The status code and body can be captured together by writing the status with -w to the end of the output and splitting it afterwards, or by saving the body to a file with -o and reading %{http_code} from standard output. Check the status before trusting the body: without -f, curl exits with 0 on a 500 response and hands you an HTML error page where JSON was expected, and the next stage fails with a parse error that hides the real cause.

When curl is the wrong choice

For a long chain of requests that share state, pagination or retries with backoff, a short Python or Go program is easier to test than a shell script with a dozen curl calls. For a quick download with resume and mirrors, wget and aria2 have features curl lacks. For debugging HTTP/2 or HTTP/3, curl has flags such as --http2 and --http3 that show which protocol was used, and the build must include support for the latter.

A reusable request helper

Teams that call the same API from many scripts benefit from one small shell function that sets the defaults once: the timeouts, the retry options, the failure flag, the user agent and the authorization header read from a protected file. Each script then calls the function with a path, and the policy is changed in one place. Keep the function quiet on success, print the status and the first lines of the body on failure, and return the curl exit code unchanged so that callers can distinguish a timeout, which is code 28, from a failed connection, code 7, and from an HTTP error under -f, code 22.

Where this does not apply

Piping curl into a shell, curl ... | sh, runs whatever the server sends; read the script first, as discussed in the risks of curl pipe bash. For complex API work with auth flows, a Python client or httpie is easier to maintain. Never disable certificate checks with -k as a fix: see curl SSL certificate problem for the real causes. The complete reference is the curl manual page, and the exit codes are listed in libcurl errors.