How to Use gron and jq for Exploring Unknown JSON Fast

An API returns two thousand lines of JSON, and somewhere in it is the field you need. Scrolling through nested braces is slow, and writing a jq filter for a structure you have not seen is guesswork. gron solves the first step: it flattens JSON into plain lines that grep understands, each carrying its path. This post shows the loop of finding with gron and extracting with jq. It builds on jq for API responses and calling REST APIs from the shell. Examples use gron 0.7 and jq 1.7.

Flatten and search

curl -s https://api.example.org/v1/orders/8812 | gron | head -n 8

json = {};
json.customer = {};
json.customer.email = "[email protected]";
json.customer.name = "Ada Lovelace";
json.id = 8812;
json.items = [];
json.items[0] = {};
json.items[0].sku = "A-100";

Every value gets its own line with the complete path from the root, called json. Now search the way you would search any text:

curl -s https://api.example.org/v1/orders/8812 | gron | grep -i 'sku'
curl -s https://api.example.org/v1/orders/8812 | gron | grep '"A-100"'

json.items[0].sku = "A-100";
json.items[1].sku = "B-220";

You see at once that there are several items, each with a sku field, and where they live. Searching for a value you know (an email, an ID) finds the path to it even in a document you have never seen. Add gron --stream for large files, and pass a file or URL directly: gron https://api.example.org/v1/orders/8812.

Turn the path into a jq filter

curl -s https://api.example.org/v1/orders/8812 | jq -r '.items[].sku'
curl -s https://api.example.org/v1/orders/8812 | jq '{id, email: .customer.email, skus: [.items[].sku]}'

A-100
B-220
{
  "id": 8812,
  "email": "[email protected]",
  "skus": ["A-100", "B-220"]
}

The gron path json.items[0].sku becomes the jq path .items[0].sku (drop the leading json), and [] on an array iterates all elements. Arrays make gron lines repeat, which is also the hint that you want []. Build the final filter in jq: it is faster on large data, and produces JSON.

Filter and rebuild with ungron

gron order.json | grep -v '\.internal' | gron -u
gron order.json | grep -E '^json\.(id|customer)' | gron -u

gron -u (also --ungron) rebuilds JSON from the lines left after filtering, which makes grep a quick way to drop or keep parts of a document without learning jq syntax. The result keeps the structure only for the paths that remain, so objects that lost all their fields appear as empty ones.

Explore with jq alone

jq -r 'paths(scalars) | map(tostring) | join(".")' order.json | sort -u | head
jq 'keys' order.json
jq '.items | length, (.[0] | keys)' order.json
jq '[.. | strings | select(test("example.org"))] | unique' order.json
jq -r '.. | objects | select(has("sku")) | .sku' order.json

When gron is not installed, paths(scalars) lists the path of every leaf value, keys shows the names at one level and .. (recursive descent) visits every value, which lets you select by shape instead of by position. Start at the top: how many keys, which types. A one-time look like this saves many failed filters. For reading JSON interactively, jless or fx offer a collapsible viewer, as compared in jq, gron and jless.

Where this does not apply

gron output for big documents is large (one line per value), so pipe through head or grep. Keys with special characters are printed in bracket form, json["a-b"], which you must write the same way in jq: .["a-b"]. Do not feed untrusted data into eval because gron's output looks like JavaScript. gron cannot do arithmetic or reshaping, which is jq's job. For responses that are not JSON (HTML, XML), use the right tool for those.

Why a flat view helps with an unfamiliar document

Nested JSON hides where things are. To find the field that holds a user's email in a response of three hundred lines, you scroll and guess, then write a jq filter, run it, fix the path and run it again. gron turns the document into one assignment per line, with the full path on the left, as in json.data.users[0].email = "[email protected]". Plain grep then does the searching, and each hit carries its own path, so you can read off the filter to write. That reverses the usual order of work: find the value first, learn its path second. The output is also valid JavaScript, which is how the tool got its name, and it can be turned back into JSON with gron --ungron, so you can filter lines and rebuild a smaller document that holds only what matched. For exploration of an API you have never called, this is faster than reading documentation that may not exist or may not match what the server really returns.

Flat output has costs. It repeats the path on every line, so a large document grows several times in size, and the tool reads the whole input into memory, which is a limit for files of hundreds of megabytes.

Reading the structure with jq alone

When gron is not installed, jq can do much of the same exploration. jq 'keys' lists the top-level fields, jq 'map(keys) | add | unique' over an array lists every field that appears in any element, which is useful when records are not uniform. jq '.[0]' shows one sample, and jq 'type' tells you whether you hold an object, an array or a scalar. To list every path to a scalar value, use jq -c 'paths(scalars)', and to see paths with values, jq -c '[paths(scalars) as $p | {path: $p, value: getpath($p)}]' is verbose and does the job. The recursive descent operator, .., finds a field wherever it is: jq '.. | .email? // empty' prints every email at any depth. Compare how the same field is named in different records, since inconsistent naming is common in real APIs. A small number of these commands, run in sequence, gives you a map of the response before you write the final filter.

For interactive reading, a pager such as jless lets you fold and unfold nodes and shows the path of the selected node, which is the quickest way to learn a document you will look at for a while.

Dealing with arrays, nulls and surprises

Real responses vary from one record to the next. A field may be missing in some elements, null in others and of a different type in a few, such as a number that is a string when it is large. Filters written from the first record fail on the tenth. Look at several samples and not one, and check for the unusual: jq '[.[] | .field | type] | group_by(.) | map({type: .[0], count: length})' shows the distribution of types for a field. Be aware that the paths gron prints use numeric indices, which are specific to the document you looked at. When you turn a path into a filter, replace an index by [] to cover all elements, as in .data.users[].email. Beware of pagination: the document you explored may be one page of many, with the total and a cursor at the top, and the filter you write must loop. Keep a saved sample of the response next to the script, so you can re-explore it offline and write tests against it.

Turning what you found into a script

Once you know the path, write the filter against a saved sample and check it on several responses before it goes into a script. Save one or two real responses to files, run the filter on them, and use the output to build a test that fails if the API changes shape. Guard against absent fields with the alternative operator, .email // "none", and decide what the script does when the structure is not what it expected: print the unexpected part to standard error and exit nonzero, instead of continuing with empty values. Redact tokens and personal data in saved samples before committing them, since API responses often contain more than you meant to keep.

See the gron project and the jq manual.