JSONL vs JSON: Differences and When to Use Each

Three scraped products as a JSON array:

[
  {"name": "Desk lamp", "price": 24.99},
  {"name": "Office chair", "price": 149.00},
  {"name": "Monitor arm", "price": 39.50}
]

The same three records as JSONL. There are no brackets and no commas between records; each line is a complete JSON object:

{"name": "Desk lamp", "price": 24.99}
{"name": "Office chair", "price": 149.00}
{"name": "Monitor arm", "price": 39.50}

The rules are short: the file is UTF-8, every line is one valid JSON value, and lines are separated by \n. A line can't be pretty-printed across several lines, because a line break ends the record.

Use JSONL when:

• The data is a long list of similar records, such as scraped items, log events, or rows from a database export.
• You write records as they arrive. A scraper can append one line per page, and a crash only loses the line being written, not the whole file.
• The file is too large to load into memory. You can process it line by line, or split it with standard tools like head, split, and wc -l.
• A tool expects it. The OpenAI Batch API and fine-tuning uploads, BigQuery's newline-delimited JSON loads, and the Elasticsearch _bulk API all take this format.

Use JSON when:

• The data is one document with nested structure, such as a config file or a single API response.
• You return a response from a web API. Clients expect one parseable body.
• People will read or edit the file by hand, where pretty-printed JSON is easier to scan.

In Python, write one json.dumps() per line and read the file line by line:

import json

records = [{"name": "Desk lamp", "price": 24.99}, {"name": "Office chair", "price": 149.00}]

# Write (or append with "a")
with open("products.jsonl", "w", encoding="utf-8") as f:
    for r in records:
        f.write(json.dumps(r, ensure_ascii=False) + "\n")

# Read one record at a time
with open("products.jsonl", encoding="utf-8") as f:
    for line in f:
        if line.strip():
            print(json.loads(line))

With pandas, pd.read_json("products.jsonl", lines=True) loads JSONL into a DataFrame, and df.to_json("out.jsonl", orient="records", lines=True) writes it back.

To convert on the command line with jq:

# JSON array -> JSONL
jq -c '.[]' products.json > products.jsonl

# JSONL -> JSON array
jq -s '.' products.jsonl > products.json

Scrapy writes JSONL directly when the output file ends in .jsonl, for example scrapy crawl products -o products.jsonl.