Skip to content
On this page

To scrape website data into Excel, fetch the page through a scraping API that returns structured JSON, load the rows into a Python list of dictionaries, then write them to an .xlsx file with pandas (DataFrame.to_excel) or openpyxl. This works even on JavaScript-rendered tables and pages behind browser verification, where Excel's built-in Power Query "Get Data from Web" returns an empty preview because it only sees the raw HTML, not the rendered DOM. The pattern is always the same three steps: get JSON, shape it into rows, write the workbook.

Quick facts

Best Python librarypandas (one-line to_excel) or openpyxl (fine-grained cell control)
Output format.xlsx (native Excel) or .csv for no-code import via Data > From Text/CSV
Why not Power QueryIt reads raw HTML only; JS-rendered or verification-gated tables come back empty
Engine neededpip install openpyxl (pandas uses it as the .xlsx writer engine)
Scrappey outputautoparse:true returns parsed JSON ready to drop into a DataFrame

Write scraped rows to .xlsx with pandas

The fastest path is pandas: collect your scraped records as a list of dictionaries and call DataFrame.to_excel("out.xlsx", index=False). pandas uses openpyxl under the hood as the .xlsx engine, so pip install pandas openpyxl is all you need.

The key idea is that a scraping API gives you JSON, and JSON maps cleanly onto rows and columns. Each dictionary becomes one Excel row; the dictionary keys become the header row. For a product table you might collect {"name": ..., "price": ..., "url": ...} per item, append each to a list, then hand the whole list to pd.DataFrame(rows). Setting index=False keeps Excel from adding an extra unnamed column. To split data across multiple sheets in one workbook, open a pd.ExcelWriter and call to_excel once per sheet with different sheet_name values. If a page already exposes a clean HTML <table>, pandas.read_html() can pull every table into DataFrames in a single line.

When you need openpyxl directly

Use openpyxl directly when you want Excel-specific formatting, multiple sheets built incrementally, or to append rows to an existing workbook without reloading everything into memory. pandas is great for a clean one-shot dump; openpyxl gives you per-cell control.

With openpyxl you create a Workbook, grab the active sheet, write a header with ws.append([...]), then loop your scraped records calling ws.append([row["name"], row["price"]]) for each. From there you can bold the header (cell.font = Font(bold=True)), set column widths (ws.column_dimensions["A"].width = 40), freeze the top row (ws.freeze_panes = "A2"), or add a new sheet per category with wb.create_sheet. This streaming append style also lets you handle pagination: keep appending rows as you fetch each page, then call wb.save() once at the end so the entire crawl lands in a single workbook.

No-code route, blocks, and empty tables

If you would rather not write Excel code, write the rows to a CSV file and open it in Excel via Data > From Text/CSV; if your table comes back empty, the page is rendering with JavaScript and needs a real browser fetch first.

One Excel-specific CSV gotcha: write the file as UTF-8 with a BOM so accented characters and currency symbols display correctly. In Python that is open("out.csv", "w", newline="", encoding="utf-8-sig") with csv.DictWriter, or simply df.to_csv("out.csv", index=False, encoding="utf-8-sig"). CSV does flatten structure, so nested or list-valued fields get stringified; prefer .xlsx for nested data. The other common failure is an empty file because the page renders client-side or returns a blocking response. Fetching through a scraping API that renders dynamic content and routes through residential proxies returns the fully rendered HTML or parsed JSON, which you then pass to pandas. For delimiters, escaping, and JSON output, see the sibling guide on exporting scraped data to CSV and JSON.

Code example

python
import requests
import pandas as pd

# 1. Fetch the page through Scrappey. autoparse returns structured JSON;
#    the raw rendered body is always at solution.response.
API_KEY = "YOUR_API_KEY"
resp = requests.post(
    f"https://publisher.scrappey.com/api/v1?key={API_KEY}",
    json={
        "cmd": "request.get",
        "url": "https://example.com/products",
        "proxyCountry": "UnitedStates",
        "session": "excel-export",
        "autoparse": True,
    },
    timeout=180,
)
resp.raise_for_status()
solution = resp.json()["solution"]

# 2. Shape the data into a list of dicts (one dict == one Excel row).
#    Replace this with the fields your target page returns.
rows = []
for item in solution.get("response", {}).get("products", []):
    rows.append({
        "name": item.get("title"),
        "price": item.get("price"),
        "url": item.get("link"),
    })

# 3a. One-line write with pandas (uses openpyxl as the .xlsx engine).
df = pd.DataFrame(rows)
df.to_excel("scraped_products.xlsx", index=False, sheet_name="Products")
print(f"Wrote {len(df)} rows to scraped_products.xlsx")

# 3b. No-code alternative: CSV with a BOM so Excel renders UTF-8 correctly,
#     then open via Data > From Text/CSV.
df.to_csv("scraped_products.csv", index=False, encoding="utf-8-sig")

# pip install requests pandas openpyxl

Next in Extracting and exporting data · 7 of 7

And the cloud equivalent.

Web Scraping to Google Sheets

Related terms

Concept map

Concept map

How How to Scrape Website Data to Excel connects

The terms most directly tied to this one. Hover a node to see its neighbours, click to preview, drag to rearrange.

0 terms · 0 connections
You are here · Web Scraping APIs
Building map…

Frequently asked questions

Why does Excel Get Data from Web return an empty table?

Excel Power Query fetches the raw HTML of a page, not the JavaScript-rendered DOM. If the table is built client-side or the page is gated behind browser verification, Power Query sees an empty or partial document. Fetching through a scraping API that renders JavaScript and returns structured JSON, then writing the rows to .xlsx in Python, solves this.

Do I need a special library to write .xlsx files in Python?

Yes, you need openpyxl. pandas does not write .xlsx on its own; it uses openpyxl as the engine for DataFrame.to_excel. Install both with pip install pandas openpyxl. If you only need plain CSV, no extra library is required because csv is in the standard library.

How do I get paginated results into a single Excel workbook?

Loop over the pages, fetch each one through the API, and keep appending the parsed rows to one list (or append directly to an openpyxl sheet with ws.append). After the loop finishes, build the DataFrame or call wb.save() once. That puts every page into a single workbook instead of one file per page.

Last updated: 2026-06-08