Web Scraping APIs

What Is curl_cffi?

What Is curl_cffi? — conceptual illustration
On this page

curl_cffi is a Python HTTP client that produces TLS fingerprints identical to real Chrome, Firefox, or Safari. It wraps curl-impersonate — a patched build of curl that uses BoringSSL with the exact cipher list, extensions, and HTTP/2 SETTINGS frames a real browser sends. It is a drop-in replacement for the requests library: the only code change is adding impersonate="chrome131" to your call.

Quick facts

LanguagePython (CFFI bindings to curl-impersonate)
Drop-in forrequests, httpx — same API surface
SpoofsTLS JA4, HTTP/2 SETTINGS, header order, GREASE values
BypassesMost Cloudflare, DataDome XHR, medium Akamai, all small WAFs
Does not bypassCloudflare Turnstile, Akamai sensor.js scoring, JS challenges

Why curl_cffi exists

Python's requests library is built on urllib3 → Python's ssl module → OpenSSL with Python's default cipher list. That list has been stable for years. Every commercial anti-bot has its JA3/JA4 hash catalogued. You can spoof every header, every cookie, every URL — the TLS handshake gives you away first.

curl-impersonate solves this by patching curl to use BoringSSL (Chrome's TLS library) with Chrome's exact cipher list, extension order, GREASE values, and HTTP/2 SETTINGS. The result is a JA4 indistinguishable from real Chrome. curl_cffi is the Python binding to that patched curl, exposed through a requests-compatible API.

What changes when you flip the switch

A single impersonate argument changes:

  • TLS cipher suite list and order
  • TLS extensions (ALPN, SNI, supported_groups, signature_algorithms, etc.) and their order
  • GREASE values (Chrome's randomised dummy extensions)
  • HTTP/2 SETTINGS frame contents (HEADER_TABLE_SIZE, INITIAL_WINDOW_SIZE, MAX_CONCURRENT_STREAMS)
  • Header order and casing

Every one of those is fingerprinted by modern anti-bots. Forgetting any one (the most common mistake: spoofing User-Agent without TLS) flags you faster than no spoofing at all because the mismatch itself is a signal.

What curl_cffi is and is not for

Use it when: the target is protected by a TLS-fingerprinting WAF (Cloudflare without Turnstile, DataDome XHR endpoints, medium-strength Akamai), or you are tired of requests getting flagged by every commercial anti-bot. About 60–80% of protected targets are clear with curl_cffi + a residential proxy.

Do not use it when: the page requires JS execution (Cloudflare Turnstile, Akamai sensor.js, F5 Shape custom VM). For those you need a real browser — Camoufox, CloakBrowser, or a managed API.

Code example

python
from curl_cffi import requests

# Drop-in replacement for the requests library
r = requests.get(
    "https://protected-site.com/api/data",
    impersonate="chrome131",
    headers={"Accept-Language": "en-US,en;q=0.9"},
    proxies={"https": "http://user:pass@residential-isp:port"},
    timeout=20,
)

# Sessions preserve cookies and TLS state across requests
s = requests.Session(impersonate="chrome131")
s.get("https://protected-site.com/")              # warm up
data = s.get("https://protected-site.com/api").json()

Related terms

What Is TLS Fingerprinting (JA3/JA4)?
TLS fingerprinting is a technique that identifies an HTTP client from its TLS handshake — before the server reads a single request byte. The…
What Is Web Scraping?
Web scraping is the automated extraction of structured data from websites. A scraper sends HTTP requests to a target URL, parses the HTML or…
What Is a Residential Proxy?
A residential proxy routes your HTTP traffic through a real residential internet connection — a home broadband or fiber line — instead of th…
What Is Anti-Bot Detection?
Anti-bot detection is the set of techniques websites use to distinguish automated traffic from human users — and to block, challenge, or thr…
What Is the Scrapy + Go TLS Sidecar Architecture?
The Scrapy + Go TLS sidecar architecture is the most common production pattern for scraping Akamai- and Cloudflare-protected sites at scale.…
Web Scraping Tools 2026 — A Comparison
The web-scraping toolbox in 2026 is large but well-stratified. Each tool occupies one of seven roles — HTTP/TLS impersonation, browser autom…
What Is HTTP/2 Fingerprinting?
HTTP/2 fingerprinting identifies an HTTP client from its SETTINGS frame and frame-level behaviour, independent of the TLS layer. Every HTTP/…
What Is JA4 Fingerprinting?
JA4 is a TLS client fingerprint that replaced JA3 after Chrome began randomising the order of its TLS extensions. JA3 hashed the extension l…

Concept map

How curl_cffi connects

The terms most directly tied to this one. Hover a node to see its neighbours, click to preview, drag to rearrange.

0 terms · 0 connections
You are here · Web Scraping APIs
Building map…

Tools & solutions for this topic

Frequently asked questions

Is curl_cffi the same as requests?

API-compatible, but completely different under the hood. requests uses OpenSSL via Python’s ssl module. curl_cffi uses BoringSSL via patched curl. The function signatures are nearly identical (.get, .post, .Session) so most scrapers can switch with a single import change.

Does curl_cffi handle HTTP/2 correctly?

Yes — it ships HTTP/2 SETTINGS frames matching the impersonated browser. This matters because HTTP/2 frame fingerprinting is a secondary signal that catches scrapers who only fixed TLS.

How often do I need to update the impersonation profile?

Every couple of months. A Chrome 120 fingerprint in 2026 is itself suspicious because real users have updated. Bump to the latest profile curl_cffi ships and re-check your success rate.

Does curl_cffi solve JavaScript challenges?

No. It is purely an HTTP client — it does not execute JavaScript. For JS challenges (Cloudflare Turnstile, Akamai sensor.js, F5 Shape) you need a real browser like Camoufox or a managed scraping API.

Last updated: 2026-05-26