Web Scraping APIs

What Is curl_cffi?

What Is curl_cffi? — conceptual illustration
On this page

curl_cffi is a Python HTTP client whose TLS fingerprint looks exactly like real Chrome, Firefox, or Safari. TLS is the encryption layer behind https, and the way a client negotiates it leaves a recognisable signature. curl_cffi wraps curl-impersonate — a modified build of curl that uses BoringSSL with the exact cipher list, extensions, and HTTP/2 SETTINGS frames a real browser sends. It works as a drop-in replacement for the requests library: the only code change is adding impersonate="chrome131" to your call.

Quick facts

LanguagePython (CFFI bindings to curl-impersonate)
Drop-in forrequests, httpx — same API surface
SpoofsTLS JA4, HTTP/2 SETTINGS, header order, GREASE values
Matches browser handshake againstCloudflare TLS checks, DataDome XHR, Akamai, common WAFs
Cannot handleCloudflare Turnstile, Akamai sensor.js scoring, JS challenges

Why curl_cffi exists

Here is the problem it solves. Python's requests library is built on a stack of layers: urllib3 → Python's ssl module → OpenSSL, using Python's default cipher list. That list has barely changed in years, so it produces one predictable TLS signature. Anti-bot services catalogue that signature as a JA3/JA4 hash (a fingerprint computed from the TLS handshake) and recognise it instantly. You can fake every header, cookie, and URL, but the handshake happens first and gives you away.

curl-impersonate fixes this by patching curl to use BoringSSL — the TLS library inside Chrome — with Chrome's exact cipher list, extension order, GREASE values, and HTTP/2 SETTINGS. The resulting JA4 is indistinguishable from real Chrome. curl_cffi is the Python binding to that patched curl, wrapped in a requests-compatible API.

What changes when you flip the switch

Adding a single impersonate argument changes everything the handshake reveals:

  • The TLS cipher suite list and its order
  • The TLS extensions (ALPN, SNI, supported_groups, signature_algorithms, and so on) and their order
  • GREASE values — Chrome's deliberately randomised dummy extensions
  • The HTTP/2 SETTINGS frame contents (HEADER_TABLE_SIZE, INITIAL_WINDOW_SIZE, MAX_CONCURRENT_STREAMS)
  • Header order and casing

Modern anti-bots fingerprint every one of these. Forgetting just one — the classic mistake is spoofing the User-Agent string but leaving the TLS untouched — flags you faster than not spoofing at all, because the mismatch (a browser User-Agent paired with a Python handshake) is itself a giveaway.

What curl_cffi is and is not for

Use it when: the target is guarded by a TLS-fingerprinting WAF — a web application firewall that filters traffic before it reaches the site (Cloudflare without Turnstile, DataDome XHR endpoints, medium-strength Akamai) — or you are simply tired of requests getting flagged everywhere. Roughly 60–80% of protected targets become reachable with curl_cffi plus a residential proxy.

Do not use it when: the page needs JavaScript to run (Cloudflare Turnstile, Akamai sensor.js, F5 Shape custom VM). curl_cffi only sends HTTP requests; it cannot execute JS. For those you need a real browser — Camoufox, CloakBrowser, or a managed API.

Code example

python
from curl_cffi import requests

# Drop-in replacement for the requests library
r = requests.get(
    "https://protected-site.com/api/data",
    impersonate="chrome131",
    headers={"Accept-Language": "en-US,en;q=0.9"},
    proxies={"https": "http://user:pass@residential-isp:port"},
    timeout=20,
)

# Sessions preserve cookies and TLS state across requests
s = requests.Session(impersonate="chrome131")
s.get("https://protected-site.com/")              # warm up
data = s.get("https://protected-site.com/api").json()

Related terms

What Is TLS Fingerprinting (JA3/JA4)?
TLS fingerprinting is a way to recognize what software made a connection just by looking at how it sets up encryption — before the server re…
What Is Web Scraping?
Web scraping is the automated extraction of structured data from websites. Instead of a person copying and pasting, a program (a "scraper") …
What Is a Residential Proxy?
A residential proxy sends your web traffic through a real home internet connection — a regular broadband or fiber line — instead of through …
What Is Anti-Bot Detection?
Anti-bot detection is the set of techniques websites use to tell automated traffic apart from real human visitors — and then block, challeng…
What Is the Scrapy + Go TLS Sidecar Architecture?
The Scrapy + Go TLS sidecar architecture is the most common production pattern for scraping Akamai- and Cloudflare-protected sites at scale.…
Web Scraping Tools 2026 — A Comparison
"Web scraping tools" is the whole family of software you use to pull data off websites — and in 2026 that family is big but neatly sorted in…
What Is HTTP/2 Fingerprinting?
HTTP/2 fingerprinting identifies an HTTP client from its SETTINGS frame and frame-level behaviour, independent of the TLS layer. Think of it…
What Is JA4 Fingerprinting?
JA4 is a way to identify a browser by the fingerprint of its TLS handshake — TLS being the encryption layer behind https. It replaced the ol…
Web Scraping With curl: A Complete 2026 Guide
Web scraping with curl means fetching pages directly from the command line, setting headers, cookies, and proxies with curl's flags, then pi…

Concept map

How curl_cffi connects

The terms most directly tied to this one. Hover a node to see its neighbours, click to preview, drag to rearrange.

0 terms · 0 connections
You are here · Web Scraping APIs
Building map…

Frequently asked questions

Is curl_cffi the same as requests?

It has a compatible API but is completely different underneath. requests uses OpenSSL through Python's ssl module; curl_cffi uses BoringSSL through patched curl, so its TLS fingerprint matches a real browser. The function signatures are nearly identical (.get, .post, .Session), so most scrapers can switch with a single import change.

Does curl_cffi handle HTTP/2 correctly?

Yes. It sends HTTP/2 SETTINGS frames matching the browser it is impersonating. This matters because the contents of those frames are a second fingerprint, and they catch scrapers who fixed only their TLS.

How often do I need to update the impersonation profile?

Every couple of months. Real users keep upgrading, so a Chrome 120 fingerprint in 2026 looks suspicious on its own. Bump to the latest profile curl_cffi ships and re-check your success rate.

Does curl_cffi solve JavaScript challenges?

No. It is purely an HTTP client and does not run JavaScript. For JS challenges (Cloudflare Turnstile, Akamai sensor.js, F5 Shape) you need a real browser like Camoufox or a managed scraping API.

Last updated: 2026-05-31