Skip to content
Web Scraping APIs

Why Do HTTP-Only Challenge Reimplementations Keep Breaking?

Pim

Pim · Scrappey Research

October 28, 2026 4 min read

Why Do HTTP-Only Challenge Reimplementations Keep Breaking? — conceptual illustration
On this page

An HTTP-only reimplementation answers a challenge widget's questions without running a browser: it deobfuscates the script, decodes its virtual machine, computes or replays each answer and encodes the payload itself. It saves the memory and CPU of a browser process, but it has to understand the challenge code, and that code changes on purpose and often. A real browser only has to run the new code. One public open-source project needed 99 changed files and 14,578 changed lines 49 hours after its first release, when the widget shipped a new build.

Quick facts

Public caseOpen-source Rust project, 43,314 lines, released 25 Oct 2026
First new build49 hours later: 99 files, +8,976 / -5,602 lines
Must track per buildObfuscator, VM, field names, payload encoding, probes, transport
New probes in that buildWebSocket round-trip echo, timed fetches from a worker
Browser approachRuns whatever ships; maintenance moves to identity consistency

Two ways to answer a challenge

A challenge widget is a program that runs in the browser, asks the browser questions and sends the answers to a server. An automated client can produce those answers in two ways. It can run the program in a real browser, such as a headless browser, which answers every question with its own engine. Or it can understand the program well enough to answer without running it: undo the obfuscation (see how deobfuscation works), decode the bytecode, then compute or replay each answer.

The second approach looks cheaper because there is no browser process. Its real cost is that it depends on the program's internals, and those are the part the challenge vendor changes most.

A public case: 49 hours

An open-source HTTP-only implementation for a widely deployed challenge widget was published on 25 October 2026: 43,314 lines of Rust, a deobfuscator with more than twenty passes, a symbolic interpreter for the widget's bytecode and one recorded iPhone whose answers it replays. Its documentation already marked "a new challenge build" as a non-retryable error.

The first new build arrived about 49 hours later. The fix changed 99 files, adding 8,976 lines and removing 5,602:

  • Deobfuscation: six new passes and four removed, because the obfuscator switched its string-table scheme.
  • Virtual machine: dispatcher rewritten, register handling added.
  • New probes: a WebSocket echo that measures round-trip time (see latency-based proxy detection) and a worker that times fetches to the vendor's hosts.
  • Recorded answers: the device profile re-recorded, about 2,500 lines, including a new header order for the WebSocket handshake.

Other public evidence points the same way: one browser project's bug tracker documented at least 31 distinct builds of another vendor's challenge script in October 2026, with a new probe appearing between two builds 23 minutes apart.

What each approach has to maintain

ItemHTTP-only reimplementationReal browser
Obfuscator transformsA pass for each; new ones break the pipelineNothing; the engine runs the code
Per-session virtual machineRecover handlers and opcodes every sessionNothing
Randomised field names, payload encodingMap and re-encode per buildNothing
New probes (latency, hash rate)A recorded or modelled answer for eachAnswered by the machine itself
Transport (TLS, HTTP/2, header order)Impersonate per request kindNative
Identity consistencyOne replayed device in every sessionKeep hardware, locale and network coherent

Replay also has a ceiling no update fixes: one recorded device answers for every session from every IP, and each probe that measures something physical, such as latency or a proof-of-work hash rate, needs a fresh model of what that device would have measured.

Browsers cost memory and CPU and need fleet management, so the trade is real: pay for browser processes, or pay a code change every time the challenge ships a build. A managed web data API takes the browser side of that trade and runs the fleet, so the integration does not change when a challenge does.

Code example

python
# The browser approach: no per-build logic. When the challenge script
# changes, this code does not.
from playwright.sync_api import sync_playwright

URL = "https://example.com/"

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto(URL, wait_until="networkidle")  # the page runs its own scripts
    html = page.content()
    browser.close()

print(len(html), "bytes of rendered HTML")

# An HTTP-only client would instead need, per challenge build:
#   deobfuscate(script) -> decode_vm(bytecode) -> answer(probes)
#   -> encode(payload) -> post(headers_for_each_request_kind)
# and each of those steps can change with the next build.

Related terms

Concept map

Concept map

How Why Do HTTP-Only Challenge Reimplementations Break connects

The terms most directly tied to this one. Hover a node to see its neighbours, click to preview, drag to rearrange.

0 terms · 0 connections
You are here · Web Scraping APIs
Building map…

Frequently asked questions

Why not answer a challenge without a browser to save resources?

It does save memory and CPU per request, but the code that produces the answers has to follow every change the challenge ships. In one public case the first new build required changing 14,578 lines within two days, and every change needs fresh recordings from a real browser anyway.

How often do challenge scripts change?

Often and on purpose. Public trackers documented at least 31 distinct builds of one vendor's script in October 2026, and a new probe appeared between two builds 23 minutes apart.

Does a real browser need any maintenance?

Yes, but of a different kind. It does not need updates when the challenge changes; it needs its own identity to stay consistent, meaning hardware claims, locale, network path and browser version that agree with each other.

Last updated: 2026-10-28 · Facts last verified: 2026-10-28