What Is Anubis? Open-Source Anti-AI-Scraper Firewall

Paste into ChatGPT, Claude, or any LLM

On this page

Anubis is an open-source MIT-licensed reverse proxy that issues a SHA-256 proof-of-work challenge before serving HTTP requests, built specifically to slow down AI scrapers that ignore robots.txt. Released on January 19, 2025 by Xe Iaso (now maintained by Techaro), it has been adopted by GNOME GitLab, the Linux kernel mailing list archives, FFmpeg, Wine, UNESCO, FreeCAD, Duke University digital archives, and most non-Cloudflare FOSS projects. Recognisable by its anime "weighing the soul" mascot illustration during the challenge.

Released	19 January 2025 by Xe Iaso, now Techaro
License	MIT
GitHub stars	19.6k+ (May 2026)
Algorithm	Hashcash-style SHA-256 PoW (default: 5 leading zeros)
Notable deployments	GNOME GitLab, Linux kernel archives, FFmpeg, Wine, UNESCO, FreeCAD, sourcehut

How the challenge works

When a client requests a protected page, Anubis returns a challenge: a random number plus a difficulty parameter. The client must find a nonce such that SHA-256(challenge || nonce) has a pre-specified number of leading zeros — five by default. This is the same Hashcash idea Bitcoin mining uses, miniaturised to take a real browser about a second to solve on a laptop.

Once solved, the response is stored as a cookie (techaro.lol-anubis-auth) for roughly a week, after which the client must solve a new challenge. The economic logic: a real human visiting once a week pays a one-second tax; an AI scraper visiting 10,000 pages a day pays an unbearable CPU cost in aggregate.

Why FOSS projects deployed it

Anubis was created in response to Amazon's AI crawler overloading Xe's Git server while ignoring robots.txt. Within months, projects that had been bleeding bandwidth to ChatGPT, Claude, and Perplexity-style crawlers deployed it. The Linux kernel mailing list archive, sourcehut, FFmpeg, Wine, GNOME's GitLab, FreeCAD, and Duke's digital archives all run it. UNESCO digital repositories run it. The shared problem: small infra budgets vs. industrial-scale crawling that does not respect any opt-out signal.

Has it been bypassed?

Yes. Codeberg reported in August 2025 that "many AI scraper bots had learned how to solve the Anubis challenges." Codeberg still considered the protection useful — it had blocked the bulk of scraping for several months — but acknowledged adaptation.

Security researcher Tavis Ormandy demonstrated that native-code solvers (Go, Rust, C) can solve Anubis challenges far more efficiently than the JavaScript implementation ordinary users run, so a determined operator with a dedicated solver pays a fraction of the cost real users pay.

The practical takeaway: Anubis slows scrapers, raises their operational cost, and stops the cheapest ones outright. It does not stop a motivated operator who is willing to ship a native solver, and headless Chromium with JS enabled solves it naturally (just more slowly than a CPU-optimised binary).

Code example

python

# Anubis with headless Chromium solves naturally — JS runs, PoW completes.
# Persist the techaro.lol-anubis-auth cookie across requests to avoid re-solving.

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    ctx = browser.new_context()
    page = ctx.new_page()

    # Visit any protected URL once — Anubis serves challenge, JS solves it
    page.goto("https://lkml.iu.edu/")
    page.wait_for_load_state("networkidle")

    # Save the auth cookie and reuse for ~1 week
    cookies = ctx.cookies()
    anubis_cookie = next(c for c in cookies if "anubis-auth" in c["name"])
    print("Solved. Reuse this cookie for ~7 days:", anubis_cookie["value"][:40], "...")

Related terms

What Is Anti-Bot Detection?

Anti-bot detection is the set of techniques websites use to distinguish automated traffic from human users — and to block, challenge, or thr…

What Is a DOM Honeypot?

A DOM honeypot is an invisible form field or link that humans never see but bots fill in or click. The moment you interact with it, the site…

What Is Cloudflare Turnstile?

Cloudflare Turnstile is a CAPTCHA-replacement service that verifies a visitor is a human without showing a traditional puzzle. It runs a ser…

What Is Web Scraping?

Web scraping is the automated extraction of structured data from websites. A scraper sends HTTP requests to a target URL, parses the HTML or…

What Is Battery Status API Fingerprinting?

Battery Status API fingerprinting used the precise charge level and charging/discharging times exposed by navigator.getBattery() as a short-…

What Is Favicon Fingerprinting (Supercookies)?

Favicon fingerprinting (the “Supercookie” technique) abuses the browser’s separate, long-lived favicon cache to store a persistent identifie…

What Is Math & JS Engine Fingerprinting?

Math fingerprinting identifies a runtime by computing transcendental functions (sin, cos, tan, exp, log, pow) at fixed inputs and reading th…

Concept map

How Anubis (Anti-AI-Scraper Firewall) connects

The terms most directly tied to this one. Hover a node to see its neighbours, click to preview, drag to rearrange.

0 terms · 0 connections

You are here · Anti-Bot

Tools & solutions for this topic

Frequently asked questions

Is Anubis the same as Cloudflare Turnstile?

Both issue proof-of-work challenges, but Anubis is self-hosted open-source software (MIT) and Turnstile is a Cloudflare-managed service. Anubis is what small FOSS projects without enterprise-grade infra deploy; Turnstile is what sites already on Cloudflare turn on. The PoW idea is the same; the operational model is different.

How long is the Anubis cookie valid?

About one week by default. Once the client has solved a challenge, the techaro.lol-anubis-auth cookie is persisted, and subsequent requests within the validity window are served without re-solving. This makes the cost per real user negligible while still penalising high-volume scrapers.

Does Anubis block search engines?

By default it can — Google, Bing, and DuckDuckGo crawlers running standard HTTP without JS won't solve the challenge. Anubis ships a configurable allowlist for "known good" bots based on User-Agent and reverse-DNS verification. Site operators configure which crawlers to let through.

Will Anubis still be effective in 2027?

The proof-of-work tax remains real regardless of scraper sophistication — even with a native solver, scraping 10,000 protected pages costs measurable CPU time. The frontier is challenge difficulty: Anubis can raise the leading-zero count for suspected scrapers and lower it for likely humans. The arms race continues; the tool keeps the cost asymmetry.

Last updated: 2026-05-26