Anti-Bot

What Is Anubis? Open-Source Anti-AI-Scraper Firewall

What Is Anubis? Open-Source Anti-AI-Scraper Firewall — conceptual illustration
On this page

Anubis is a free, open-source MIT-licensed "gatekeeper" that sits in front of a website (a reverse proxy - software that intercepts requests before they reach the real server) and forces each visitor's browser to solve a small math puzzle before any page is served. The puzzle is a SHA-256 proof-of-work challenge - a calculation that is hard to compute but easy to verify - designed to slow down AI scrapers that ignore robots.txt (the file where sites politely ask bots not to crawl them). Released on January 19, 2025 by Xe Iaso (now maintained by Techaro), it has been adopted by GNOME GitLab, the Linux kernel mailing list archives, FFmpeg, Wine, UNESCO, FreeCAD, Duke University digital archives, and most non-Cloudflare FOSS projects. You can recognise it by its anime "weighing the soul" mascot illustration shown while the challenge runs.

Quick facts

Released19 January 2025 by Xe Iaso, now Techaro
LicenseMIT
GitHub stars19.6k+ (May 2026)
AlgorithmHashcash-style SHA-256 PoW (default: 5 leading zeros)
Notable deploymentsGNOME GitLab, Linux kernel archives, FFmpeg, Wine, UNESCO, FreeCAD, sourcehut

How the challenge works

When a browser asks for a protected page, Anubis does not answer right away. Instead it hands back a puzzle: a random number plus a difficulty setting. The browser then has to keep trying different values (a nonce - a throwaway number) until it finds one where SHA-256(challenge || nonce) produces a hash that starts with a set number of zeros - five by default. There is no shortcut; you just try numbers until one works. This is the same Hashcash trick Bitcoin mining uses, shrunk down so a real browser solves it in about a second on a laptop.

Once the browser solves it, the answer is saved as a cookie (techaro.lol-anubis-auth) that lasts roughly a week, after which it must solve a fresh puzzle. The economics are the point: a real person who visits once a week pays a one-second tax and never notices. An AI scraper hitting 10,000 pages a day has to solve thousands of puzzles, and that CPU cost piles up until scraping becomes too expensive to be worth it.

Why FOSS projects deployed it

Anubis was built after Amazon's AI crawler hammered Xe's Git server while ignoring robots.txt. Within months, projects that had been losing bandwidth (and money) to ChatGPT, Claude, and Perplexity-style crawlers turned it on. The Linux kernel mailing list archive, sourcehut, FFmpeg, Wine, GNOME's GitLab, FreeCAD, and Duke's digital archives all run it. UNESCO digital repositories run it. They share the same problem: small hosting budgets up against industrial-scale crawling that ignores every opt-out signal.

How effective is it in practice?

Its effectiveness has limits. Codeberg reported in August 2025 that "many AI scraper bots had learned how to solve the Anubis challenges." Codeberg still found it useful - it had blocked most scraping for several months - but noted the bots had adapted.

Security researcher Tavis Ormandy documented that a proof-of-work solver written in fast native code (Go, Rust, C) computes Anubis challenges far quicker than the JavaScript version that runs in a normal visitor's browser, so the per-challenge cost is lower for native solvers than for ordinary browsers.

The practical takeaway: Anubis slows high-volume crawling down, raises the operating cost, and stops the cheapest scrapers entirely. It does not stop an operator willing to build a native solver, and a headless Chromium (a real browser engine running without a visible window, with JavaScript on) completes the challenge the same way any browser does - just slower than a purpose-built binary.

Code example

python
# Anubis with headless Chromium solves naturally — JS runs, PoW completes.
# Persist the techaro.lol-anubis-auth cookie across requests to avoid re-solving.

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    ctx = browser.new_context()
    page = ctx.new_page()

    # Visit any protected URL once — Anubis serves challenge, JS solves it
    page.goto("https://lkml.iu.edu/")
    page.wait_for_load_state("networkidle")

    # Save the auth cookie and reuse for ~1 week
    cookies = ctx.cookies()
    anubis_cookie = next(c for c in cookies if "anubis-auth" in c["name"])
    print("Solved. Reuse this cookie for ~7 days:", anubis_cookie["value"][:40], "...")

Related terms

Concept map

How Anubis (Anti-AI-Scraper Firewall) connects

The terms most directly tied to this one. Hover a node to see its neighbours, click to preview, drag to rearrange.

0 terms · 0 connections
You are here · Anti-Bot
Building map…

Frequently asked questions

Is Anubis the same as Cloudflare Turnstile?

Both hand out proof-of-work puzzles, but they are run very differently. Anubis is software you host yourself, open-source under the MIT license. Turnstile is a service Cloudflare runs for you. Anubis is what small FOSS projects without enterprise infrastructure deploy on their own servers; Turnstile is a switch you flip if your site already sits behind Cloudflare. Same puzzle idea, different operating model.

How long is the Anubis cookie valid?

About one week by default. After a browser solves a challenge once, Anubis saves the techaro.lol-anubis-auth cookie, and any requests within that week go straight through without solving again. That keeps the cost for a real visitor near zero while still punishing high-volume scrapers, which solve puzzles constantly.

Does Anubis block search engines?

By default it can. Crawlers from Google, Bing, and DuckDuckGo make plain HTTP requests without running JavaScript, so they never solve the puzzle and get blocked. To avoid that, Anubis includes a configurable allowlist of "known good" bots, identified by their User-Agent string and confirmed with reverse-DNS lookup (checking that the IP really belongs to the claimed crawler). The site operator decides which crawlers to wave through.

Will Anubis still be effective in 2027?

The proof-of-work tax stays real no matter how advanced scrapers get - even with a fast native solver, crawling 10,000 protected pages still burns measurable CPU time. The next frontier is tuning difficulty: Anubis can crank up the required number of leading zeros for visitors it suspects are bots and ease off for likely humans. The arms race continues, but the tool keeps the cost lopsided in the site's favor.

Last updated: 2026-05-31