What a 406 Not Acceptable means
Accept, Accept-Language, or Accept-Encoding) tell the server what you can handle — for example, JSON instead of HTML, or English instead of French. A 406 means the server has no version that matches those demands, so content negotiation failed. In short, a 406 status code is the server saying it can't return content matching your Accept headers.Why scrapers see 406
Accept header, an empty one, or a combination no real browser would. Some anti-bot setups deliberately return 406 to requests whose Accept-* headers look automated, because real browsers send a very specific, consistent set.How to fix a 406 error
Accept, Accept-Language, and Accept-Encoding headers — copy them verbatim from a real browser and keep them consistent with your User-Agent (the string that identifies your client). Even just stripping compression headers to make parsing easier can trigger 406. When 406 is an anti-bot signal, a full real-browser request via Web Access API resolves it.