Hacker News AI· ArcHound·· 7 hr agoAI score22
Overwhelmed by AI Bots
Overwhelmed by AI Bots
AI summary
A self-hosted blog admin analyzing AI bot traffic found roughly 8% potential AI bot traffic (including impostors) at about 20 requests per hour, and gave up on building a simple admin guideline because the bots are too numerous and change too frequently. The notes cover three AI bot categories (crawler, assistant, search), Cloudflare Radar as the best but incomplete and fickle intel source, user-agent substring matching per RFC9309, an emerging JSON format for bot IP ranges, HTTP Message Signatures RFC9421, Petalbot's reverse-DNS approach, and JA4 as a weak signal. The author plans to try Fingerprint.js or Anubis later while relying on rate-limiting in the meantime.
Source: Hacker News AI · blog.miloslavhomer.cz