Hacker News AI· speckx·· 10 hr agoAI score22
Ready or Not, Here Comes AI, by Chris Shiflett
Ready or Not, Here Comes AI, by Chris Shiflett
AI summary
Chris Shiflett argues that AI crawlers like GPTBot and ClaudeBot resemble the text-only Googlebot of 2004, fetching HTML and largely not executing JavaScript, and that this makes clean, well-structured HTML newly important for being found. He describes building tools to check whether AI can find his sites and whether it actually recommends them, concluding that findability depends on basics like robots.txt, sitemaps, real HTML content and structured data, while authority comes from what others say about you.
Source: Hacker News AI · shiflett.org