Twenty Percent of all 'ok' web responses AI gets are not okay

A new paper estimates that 20.9% of pages returning HTTP status 200 to a static client contain none of the page's main content, instead delivering app shells, bot challenges, block pages, error or placeholder pages, and partial content. Testing 10,307 web addresses over three days, the study found apparent citation drift rose from 5.8% to 21.9% when all 200 responses were accepted as successful retrievals, and recommends that fetching tools report what was actually retrieved rather than relying on status codes.