Open benchmark of AI decision APIs vs. XGBoost for fraud triage

Smart Mobile House released fraud-decision-lab, a vendor-neutral benchmark that sends the same transactions to TypeSafe Jev, OpenAI Decisions, Cloudflare Clef/Clef-flash and a structured-output LLM, then measures how much labelled fraud each ranks into the top 1% and 5% of an analyst review queue against an XGBoost baseline, with cost and latency. It runs offline by default on bundled synthetic data, saves predictions and experiment metadata for local verification, and requires API keys, --allow-external-requests and a --max-cost-usd estimate for live provider runs. The project notes that XGBoost learns from labelled history while the AI models score transactions zero-shot, that the bundled data is synthetic, and that the current source distribution contains no benchmark result files.