Hacker News AI· jethkl·· 3 days agoAI score42
Show HN: A tool to optimize AI model cost without benchmark score trade-offs
Show HN: A tool to optimize AI model cost without benchmark score trade-offs
AI summary
This interactive page applies data envelopment analysis (following Zhou et al., 2022) to compare AI models' API cost per 1,000 benchmark tasks against their benchmark scores, identifying cheaper models or blends that match a given score. It notes that savings are benchmark estimates excluding latency, quotas, switching and other deployment costs, and that users should test alternatives on their own tasks. The single-slide creation task uses the free disposal hull variant and compares single models only, since its preference ratings cannot be averaged.
Source: Hacker News AI · jeff-kline.github.io