Hacker News AI· mikeamm·· 38 min agoAI score58
Benchmarks for document tasks for local AI models
Benchmarks for document tasks for local AI models
AI summary
DocBlocks created "docbench," a 100-point weighted benchmark scoring 24 local models (0.8B–35B parameters) on six document-task families—review, writing, diagrams, slides, study aids and narration cleanup—using 102 test cases with automatic checks and AI judging. Qwen 3.8 27B Q4 led local models at 81/100, with Gemma 4 12B statistically level at 79 and the best 4B-and-under models reaching about 71, while cloud models scored 87–91. Review separated models most: large models found real problems, whereas 1–2B models padded reviews with trivial or wrong suggestions.
Source: Hacker News AI · bendyline.com