iCreatAI News
Skip to content
Hacker News AI· mikeamm·· 38 min agoAI score58

Benchmarks for document tasks for local AI models

Benchmarks for document tasks for local AI models

AI summary

DocBlocks created "docbench," a 100-point weighted benchmark scoring 24 local models (0.8B–35B parameters) on six document-task families—review, writing, diagrams, slides, study aids and narration cleanup—using 102 test cases with automatic checks and AI judging. Qwen 3.8 27B Q4 led local models at 81/100, with Gemma 4 12B statistically level at 79 and the best 4B-and-under models reaching about 71, while cloud models scored 87–91. Review separated models most: large models found real problems, whereas 1–2B models padded reviews with trivial or wrong suggestions.

Source: Hacker News AI · bendyline.com