The Decoder· Jonathan Kemper·· 8 hr agoAI score47
Nvidia researchers argue huge AI context windows are a dead end
Nvidia researchers argue huge AI context windows are a dead end
AI summary
An Nvidia research team argues that very large context windows are often overkill and expensive, since in many tests periodic summaries and targeted lookups performed about as well. The piece notes that context windows have grown from GPT-4's 8,192 tokens in March 2023 to roughly one million tokens in GPT 5.6 Sol, enough for Tolstoy's "War and Peace."
Source: The Decoder · the-decoder.com