Meta, MIT and UW researchers unveil 'Context Language Models' that edit their own context
Researchers from Meta, MIT and the University of Washington describe Context Language Models, which manage and edit their own context instead of relying on fixed summarization or retrieval, and report higher accuracy with less compute.
Photo: Peacearth / Wikimedia Commons, CC BY-SA 4.0
Researchers from Meta, MIT and the University of Washington have introduced Context Language Models (CLMs), an approach that lets a language model manage and edit its own context rather than relying on predefined summarization, compression or retrieval, InfoQ reported on October 11.
According to the researchers, zero-shot CLMs achieved 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus and 5% higher scores with 59% fewer FLOPs on a 12-hour EdgeBench task.
Reinforcement learning raised Qwen3.5-9B's score on BrowseComp-Plus from 28.8% to 42.5% while using 12% fewer FLOPs, InfoQ reported.