Measuring AI Ability by Task Length

METR: Measuring AI Ability by Task-Length 

This section is about how researchers at METR (Model Evaluation and Task Reliability) propose to measure AI progress. Instead of focusing only on raw compute power or algorithms, they suggest looking at “task-length” , the duration and complexity of tasks that AI systems can reliably complete.

Raw compute, how much hardware and energy we devote to training, matters just as much as clever algorithms. By tracking how long and complex the tasks are that AI can sustain, we get a sharper picture of progress than by asking “How smart is this AI?” It shows us where models are improving, and what orders of magnitude in compute and design drive breakthroughs.