“Introducing the Epoch Capabilities Index (ECI)” by luke_emberson, YafahEdelman, Jsevillamol
Update: 2025-10-29
Description
We at Epoch AI have recently released a new composite AI capability index called the Epoch Capabilities Index (ECI), based on nearly 40 underlying benchmarks.
Some key features...
- Saturation-proof: ECI "stitches" benchmarks together, to enable comparisons even as individual benchmarks become saturated.
- Global comparisons: Models can be compared, even if they were never evaluated on the same benchmarks.
- Difficulty-based task weighting: ECI uses a simple statistical model (similar to those used in Item Response Theory) under which models deemed more capable if they score well on difficult benchmarks, and benchmarks are deemed more difficult if capable models are unable to score highly on them.
ECI will allow us to track trends in capabilities over longer spans of time, potentially revealing changes in the pace of progress. It will also improve other analyses that would otherwise depend on a single benchmark for comparison.
You can find more details about [...]
---
First published:
October 28th, 2025
Source:
https://www.lesswrong.com/posts/2RtuThoZwP4o8aEpS/introducing-the-epoch-capabilities-index-eci
---
Narrated by TYPE III AUDIO.
Comments
In Channel
Download from Google Play
Download from App Store
United States00:00
00:00
x
