DiscoverBest AI papers explainedSample Complexity and Representation Ability of Test-time Scaling Paradigms
Sample Complexity and Representation Ability of Test-time Scaling Paradigms

Sample Complexity and Representation Ability of Test-time Scaling Paradigms

Update: 2025-09-09
Share

Description

This paper explores theoretical foundations** for **test-time scaling paradigms** in large language models (LLMs). It **analyzes the sample efficiency** of repeated sampling methods like **self-consistency**, finding it requires more samples (Θ(1/∆²)) than **best-of-n** (Θ(1/∆)) for reliable answers. Furthermore, the paper **investigates the expressive power of self-correction**, demonstrating that Transformers with verifier feedback can simulate online learning, enabling a **single Transformer architecture to solve multiple tasks** without prior task knowledge. The authors **empirically validate their theoretical findings**, showing that self-correction significantly enhances accuracy, especially in larger models.

Comments 
In Channel
loading
00:00
00:00
x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

Sample Complexity and Representation Ability of Test-time Scaling Paradigms

Sample Complexity and Representation Ability of Test-time Scaling Paradigms

Enoch H. Kang