DiscoverDeep PapersExplaining Grokking Through Circuit Efficiency
Explaining Grokking Through Circuit Efficiency

Explaining Grokking Through Circuit Efficiency

Update: 2023-10-17
Share

Description

Join Arize Co-Founder & CEO Jason Lopatecki, and ML Solutions Engineer, Sally-Ann DeLucia, as they discuss “Explaining Grokking Through Circuit Efficiency." This paper explores novel predictions about grokking, providing significant evidence in favor of its explanation. Most strikingly, the research conducted in this paper demonstrates two novel and surprising behaviors: ungrokking, in which a network regresses from perfect to low test accuracy, and semi-grokking, in which a network shows delayed generalization to partial rather than perfect test accuracy.

Find the transcript and more here: https://arize.com/blog/explaining-grokking-through-circuit-efficiency-paper-reading/

To learn more about ML observability, join the Arize AI Slack community or get the latest on our LinkedIn and Twitter.

Comments 
In Channel
Anthropic Claude 3

Anthropic Claude 3

2024-03-2543:01

RAG vs Fine-Tuning

RAG vs Fine-Tuning

2024-02-0839:49

Phi-2 Model

Phi-2 Model

2024-02-0244:29

loading
00:00
00:00
x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

Explaining Grokking Through Circuit Efficiency

Explaining Grokking Through Circuit Efficiency

Arize AI