DiscoverLarge Language Model (LLM) TalkMixture-of-Recursions (MoR)
Mixture-of-Recursions (MoR)

Mixture-of-Recursions (MoR)

Update: 2025-07-18
Share

Description

Mixture-of-Recursions (MoR) is a unified framework built on a Recursive Transformer architecture, designed to enhance the efficiency of large language models. It achieves this by combining three core paradigms: parameter sharing (reusing shared layers across recursion steps), adaptive computation (dynamically assigning different processing depths to individual tokens via lightweight routers), and efficient Key-Value (KV) caching (selectively storing or sharing KV pairs). This integrated approach enables MoR to deliver large-model quality with significantly reduced computational and memory overhead, improving efficiency for both training and inference.

Comments 
In Channel
Kimi K2

Kimi K2

2025-07-2215:30

MeanFlow

MeanFlow

2025-07-1006:47

Mamba

Mamba

2025-07-1008:14

LLM Alignment

LLM Alignment

2025-06-1420:06

Why We Think

Why We Think

2025-05-2014:20

Deep Research

Deep Research

2025-05-1211:35

vLLM

vLLM

2025-05-0413:06

DeepSeek-Prover-V2

DeepSeek-Prover-V2

2025-05-0111:04

DeepSeek-Prover

DeepSeek-Prover

2025-05-0108:37

Agent AI Overview

Agent AI Overview

2025-03-1721:06

FlashAttention-3

FlashAttention-3

2025-03-0713:43

FlashAttention-2

FlashAttention-2

2025-03-0510:50

FlashAttention

FlashAttention

2025-03-0510:55

loading
00:00
00:00
x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

Mixture-of-Recursions (MoR)

Mixture-of-Recursions (MoR)

AI-Talk