DiscoverMicrosoft Research PodcastAbstracts: July 29, 2024
Abstracts: July 29, 2024

Abstracts: July 29, 2024

Update: 2024-07-29
Share

Description

A lack of appropriate data, decreased model performance, and other obstacles have made it difficult to expand the input language models can receive. Li Lyna Zhang introduces LongRoPE, a method capable of extending content windows to more than 2 million tokens.

Read the paper

Get the code

Comments 
loading
00:00
00:00
1.0x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

Abstracts: July 29, 2024

Abstracts: July 29, 2024

Researchers across the Microsoft research community