DiscoverArxiv PapersSiRA: Sparse Mixture of Low Rank Adaptation
SiRA: Sparse Mixture of Low Rank Adaptation

SiRA: Sparse Mixture of Low Rank Adaptation

Update: 2023-11-16
Share

Description

SiRA is a sparse mixture of low rank adaption approach that leverages sparse computation to improve the performance of large language models on downstream tasks. It outperforms other approaches in single task and multitask settings.




https://arxiv.org/abs//2311.09179




YouTube: https://www.youtube.com/@ArxivPapers




TikTok: https://www.tiktok.com/@arxiv_papers




Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016




Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers



Comments 
In Channel
loading
00:00
00:00
x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

SiRA: Sparse Mixture of Low Rank Adaptation

SiRA: Sparse Mixture of Low Rank Adaptation

Igor Melnyk