SiRA: Sparse Mixture of Low Rank Adaptation
Update: 2023-11-16
Description
SiRA is a sparse mixture of low rank adaption approach that leverages sparse computation to improve the performance of large language models on downstream tasks. It outperforms other approaches in single task and multitask settings.
https://arxiv.org/abs//2311.09179
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
Comments
In Channel




