📝 Hydra: Multi-Head Low-Rank Adaptation for Parameter Efficient Fine-Tuning 🔭

"Proposes a multi-branch adaption method named Hydra for fine-tuning large language models, which leverages both parallel and sequential adaptation methods simultaneously to combine the benefits of both." [gal30b+] 🤖 #CV

⚙️ https://github.com/extremebird/Hydra
🔗 https://arxiv.org/abs/2309.06922v1 #arxiv

https://creative.ai/system/media_attachments/files/111/067/599/465/898/447/original/9a67dc74843797b0.jpg

https://creative.ai/system/media_attachments/files/111/067/599/523/105/724/original/71cb40524aba6c0f.jpg

https://creative.ai/system/media_attachments/files/111/067/599/578/586/565/original/cb0de4240f71e918.jpg

https://creative.ai/system/media_attachments/files/111/067/599/637/465/725/original/627e9f0a0a78ca9e.jpg