Publications
You can also find my articles on my Google Scholar profile.
LoLDU: Low-Rank Adaptation via Lower-Diag-Upper Decomposition for Parameter-Efficient Fine-Tuning
Yiming Shi, Jiwei Wei, Yujia Wu, Ran Ran, Chengwei Sun, Shiyuan He, Yang Yang
arXiv 2024
TL;DR: LoLDU introduces a novel Parameter-Efficient Fine-Tuning (PEFT) method that reduces trainable parameters by 2600 times compared to traditional methods, using Lower-Diag-Upper Decomposition for faster convergence and orthogonality, while maintaining performance across various datasets and models.
DiffLoRA: Generating Personalized Low-Rank Adaptation Weights with Diffusion
Yujia Wu, Yiming Shi, Jiwei Wei, Chengwei Sun, Yuyang Zhou, Yang Yang, Heng Tao Shen
arXiv 2024
TL;DR: DiffLoRA leverages diffusion models to predict personalized low-rank adaptation weights, achieving efficient and identity-fidelity text-to-image generation without further training, by integrating these weights into the model during inference.