🔍 Click image to zoom
Frequently Asked Questions
What is Low-Rank Adaptation?
A parameter-efficient fine-tuning method that adapts large models by training only small low-rank weight matrices, reducing memory and compute requirements by 10–1000×. LoRA (Low-Rank Adaptation) is a PEFT technique introduced by Hu et al. in 2021 that approximates weight updates as the product of two small low-rank matrices rather than updating all model parameters.
How is Low-Rank Adaptation used in practice?
LoRA reduces the number of trainable parameters during fine-tuning by over 10,000× for large models, making it possible to fine-tune billion-parameter models on consumer-grade hardware.
Why is Low-Rank Adaptation important in AI?
Low-Rank Adaptation is a foundational concept in Training Technique. A parameter-efficient fine-tuning method that adapts large models by training only small low-rank weight matrices, reducing memory and compute requirements by 10–1000×.