The LoRA block decides which part of the model trains and with what capacity. Base weights stay frozen; only low-rank matrices learn — that is why memory and disk needs stay small.
- rank (r): training capacity; 8 for light style tasks, 16–32 for richer behaviour. Bigger r = bigger adapter.
- alpha: output scale of the LoRA path; the common convention is
alpha = 2×r. - dropout: 0.05 is typical and helps on small datasets.
- targets: attention projections (q,k,v,o) plus MLP (gate,up,down) — a solid default.
- QLoRA: loads the base model in 4-bit (NF4) so bigger models fit in less memory.
Tip: with the 0.5B model and r=8 you get ~4–9M trainable params — the final adapter file is tens of megabytes.
