Fine-Tune LLMs with LoRA and QLoRA Using Hugging Face PEFT

PEFT (Parameter-Efficient Fine-Tuning) is the family of techniques — and the Hugging Face library of the same name — for adapting a large pretrained model to a new task by training a small fraction of its weights instead of all of them, and LoRA (Low-Rank Adaptation) is by far its most widely used method. Instead of touching every parameter in a 7-billion or 70-billion parameter model, LoRA freezes the original weights entirely and trains a pair of tiny matrices bolted onto specific layers — often adjusting well under 1% of the total parameter count while getting most of the way to full fine-tuning quality. 🧬

This matters because it changes who gets to fine-tune a model at all. Full fine-tuning a 7B-parameter model can require well over 100GB of GPU memory once you count optimizer states and gradients — squarely hyperscaler territory. Layer in 4-bit quantization on top of LoRA (the "QLoRA" recipe), and PEFT's own documentation points to a model with 65 billion parameters being trainable on hardware as modest as one 48-gigabyte card. That's the difference between "only a handful of labs can do this" and "a single consumer GPU can do this on a weekend project" — and it's why LoRA adapters, not full checkpoints, are now the default unit of sharing on the Hugging Face Hub for anyone adapting an open model to a specific task. ⚡

The core LoRA idea this post builds from: freeze W, train only a low-rank B·A update.

📑 In This Post

1. What PEFT Actually Does (and Why Full Fine-Tuning Doesn't Scale)
2. How LoRA Works: Low-Rank Decomposition Explained
3. Configuring LoraConfig: r, alpha, target_modules, and the Knobs That Matter
4. QLoRA: Fine-Tuning a Quantized Model with bitsandbytes
5. Training a LoRA Adapter with Transformers and TRL
6. Beyond Vanilla LoRA: DoRA and Rank-Stabilized LoRA
7. Saving, Merging, and Serving Multiple Adapters
8. Evaluating a LoRA Fine-Tune Before You Trust It
9. Rolling Out PEFT/LoRA at Organizational Scale
10. Hands-On Lab: Fine-Tune Your First LoRA Adapter
11. Common Mistakes (and Why They Bite)
12. FAQ
13. References & Further Reading
14. Summary

🔀 Quick Comparison

Before the deep dive, here's the decision most people need to make first: how much of the model do you actually need to touch?

Approach	Trainable parameters	GPU memory (rough)	Best for
Full fine-tuning	100% of the model	Highest — weights + gradients + optimizer states, all in full precision	Small models, or when you have hyperscaler-grade compute
LoRA	Often well under 1% (only the injected B/A matrices)	Base model in full/half precision + small adapter gradients	Most task-specific fine-tuning on a single GPU
QLoRA	Same as LoRA	Base model quantized to 4-bit — lowest of the three	Fitting a much larger base model on limited GPU memory
Prompt/prefix tuning	A small set of learned embedding vectors, no weight changes	Lowest of all — no gradient flows through frozen weights	Very lightweight steering when LoRA is still more than you need

1. 🧊 What PEFT Actually Does (and Why Full Fine-Tuning Doesn't Scale)

✅ Real example: Hugging Face's own PEFT documentation carries forward the headline result from the original QLoRA research: combine 4-bit quantization with a PEFT adapter, and a 65-billion-parameter model becomes trainable using nothing more exotic than one 48GB card. Attempting the equivalent as a full fine-tune, by contrast, would demand far more memory than any single consumer or prosumer GPU offers — the optimizer bookkeeping alone for that many parameters outgrows the card before training even begins.

Kid analogy first: imagine a giant coloring book that a professional artist already filled in beautifully — every page is detailed, shaded, and correct. If you want to adapt just one picture for a birthday card, you wouldn't erase and redraw the whole book. You'd lay a thin sheet of transparent paper over just that one page and draw your changes on the transparent sheet. The original page underneath never gets touched, and you can always lift the transparent sheet off later. That transparent sheet is a PEFT adapter.

Full fine-tuning updates every single weight in a model, which means the optimizer needs to track a gradient and (for optimizers like Adam) two additional moment estimates for every one of those weights — multiplying the effective memory footprint several times over the base model's own size. PEFT methods sidestep this by freezing the pretrained weights entirely and introducing a much smaller set of new, trainable parameters. Because the frozen weights need no gradients or optimizer state, the memory savings compound: less to store, less to compute gradients for, and a far smaller file to save at the end.

💡 Contrasting case: PEFT isn't one technique — it's a family. LoRA is the most popular member, but the same library also implements prompt tuning, prefix tuning, P-tuning, and IA3, each trading off differently between how much they can change the model's behavior and how few parameters they touch. This post focuses on LoRA and its quantized cousin QLoRA because they currently offer the best balance of quality and adoption for most fine-tuning jobs — but if you find LoRA is still more compute than you need, it's worth knowing lighter PEFT methods exist.

🎯 Use a PEFT method whenever you're adapting a model that's expensive to fully retrain — which, for anything above a few hundred million parameters, is almost always.

2. 🧮 How LoRA Works: Low-Rank Decomposition Explained

✅ Real example: Predibase's "LoRA Land" collection — 25-plus task-specific adapters trained on top of a single shared Mistral-7B base and published to the Hugging Face Hub under the predibase organization — is a large-scale, public demonstration of exactly this mechanic: the same frozen base model, with a different small low-rank update swapped in depending on the task, from sentence-similarity scoring to SQL generation.

Kid analogy first: think about how you'd describe a big rectangular field of grass to a friend without drawing every single blade. Instead, you could say "picture a base green rectangle, then imagine a much simpler pattern — a few stripes going one way, a few going another — laid on top." Multiplying two small, simple patterns together can approximate a much more complicated one. LoRA does exactly this with a weight matrix.

Formally, LoRA represents the update to a weight matrix W (of size d × k) as the product of two much smaller matrices: B (size d × r) and A (size r × k), where the rank r is chosen to be far smaller than either d or k. During training, the original W stays frozen and only B and A receive gradient updates. At inference time, the effective weight is W + B·A — the low-rank update is simply added on top. Because r is small (commonly 8, 16, or 64), the number of trainable parameters in B and A combined is a tiny fraction of what's in W itself.

Out of the box, PEFT applies this low-rank decomposition specifically to two attention projections in every block — the ones handling queries and values — reflecting which layers researchers found move the needle most on adapted behavior; which layers actually get touched remains fully configurable, as the next section covers.

💡 Contrasting case: a higher rank r gives the adapter more capacity to represent complex changes, but it isn't free — more rank means more trainable parameters, more optimizer memory, and a larger adapter file to store and share. Predibase's own published guidance for its LoRA Land models used comparatively modest ranks precisely because the tasks were narrow and specific; a broad, multi-skill instruction-tuning run typically needs more capacity than a single narrow classification task does.

🎯 Use this mental model — frozen W, trainable low-rank B·A, added back together — every time you're deciding how aggressive an adapter configuration needs to be.

3. ⚙️ Configuring LoraConfig: r, alpha, target_modules, and the Knobs That Matter

✅ Real example: PEFT's own documentation for reproducing QLoRA-style training points users toward setting target_modules="all-linear" — applying LoRA to every linear layer in the transformer rather than only the attention projections — noting this reaches performance comparable to full fine-tuning, and that it's easier than hand-listing module names that vary from one model architecture to the next.

Kid analogy first: imagine adjusting a bicycle before a big race — seat height, gear ratio, tire pressure. Each adjustment is small on its own, but together they decide whether the ride is smooth or wobbly. LoraConfig is that set of adjustments for your adapter.

Here's a practical, original example touching the parameters that matter most day-to-day:

from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-0.5B")

lora_config = LoraConfig(
    r=16,                       # rank: how much capacity the adapter update gets
    lora_alpha=32,              # scaling factor, often set to 2x the rank
    target_modules=["q_proj", "v_proj"],  # which layers get an adapter
    lora_dropout=0.05,          # regularization on the adapter path only
    bias="none",                # "none", "all", or "lora_only"
    task_type=TaskType.CAUSAL_LM,
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# e.g. trainable params: 1,081,344 || all params: 495,114,240 || trainable%: 0.22

A few of these deserve a second look. r sets the rank — the capacity ceiling of the update. lora_alpha is a scaling factor applied to the low-rank update before it's added back to the frozen weights; a common convention is setting it to roughly twice the rank, though this is a starting point to tune, not a law. target_modules decides which layers actually get an adapter — the officially documented default targets only the query and value attention projections, but QLoRA-style training instead targets every linear layer via the "all-linear" shorthand. bias controls whether the small bias terms in the targeted layers are also trained. And task_type tells PEFT which model head shape to expect — causal language modeling, sequence classification, sequence-to-sequence, and so on — so it wires the adapter into the right place.

💡 Contrasting case: target_modules names vary by model family — q_proj/v_proj for many Llama-style architectures, different names for others — which is exactly the naming inconsistency the "all-linear" shortcut exists to sidestep. Hardcoding module names copied from a tutorial for a different model family is a common way to silently train zero adapters on the layers you actually meant to target.

🎯 Use this when you're setting up a new fine-tuning run — start from attention-only targeting for lighter tasks, and reach for "all-linear" when you need full-fine-tuning-level quality.

4. 🧊 QLoRA: Fine-Tuning a Quantized Model with bitsandbytes

✅ Real example: Hugging Face's own blog post announcing 4-bit quantization support explains that this integration made it possible to reproduce the QLoRA paper's results directly through the Transformers and PEFT libraries — training LoRA adapters on top of a model quantized to 4 bits rather than the usual 16 or 32, which is exactly the recipe behind many of the community fine-tunes now published on the Hub.

Kid analogy first: imagine a huge, detailed LEGO castle. Storing every single 1x1 brick's exact position takes a lot of space in your memory. Now imagine a compressed instruction sheet that says "this whole wall section is basically one repeated pattern" — you can rebuild something very close to the original castle from far less information. Quantization is that compression applied to a model's weights, and QLoRA is training a LoRA adapter on top of a model stored this compressed way.

QLoRA loads the frozen base model in 4-bit precision using bitsandbytes, then trains a full-precision LoRA adapter on top of it. Because the base weights themselves are frozen throughout, their reduced precision barely affects final quality — but it dramatically cuts the memory needed just to hold the model in GPU memory in the first place.

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
import torch

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    quantization_config=bnb_config,
    device_map="auto",
)

model = prepare_model_for_kbit_training(model)

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules="all-linear",
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)
model = get_peft_model(model, lora_config)

The call to prepare_model_for_kbit_training() isn't optional decoration — quantized weights are numerically fragile for direct training, and this function handles the preprocessing (like casting certain layers back to full precision and enabling gradient checkpointing hooks) that makes training a quantized model stable in the first place.

💡 Contrasting case: scaling QLoRA across multiple GPUs with Fully Sharded Data Parallel (FSDP) needs one more setting most single-GPU tutorials skip — bnb_4bit_quant_storage in BitsAndBytesConfig, which controls what data type the quantized weights are stored in so FSDP can shard them the same way it shards ordinary layers. Skipping it is a common reason FSDP-QLoRA setups that work on one GPU fail to scale to several.

🎯 Use QLoRA when the base model itself — not just the fine-tuning gradients — is the thing that doesn't fit in your GPU's memory.

5. 🏋️ Training a LoRA Adapter with Transformers and TRL

✅ Real example: Hugging Face's TRL (Transformer Reinforcement Learning) library documents training instruction-following models with its SFTTrainer and a use_peft flag, using a compact base model like Qwen2-0.5B and standard LoRA hyperparameters (rank 32, alpha 16) as its own quickstart example — the same basic recipe scales up to the much larger instruction-tuned models the community publishes on the Hub.

Kid analogy first: once you've decided which transparent sheet (adapter) to use and how it's cut (the config), you still need to actually practice drawing on it — running through examples, checking your work, adjusting your hand. That practice loop is the training loop.

Once get_peft_model() has wrapped your base model, the object behaves like an ordinary Transformers model for training purposes — you can hand it to the standard Trainer, or use TRL's SFTTrainer for instruction-tuning-style datasets, which will only compute gradients for the small set of adapter parameters even though the forward pass runs through the entire frozen network.

from trl import SFTTrainer, SFTConfig
from datasets import load_dataset

dataset = load_dataset("your-org/support-replies", split="train")

trainer = SFTTrainer(
    model=model,               # the PEFT-wrapped model from the previous section
    train_dataset=dataset,
    args=SFTConfig(
        output_dir="./support-reply-adapter-run",
        per_device_train_batch_size=4,
        gradient_accumulation_steps=4,
        num_train_epochs=3,
        learning_rate=2e-4,
    ),
)
trainer.train()
trainer.save_model("./support-reply-adapter")  # saves only the adapter weights

Note the learning rate: LoRA adapters are commonly trained with a noticeably higher learning rate than full fine-tuning would use, since you're only updating a small, freshly-initialized set of parameters rather than nudging an already-converged full model.

💡 Contrasting case: if you're resuming from a checkpoint, Trainer automatically detects adapter checkpoints and restores the correct trainable state — but this only works cleanly if the adapter configuration on resume matches the one used originally. Changing r or target_modules between runs and trying to resume from an older checkpoint is a fast way to get a shape-mismatch error instead of a resumed run.

🎯 Use TRL's SFTTrainer when your data looks like instruction/response pairs; use the plain Trainer for other task types like classification — either way, PEFT sits underneath transparently.

6. 🧬 Beyond Vanilla LoRA: DoRA and Rank-Stabilized LoRA

✅ Real example: PEFT's official LoRA developer guide documents DoRA (Weight-Decomposed Low-Rank Adaptation) as a supported, drop-in variant — enabled with a single extra flag on LoraConfig — and notes it can also be combined with 4-bit quantization as "QDoRA," while flagging a known rough edge: reported issues when combining QDoRA specifically with DeepSpeed ZeRO-2.

Kid analogy first: imagine describing how you moved a piece of furniture across a room. You could describe it as one combined motion, or you could separate it into two simpler descriptions: which direction it moved, and how far. DoRA does something similar to LoRA's weight update — it separately represents the update's direction and its magnitude, which the underlying research found gives the adapter more expressive power at the same rank.

Turning it on is a one-line change to the config you already know:

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    use_dora=True,   # enables DoRA on top of standard LoRA
    task_type="CAUSAL_LM",
)

A second, distinct improvement addresses a subtler issue: as you push the rank r higher to give an adapter more capacity, the standard LoRA scaling formula can make training less stable — gradients either shrink too much or grow too large as rank increases. Rank-stabilized LoRA adjusts the scaling math specifically so that going to a higher rank keeps training well-behaved instead of degrading it, which matters for anyone treating rank as a dial they can simply turn up for more capacity.

💡 Contrasting case: DoRA is documented as not yet supporting merge and unmerge operations with certain newer layer integrations, and PEFT's own guide notes DoRA is not currently supported with Transformer Engine layers at all. Before reaching for a newer variant because a paper reported better numbers, check the current PEFT developer guide for which combinations of variant, quantization backend, and distributed-training strategy are actually supported together — this is one of the fastest-moving parts of the library.

🎯 Use DoRA when you want more accuracy at the same parameter budget and can confirm it's compatible with the rest of your training stack; reach for rank-stabilized scaling specifically when you're experimenting with higher ranks.

7. 📦 Saving, Merging, and Serving Multiple Adapters

✅ Real example: Predibase's LoRAX serving framework — the same infrastructure behind LoRA Land — is built specifically around this pattern: one shared Mistral-7B copy in GPU memory, with dozens of published adapters loaded and swapped per request, so that serving many fine-tuned "models" costs roughly the same as serving one.

Kid analogy first: it's like owning one nice camera body and several interchangeable lenses instead of buying a whole new camera for portraits, one for landscapes, and another for sports. The expensive shared part (the base model) stays put; you swap the cheap, specialized part (the adapter) depending on what you're doing right now.

An adapter's own save_pretrained() writes only adapter_config.json and adapter_model.safetensors — typically a few megabytes regardless of how large the frozen base model is. Loading it back means loading the base model once, then attaching whichever adapter you need:

from transformers import AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1")

# Swap adapters on the same base model as needed:
support_model = PeftModel.from_pretrained(base, "your-org/support-reply-adapter")
sql_model = PeftModel.from_pretrained(base, "your-org/sql-generation-adapter")

# Once you've settled on one for production, fuse it for lower latency:
merged = support_model.merge_and_unload()
merged.save_pretrained("./support-reply-merged")

merge_and_unload() fuses the adapter into the base weights and returns a plain, standard model — remember it isn't an in-place operation, so the result has to be assigned to a variable and used directly, or the merge has no effect on anything you actually deploy.

💡 Contrasting case: merging is the right move for a single, settled production variant, but it defeats the entire point of the LoRA Land-style pattern if you're serving many tasks from one base model — merging bakes in exactly one adapter and gives up the ability to swap others in without reloading the whole base model again.

🎯 Keep adapters unmerged and swappable when you're serving many task variants from one base; merge only the one variant you've committed to shipping as a standalone deployment.

8. 📊 Evaluating a LoRA Fine-Tune Before You Trust It

Kid analogy first: after adjusting the bicycle from Section 3, you don't just assume it's race-ready — you take it for a test ride on a track you know well before entering the actual race. A held-out evaluation set is that test ride for a fine-tuned model.

✅ Real example: Predibase's own published benchmarking for LoRA Land reports task-by-task comparisons against both the unmodified base model and GPT-4 rather than a single aggregate number — a reminder that "the adapter improved performance" needs to be measured per task the adapter was actually trained for, not assumed from a single global score.

Because a LoRA adapter changes model behavior just as much as full fine-tuning can, it deserves the same evaluation discipline: a held-out split the adapter never saw during training, metrics appropriate to the actual task (exact-match or F1 for extraction, BLEU or ROUGE for generation, accuracy for classification), and — ideally — a side-by-side comparison against the unmodified base model so you can see whether the adapter is actually helping on cases outside its training distribution. Hugging Face's Evaluate library provides ready-made implementations of many of these metrics so you're not reimplementing BLEU or ROUGE from scratch for every project.

💡 Contrasting case: a common false signal is a dropping training loss with no corresponding held-out evaluation — a low-rank adapter has far fewer parameters than the full model, so it's less prone to blatant memorization than full fine-tuning, but it can still overfit to quirks of a small training set, especially at higher ranks with more capacity to spare.

🎯 Use this before promoting any adapter past your own experiment — a held-out eval is what turns "the loss went down" into "this actually works."

9. 🏢 Rolling Out PEFT/LoRA at Organizational Scale

Kid analogy first: a photography studio with one shared camera body and a labeled drawer of lenses — each lens tagged with what it's for, who's allowed to use the expensive ones, and a log of which lens was on the camera for which shoot — works very differently from a pile of lenses on a table that anyone grabs at random. Once an organization has more than a couple of adapters, it needs the drawer, not the pile.

1. An adapter registry, not a folder of files: as the number of task-specific adapters grows — one per customer, one per language, one per feature — each needs a discoverable Hub repository with a clear name, an owner, and a model card describing which base model and revision it was trained against.
2. Base model licensing applies to every adapter built on it: a LoRA adapter is not a legally separate artifact from the base model it modifies — if the base model's license restricts commercial use or requires attribution, every adapter trained on top of it inherits those same restrictions, which needs to be tracked at the organization level, not left to whoever trained the adapter.
3. Revision-pinning the base model matters even more here: an adapter is trained against one specific version of a base model's weights; loading that same adapter onto a different revision of the base model later can silently produce degraded or nonsensical outputs, since the adapter's learned update was computed relative to specific frozen weights.
4. Cost governance for multi-adapter serving: the entire economic case for the LoRA Land-style pattern is serving many adapters from one shared GPU; losing track of which adapters are still receiving traffic and which are dead weight erodes exactly the cost advantage that made adopting LoRA attractive in the first place.
5. Evaluation gates per adapter, not per base model: because each adapter specializes the same base model differently, passing an eval gate for one task-specific adapter says nothing about whether a different adapter trained on different data is production-ready — gates need to be scoped to the adapter, not assumed from a sibling adapter's results.

🎯 Use this checklist the moment a team has more than one or two adapters in flight — an unlabeled pile of small adapter files is easy to create and surprisingly hard to untangle later.

10. 🧪 Hands-On Lab: Fine-Tune Your First LoRA Adapter

This lab trains a real, tiny LoRA adapter on a real (very small) model, so it runs quickly even on modest hardware or a free-tier notebook GPU. You'll need Python with transformers, peft, datasets, and trl installed.

1

Load a small base model and wrap it with a LoRA config:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model

model_id = "sshleifer/tiny-gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

lora_config = LoraConfig(r=8, lora_alpha=16, target_modules=["c_attn"], task_type="CAUSAL_LM")
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()

Expect to see: a "trainable params" line showing a trainable percentage well under 5%. Troubleshooting: if it reports 0 trainable parameters, your target_modules name doesn't match this model's actual layer names — print model to see the real module names for whichever base model you're using.

2

Build a tiny toy dataset — a handful of repeated examples is enough to prove the mechanics work:

from datasets import Dataset

examples = [{"text": "Q: What is PEFT?\nA: A way to fine-tune only a small set of new parameters.\n"}] * 20
dataset = Dataset.from_list(examples)

Expect to see: a Dataset object reporting 20 rows.

3

Train for a couple of quick epochs with TRL's SFTTrainer:

from trl import SFTTrainer, SFTConfig

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    args=SFTConfig(output_dir="./lab-adapter-run", num_train_epochs=3, per_device_train_batch_size=4),
)
trainer.train()

Expect to see: a decreasing loss value printed each logging step. Troubleshooting: if you get a tokenizer padding error, set tokenizer.pad_token = tokenizer.eos_token before training — many small GPT-2-style models don't define one by default.

4

Save just the adapter, then reload it onto a fresh copy of the base model to confirm it's genuinely portable:

model.save_pretrained("./lab-adapter")

from peft import PeftModel
fresh_base = AutoModelForCausalLM.from_pretrained(model_id)
reloaded = PeftModel.from_pretrained(fresh_base, "./lab-adapter")

Expect to see: the ./lab-adapter folder containing only adapter_config.json and adapter_model.safetensors — a few hundred kilobytes for this tiny model, nowhere close to the full model's size.

5

Optionally, push the adapter to the Hub the same way you'd push a full model:

model.push_to_hub("your-username/my-lab-lora-adapter-DELETE-ME", private=True)

Expect to see: a repo URL, and on the Hub, a repo whose "Files and versions" tab shows only the two small adapter files — not a full model checkpoint.

This toy loop — config → tiny dataset → train → save the adapter → reload onto a fresh base — is the exact same shape as the Predibase LoRA Land pattern from Section 2 and 7, just at a scale you can run on a laptop. Everything in Sections 4, 6, and 9 (quantization, DoRA, governance) is this same loop with more sophistication layered on top.

11. 🚫 Common Mistakes (and Why They Bite)

• Copying target_modules from a tutorial for a different model architecture. Module names like q_proj/v_proj aren't universal across every model family; a mismatch silently produces zero trainable parameters rather than a loud error, so always verify against print_trainable_parameters().
• Treating r as a free capacity dial with no downside. Higher rank increases trainable parameters, adapter file size, and — without rank-stabilized scaling — can actually destabilize training rather than simply improving it.
• Forgetting prepare_model_for_kbit_training() before training on a quantized model. Skipping this step on a QLoRA setup can produce unstable gradients or outright training failures that look like a data or learning-rate problem instead of the actual missing preprocessing step.
• Assuming merge_and_unload() is in-place. It returns a new model object; forgetting to reassign the result means your "merged" model is still the unmerged one, and you'll deploy the wrong artifact.
• Loading an adapter onto the wrong base model revision. An adapter's learned update is computed relative to one specific set of frozen weights; pairing it with a different revision — even a seemingly minor update to the base model — can produce degraded or incoherent outputs with no error at load time.
• Skipping a held-out evaluation because "it's just a small adapter." A LoRA adapter changes behavior just as meaningfully as full fine-tuning for the layers it touches; fewer trainable parameters reduces but does not eliminate the risk of overfitting to a narrow training set.
• Not tracking which base-model license an adapter inherits. An adapter trained on a restrictively licensed base model carries those same restrictions forward — publishing the adapter freely doesn't make the combination free to use however a downstream user likes.

❓ FAQ

Is LoRA the same thing as PEFT?

No — PEFT is the broader category (and the Hugging Face library name), and LoRA is the most widely used method inside that category. The same library also implements prompt tuning, prefix tuning, and other lighter-weight approaches.

Do I need a powerful GPU to try LoRA fine-tuning?

For small models (under a billion parameters or so), a modest single GPU is often enough for plain LoRA. For much larger base models, QLoRA's 4-bit quantization exists specifically to keep single-GPU training realistic — PEFT's own docs use a 65-billion-parameter model running on just one 48GB card as their flagship illustration of how far this stretches.

Should I always use the highest rank I can afford?

Not necessarily. Higher rank gives more capacity but also more parameters to train and store, and can introduce training instability at very high ranks without rank-stabilized scaling. Start modest (8–16) and increase only if evaluation shows the adapter is capacity-limited, not just because a bigger number seems safer.

Can I serve many LoRA adapters without needing a GPU per adapter?

Yes — this is exactly the pattern demonstrated at scale by Predibase's LoRA Land, where dozens of adapters share one base model copy on a single GPU, with adapters swapped per request instead of each requiring dedicated hardware.

Should I merge my adapter into the base model before deploying?

Merge when you're deploying exactly one adapter permanently and want to eliminate any adapter-swap overhead. Keep it unmerged if you're serving multiple task variants from the same base model, since merging locks in only one of them.

🔗 References & Further Reading

Official/primary documentation relied on for technical accuracy:

• PEFT docs — Quicktour
• PEFT docs — LoraConfig package reference
• PEFT docs — LoRA developer guide (DoRA, all-linear, QDoRA)
• PEFT docs — Quantization guide (QLoRA)
• bitsandbytes docs — FSDP-QLoRA
• Transformers docs — PEFT integration and Trainer
• Hugging Face blog — 4-bit quantization with bitsandbytes and QLoRA

Hugging Face, the Hugging Face logo, Transformers, PEFT, TRL, and related names are trademarks of Hugging Face, Inc. Mistral and Mistral-7B are associated with Mistral AI. These names are used here descriptively to refer to their actual products. 

📝 Summary

• PEFT freezes the base model and trains a much smaller set of new parameters, avoiding the memory cost of full-model gradients and optimizer states.
• LoRA's core mechanic is decomposing a weight update into two small matrices, B and A, added back onto the frozen weight as W + B·A.
• LoraConfig — r, lora_alpha, target_modules, bias, and task_type — is where you tune the tradeoff between capacity and cost, with "all-linear" as the QLoRA-style shortcut for maximum coverage.
• QLoRA combines 4-bit quantization of the frozen base with a full-precision LoRA adapter, making very large base models fine-tunable on a single consumer-class GPU.
• Training runs through the ordinary Trainer or TRL's SFTTrainer once the model is PEFT-wrapped — no special training loop required.
• DoRA and rank-stabilized LoRA refine the basic recipe for more accuracy or more stable training at higher ranks, at the cost of needing to check compatibility with the rest of your stack.
• Adapters stay small and swappable — save them separately, merge only the one variant you're committing to production, and consider a LoRA-Land-style shared-base serving pattern for many task variants.
• Evaluation and governance matter per-adapter, not just per base model — licensing, revision-pinning, and held-out evals all need to travel with each individual adapter.

Freeze what you don't need to touch, train the small part that matters, and keep the receipts on which base revision and license every adapter depends on. 🤗