CODETIPS ACADEMY / FINE-TUNING METHODS

Part 4 — Adapters, Bias-Only Updates, and Soft Prompts

Learn the PEFT family by training a bottleneck adapter and inspecting bias-only updates.

Project: Workshop Assistant · Audience: basic Python developers · Run: CPU; optional external training only in Part 10.

Keep the base; learn a smaller addition

Parameter-efficient fine-tuning, abbreviated PEFT, is a family of approaches that train a small subset of parameters or add a small trainable component. It describes where adaptation happens, not a particular loss function. A PEFT method can be used for classification or for supervised language-model training.

An adapter adds a small network to an existing computation. In our lab, the encoder produces hidden vector h with 16 numbers. The adapter compresses it to 4 numbers, applies a nonlinearity, and expands it back to 16. We add the result to h: h_new = h + up(relu(down(h))).

The original encoder and head stay frozen. Only down and up change. The skip connection preserves the original pathway. Because we initialize the up-projection weights to zero, the adapter initially contributes exactly zero. Before training, the adapted model therefore matches its base.

This is a simplified bottleneck adapter. Actual transformer adapter arrangements can use layer normalization, different insertion points, biases, and multiple modules per block. The lab illustrates the mechanism rather than claiming to reproduce every detail of a published architecture.

Count the trainable numbers

Without bias, down-projection has 4*16 = 64 numbers and up-projection has another 64. We train 128 parameters. A full 16-by-16 transform would contain 256 numbers. The savings depend on the bottleneck and the dimensions; on a tiny model, an adapter can be a large fraction of the original network.

import torch
from ft_lab import prepare, AdapterRouter, method, encode, train, counts
base, vocab, rows = prepare()
x,y = encode([r for r in rows if r['split']=='train'],vocab)
adapter = AdapterRouter(base,rank=4)
assert torch.allclose(adapter(x),base(x))
base_before = {k:v.clone() for k,v in adapter.base.state_dict().items()}
losses = train(adapter,x,y,steps=100,lr=.02)
assert all(torch.equal(v,adapter.base.state_dict()[k]) for k,v in base_before.items())
print('adapter:',counts(adapter),'loss:',round(losses[0],4),'->',round(losses[-1],4))
bias_model = method(base,'bitfit')
print('bias-only:',[name for name,p in bias_model.named_parameters() if p.requires_grad])
train(bias_model,x,y,steps=80,lr=.01)

adapter: {'trainable': 128, 'total': 611} loss: 0.0022 -> 0.0
bias-only: ['encoder.bias', 'head.bias']

allclose() verifies identical starting behavior within numerical tolerance. The frozen-state assertion checks every base tensor after training. Checking only the number of trainable parameters would not catch accidental changes caused by the wrong optimizer or an in-place operation.

Bias-only training

A linear layer computes Wx+b. Bias-only adaptation changes b while leaving W fixed. It has fewer degrees of freedom than changing all weights. In our router, changing a head bias can increase a class's score for every input, but it cannot invent a missing word feature. Print the named trainable parameters to see the actual restriction.

The lab's bitfit option demonstrates bias-only updating on this small network. It is not a replication of a large-language-model benchmark. The important question is whether the available changes are sufficient for your task.

Soft prompt tuning is a different idea

Ordinary prompting changes readable text. Soft prompt tuning learns continuous embedding vectors placed before the input embeddings of a frozen language model. Those vectors are parameters, not necessarily words you can print. For 10 virtual tokens of width 64, the prompt itself has 10*64 = 640 trainable values.

Prefix tuning is related but distinct: it learns continuous prefix information used within attention layers, commonly represented as attention key/value prefixes. It is not simply another name for editing a natural-language system prompt. Our bag-of-words router has no attention or token sequence, so we do not pretend that adding an input bias implements prefix tuning.

The optional pretrained-model notebook in Part 10 gives the context where embedding-based approaches belong. Here, understand the options before collecting libraries: inserted networks, selected existing parameters, and learned prompt vectors are different places to put adaptation.

Exercise and next step

Try adapter ranks 1, 2, 4, and 8. Record parameter count and validation results, keeping the same base. A wider adapter may fit the training examples faster without making new examples better. Choose a rank using validation, then report the final test score once.

Next, LoRA will put a small trainable update directly around an existing matrix multiplication. It will look similar to an adapter, but it is a linear weight update rather than a nonlinear hidden-state module.

Original method references: bottleneck adapters, soft prompt tuning, and prefix tuning.

← Part 3 · Part 5 →