Merging & Multiple Adapters

Merging LoRA into the base model

After training, you can merge LoRA adapter weights back into the base model for inference. This eliminates any overhead from the adapter architecture — the merged model is byte-identical in shape to the base model and can be served by anything (vLLM, TGI, Ollama, llama.cpp).

Multiple adapters on one base

You can also keep adapters separate and load multiple of them on the same base model. Useful for serving different tasks/customers without duplicating the (large) base model. vLLM and TGI both support per-request adapter switching at serving time (Track 8).

Apple Silicon path

On M-series Macs you can fine-tune with LoRA via the MLX framework (mlx-lm). It supports LoRA and QLoRA on M-series chips with unified memory. Performance is slower than CUDA H100s but practical for 7B–13B models on machines with 64 GB+ unified memory, and the export path back to GGUF for Ollama is clean.

Code

Merge LoRA into base for clean serving·python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "meta-llama/Llama-3.1-8B-Instruct"
base = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.float16, device_map="auto",
)
tok = AutoTokenizer.from_pretrained(base_id)

# Load LoRA adapter on top
model = PeftModel.from_pretrained(base, "./my-lora-adapter")

# Merge — adapters become part of base; PeftModel wrapper goes away
merged = model.merge_and_unload()

# Save the merged model
merged.save_pretrained("./merged-model")
tok.save_pretrained("./merged-model")
print("Merged model ready for vLLM/TGI/Ollama serving.")

Multiple adapters with weighted merging·python

from peft import PeftModel

# Load base + first adapter
model = PeftModel.from_pretrained(
    base, "adapter-medical", adapter_name="medical",
)

# Add more adapters on the same base
model.load_adapter("adapter-legal",   adapter_name="legal")
model.load_adapter("adapter-finance", adapter_name="finance")

# Switch active adapter per request
model.set_adapter("medical");  out_med = model.generate(...)
model.set_adapter("legal");    out_leg = model.generate(...)

# Combine adapters with weights — useful when domains overlap
model.add_weighted_adapter(
    adapters=["medical", "legal"],
    weights=[0.7, 0.3],
    adapter_name="medlegal",
    combination_type="linear",   # or "ties", "dare_ties"
)
model.set_adapter("medlegal")

Exercise

Take a LoRA adapter you've trained (or download one from the Hub). Merge it into the base model and save the result. Load both the original adapter setup and the merged model and verify they produce identical outputs on 5 test prompts. Confirm the merged model has no PEFT dependency at inference time.