Glossary
LoRA
A fine-tuning method that trains a small set of extra parameters instead of the whole model.
- Also called
- low-rank adaptation
Low-rank adaptation freezes the base weights and trains small adapter matrices alongside them. Training is dramatically cheaper, and adapters can be swapped per task at inference.
QLoRA adds quantisation, making it possible to fine-tune large models on a single GPU.
In practice LoRA is what makes domain tuning affordable for a mid-sized company. A full fine-tune of a large model needs a GPU cluster and a team to run it; a LoRA adapter on the same base can be trained overnight on a single card and swapped in and out per customer or per task. That last property matters more than the cost saving for anyone building multi-tenant software.
A practical note on serving: adapters can be loaded and unloaded at runtime, so one base model in memory can serve several tuned behaviours. That is what makes per-customer tuning economically sane in a multi-tenant product, where a separate full model per customer never would be.
Related terms, in context
The concepts you almost always meet alongside lora.
- Fine-tuning
- Further training a base model on your own examples, to fix style, format or narrow task behaviour.
- Quantisation
- Reducing the numeric precision of model weights to cut memory and increase speed.
- Open-weight models
- Models whose parameters you can download and run yourself, on your own infrastructure.
Where this shows up in our work
LoRA is not an abstraction for us. It is a decision we make on live projects. It shows up most directly in custom model fine-tuning, where getting it wrong has a cost someone can measure.
If you are evaluating a vendor on this, the useful question is not whether they can define the term. It is what they measure, what they would refuse to do, and what happens in their system when the assumption behind lora stops holding.
Questions
What is LoRA?
A fine-tuning method that trains a small set of extra parameters instead of the whole model.
Does Orqent Labs build this?
Yes, Custom Model Fine-tuning. We work across India, covering all 19,238 PIN codes remotely.
Building something that involves lora?
We will tell you honestly whether it is the right approach for your problem.
Or email bd@dtrasglobal.com · call +91 74118 77878