SDXL Fine-Tuning VRAM: 15GB for LoRA

SDXL full fine-tuning requires about 24 GB VRAM minimum and 40 GB recommended. LoRA training requires 12-16 GB VRAM depending on resolution and optimizations.

For SDXL inference VRAM needs, see the SDXL VRAM requirements guide.

SDXL inference and SDXL training are different workloads with different memory profiles. Inference loads model weights and runs a single forward pass. Training loads weights, optimizer states, gradients, and activation checkpoints simultaneously. The gap between the two is not incremental — it is a category change in VRAM consumption.

This guide breaks down where the memory goes during SDXL fine-tuning, what the theoretical requirements are, and why practical setups can run on less than the formula predicts.

Quick Answer

The theoretical memory requirement for full SDXL fine-tuning exceeds 46 GB. In practice, optimizer quantization and gradient checkpointing bring this down to a range where 24 GB GPUs can handle batch size 1. LoRA training sidesteps the problem entirely by training a small fraction of the parameters.

Why SDXL Needs So Much VRAM

The SDXL model architecture determines the baseline memory floor. The total model contains approximately 6.6 billion parameters across all components. The UNet — the component being fine-tuned — accounts for roughly 2.6 billion of those parameters.

In fp16 (half precision), each parameter occupies 2 bytes. The SDXL UNet in fp16 requires approximately 5.2 GB just for weights. (Theoretical: 2.6B x 2 bytes.) In fp32 (full precision), each parameter occupies 4 bytes, bringing the UNet weight footprint to approximately 10.4 GB. (Theoretical: 2.6B x 4 bytes.)

During inference, the GPU holds the model weights and processes one forward pass at a time — typically consuming roughly 11-21 GB depending on optimization techniques. HuggingFace measurements show approximately 21 GB with basic fp16 inference and 11-15 GB when using optimizations such as VAE slicing and CPU offloading. During training, the GPU must simultaneously hold weights, a copy of every gradient, and the optimizer's internal state tensors. This is why the same model that runs inference at 11-21 GB demands multiples of that for training.

For a broader overview of SDXL memory behavior across inference and training, see the SDXL VRAM requirements guide.

The Memory Formula for Training

HuggingFace's model parallelism documentation provides a rule of thumb for mixed-precision training with AdamW: approximately 18 bytes per parameter.

That 18-byte figure breaks down as follows:

• Model weights (fp32 copy): 4 bytes per parameter
• Gradients (fp32): 4 bytes per parameter
• Adam optimizer states: 8 bytes per parameter (momentum + variance, both fp32)
• Model weights (fp16 copy for forward/backward): 2 bytes per parameter

Total: 18 bytes per parameter.

Theoretical estimate for SDXL UNet:

2.6 billion parameters x 18 bytes = ~46.8 GB

This is the memory required for the UNet parameters, gradients, and optimizer states alone — before accounting for activations, the VAE, text encoders, or any input data. Activations add additional memory that scales with resolution and batch size.

This estimate explains why naive full fine-tuning of SDXL does not fit on consumer GPUs. A 24 GB card cannot hold 46.8 GB of parameter-related memory. The gap must be closed by optimization techniques that reduce the per-parameter memory footprint.

Why Full SDXL Fine-Tuning Can Still Run on 24 GB GPUs

The 46.8 GB theoretical estimate assumes standard fp32 AdamW with no memory optimizations. Several techniques reduce this substantially.

8-bit Adam (AdamW8bit) replaces the fp32 optimizer states with quantized 8-bit versions. Standard AdamW stores 8 bytes per parameter in optimizer state. 8-bit Adam reduces this to approximately 2 bytes per parameter. This single change reduces the theoretical total from ~18 bytes/parameter to ~12 bytes/parameter — a reduction of roughly 15.6 GB for the SDXL UNet. (Theoretical: (8 - 2) bytes x 2.6B parameters = 15.6 GB saved.)

Gradient checkpointing trades compute for memory by discarding intermediate activations during the forward pass and recomputing them during backpropagation. This reduces activation memory by approximately 30-50%, at the cost of slower training due to recomputation.

Mixed precision training keeps a master copy of weights in fp32 for numerical stability while running the forward and backward passes in fp16, reducing the memory footprint of the working copy.

Embedding precomputation pre-encodes text and image inputs before training begins, removing the text encoder and VAE from VRAM during the training loop. Only the UNet and its training state occupy the GPU.

A HuggingFace diffusers maintainer has confirmed that with 8-bit Adam, gradient checkpointing, mixed precision, and pre-saved embeddings, SDXL full fine-tuning at batch size 1 can run on a 24 GB GPU (RTX 3090 or RTX 4090). This aligns with the reduced theoretical estimate: at ~12 bytes/parameter, the UNet's parameter memory drops to approximately 31.2 GB (theoretical: 2.6B x 12 bytes), and with activation reduction from gradient checkpointing plus embedding precomputation removing the text encoder and VAE, the total fits within the 24 GB envelope.

The margin is tight. Loading additional components — a custom VAE, text encoder fine-tuning, or a validation pipeline — alongside training can push memory past the limit. On 24 GB, full fine-tuning works only when the GPU is dedicated entirely to the training process.

LoRA Training VRAM Requirements

LoRA (Low-Rank Adaptation) trains a small set of adapter parameters inserted into the existing model, rather than updating all 2.6 billion UNet parameters. The frozen model weights still occupy VRAM, but optimizer states and gradients apply only to the adapter parameters — a small fraction of the total model.

This changes the memory equation dramatically. Measurements from Puget Systems show SDXL LoRA training with gradient checkpointing peaking at approximately 13-14.8 GB VRAM. Community reports from Civitai corroborate this range, with users reporting approximately 13-15 GB peak VRAM for SDXL LoRA at 1024px resolution with gradient checkpointing enabled.

Without gradient checkpointing, LoRA training on 16 GB GPUs becomes unreliable — activation memory pushes total usage above the available headroom. With checkpointing enabled, 16 GB cards handle LoRA training at 1024px resolution.

Training at 512px resolution instead of 1024px reduces VRAM usage by approximately 40-50%, making LoRA accessible on lower-VRAM cards. However, training at reduced resolution limits the quality of the adapter when generating at SDXL's native 1024px.

For a complete breakdown of VRAM requirements across all Stable Diffusion versions and tasks, see the Stable Diffusion GPU requirements guide.

Practical GPU Tiers for SDXL Training

16 GB — RTX 4080 / RTX 4070 Ti Super

Full fine-tuning is not viable at this tier. Even with every optimization enabled, the combined memory of model weights, quantized optimizer states, and gradients exceeds 16 GB before activations are accounted for.

LoRA training works well. With gradient checkpointing, measured peak VRAM stays in the 13-15 GB range, leaving a small but usable margin on 16 GB cards. This is the entry point for SDXL LoRA on local hardware.

24 GB — RTX 4090 / RTX 3090

Full fine-tuning is possible at batch size 1 with 8-bit Adam, gradient checkpointing, mixed precision, and pre-saved embeddings. The margin is narrow — no additional models or processes should share the GPU during training.

LoRA training runs comfortably with headroom for experimentation.

This is the most common local GPU tier for SDXL training. For cloud rental economics at this tier, see the RTX 4090 cloud rental analysis.

40-48 GB — A100 40GB / A6000 48GB

Full fine-tuning runs without aggressive optimization at this tier. Standard AdamW (not quantized) becomes viable, since the 46.8 GB theoretical baseline fits within the available memory once gradient checkpointing reduces activation overhead. The additional headroom accommodates batch sizes above 1 and eliminates the fragility of 24 GB setups.

These GPUs are primarily available through cloud providers. An A100 40GB instance typically rents for approximately $0.63/hr.

80 GB — A100 80GB / H100

Full fine-tuning without constraints. The memory envelope comfortably exceeds the theoretical baseline, allowing larger batch sizes, multi-resolution training, and experimental hyperparameter sweeps. This tier is reserved for production training pipelines where stability matters more than cost per hour.

How to Reduce VRAM Usage

If training crashes with a CUDA out-of-memory error, these are the highest-impact interventions:

Enable gradient checkpointing. This is the single most effective VRAM reduction for training. It discards activation tensors during the forward pass and recomputes them during backpropagation, reducing activation memory by approximately 30-50%. The tradeoff is slower training due to recomputation during the backward pass.

Enable mixed precision. Running the forward and backward passes in fp16 while keeping the optimizer in fp32 reduces the working memory of model weights by half compared to pure fp32 training.

Use 8-bit Adam. Replacing standard AdamW with AdamW8bit reduces optimizer state memory from ~8 bytes to ~2 bytes per parameter — a theoretical reduction of approximately 15.6 GB for the SDXL UNet.

Use gradient accumulation instead of larger batch sizes. Each additional sample in a batch stores its own set of activation tensors, increasing VRAM proportionally. Accumulating gradients over multiple forward passes at batch size 1 produces the same effective gradient without the memory cost. The tradeoff is more forward passes per effective step.

Pre-save text and image embeddings. Encoding all training inputs before the training loop begins removes the text encoder and VAE from VRAM during training. Only the UNet and its training state remain on the GPU.

Use xFormers or memory-efficient attention. These attention implementations reduce the memory footprint of self-attention layers, which is particularly relevant for the SDXL UNet's transformer blocks.

For a step-by-step CUDA OOM diagnosis workflow, see the CUDA out of memory troubleshooting guide.

When to Use Cloud GPUs

Full fine-tuning creates the strongest case for cloud GPU rental in the Stable Diffusion ecosystem. The reasoning is straightforward:

LoRA training fits on consumer GPUs (16-24 GB). If you train LoRA adapters regularly, local hardware makes sense.

Full fine-tuning needs 24 GB at minimum with aggressive optimization, and 40 GB+ for stable operation without constant OOM risk. Consumer GPUs cap at 24 GB. Cloud providers offer 40-80 GB cards at hourly rates — an A100 40GB rents for approximately $0.63/hr.

Cloud also provides access to the 40-80 GB VRAM tier that does not exist in consumer hardware. Increasing batch size beyond 1 for full fine-tuning adds activation memory proportionally — each additional sample in the batch stores its own set of intermediate tensors. On 24 GB, there is no room for this. Cloud GPUs with 40 GB+ provide the headroom that batch sizes above 1 require.

Browse available GPU instances on the SynpixCloud marketplace, or estimate costs with the GPU cost calculator.

Conclusion

The theoretical memory requirement for SDXL full fine-tuning is approximately 46.8 GB before activations — derived from the 18-byte-per-parameter rule for mixed-precision AdamW training applied to the 2.6 billion parameter UNet.

In practice, 8-bit Adam, gradient checkpointing, mixed precision, and embedding precomputation reduce this enough to fit on 24 GB GPUs at batch size 1. The setup is tight but confirmed viable by HuggingFace maintainers.

LoRA training operates in a different regime entirely. Measured peak VRAM of 13-14.8 GB with gradient checkpointing means 16 GB consumer GPUs handle LoRA comfortably. If LoRA produces the results you need, there is no reason to pursue full fine-tuning's memory requirements.

The decision framework: use LoRA on local hardware when adapter-level changes suffice. Use cloud GPUs with 40 GB+ VRAM when full fine-tuning is necessary and batch size 1 on 24 GB is too constrained.

Frequently Asked Questions

How much VRAM is needed to fine-tune SDXL?

It depends on the training method. LoRA training peaks at approximately 13-14.8 GB with gradient checkpointing enabled. Full fine-tuning theoretically requires ~46.8 GB with standard AdamW, but optimizations like 8-bit Adam and gradient checkpointing can bring it within 24 GB at batch size 1. For stable full fine-tuning without aggressive optimization, 40 GB or more is recommended.

Can SDXL be fine-tuned on a 24GB GPU?

Yes. A HuggingFace diffusers maintainer has confirmed that SDXL full fine-tuning at batch size 1 can run on a 24 GB GPU (RTX 3090 or RTX 4090) when using 8-bit Adam, gradient checkpointing, mixed precision, and pre-saved embeddings. The margin is tight — the GPU should be dedicated entirely to the training process with no other models loaded.

Is LoRA training possible on a 12GB GPU?

LoRA training at 1024px resolution typically peaks at 13-14.8 GB with gradient checkpointing, which exceeds 12 GB. Training at 512px resolution reduces VRAM usage by approximately 40-50%, which can bring it within range of a 12 GB card. Without reducing resolution, a 16 GB GPU is the practical minimum for SDXL LoRA.

Why does SDXL training require more VRAM than inference?

Inference only holds model weights and processes one forward pass. Training must simultaneously store model weights, a full copy of gradients, and optimizer states — AdamW stores two additional tensors per parameter. The HuggingFace 18-byte-per-parameter rule captures this: inference needs roughly 2 bytes per parameter (fp16 weights), while training needs approximately 18 bytes per parameter before activations.

---

Last updated: March 2026