LoRA Training Cost on H100 (2026): Real GPU-Hour Math

Translating H100 hours to dollar cost requires knowing your effective tokens-per-second throughput. The number depends on model, sequence length, batch size, LoRA rank, and the training framework.

**Llama 4 70B LoRA (rank 16, 8K context, batch 4)** on 2x H100 80GB with axolotl + DeepSpeed Zero-2 achieves approximately 3,000-5,000 effective tokens/second/GPU, so 6,000-10,000 tokens/second total across the 2 GPUs. A 22.5M-token job completes in approximately 0.75-1 hour wall-clock = 1.5-2 GPU-hours per GPU × 2 GPUs = 3-4 H100-hours total. Add 20-30% overhead for warmup, checkpointing, and validation runs = 6-8 H100-hours total.

**Llama 4 70B QLoRA (rank 16, 4-bit base, 8K context, batch 2)** on 1x H100 80GB achieves approximately 2,000-3,000 effective tokens/second (slower due to 4-bit dequant overhead in forward pass, but only one GPU is used). A 22.5M-token job completes in approximately 2-3 hours wall-clock = 2-3 H100-hours. Add overhead = 3-4 H100-hours total.

**Llama 4 8B LoRA (rank 16, 8K context, batch 16)** on 1x H100 80GB achieves approximately 25,000-40,000 effective tokens/second. A 22.5M-token job completes in approximately 10-15 minutes wall-clock = 0.2-0.3 H100-hours. Including overhead, 0.5-1 H100-hour.

**Llama 4 405B LoRA (rank 16, 4K context, batch 1)** on 8x H100 80GB cluster achieves approximately 800-1,200 effective tokens/second/GPU. A 22.5M-token job completes in approximately 0.5-0.75 hour wall-clock = 4-6 GPU-hours per GPU × 8 = 32-48 H100-hours total. Including overhead, 40-60 H100-hours.

**Mistral 8x22B LoRA** behaves similarly to Llama 4 70B in throughput — ~3,500-5,000 tokens/sec on 2x H100 80GB.

**Qwen 2.5 32B LoRA** sits between Llama 4 8B and 70B — ~8,000-15,000 tokens/sec on 1x H100 80GB.

**DeepSeek-V3 LoRA** is roughly 70B-class throughput — ~3,000-5,000 tokens/sec on 2x H100 80GB.

Standardized workload: 5,000 examples × 1,500 tokens average × 3 epochs = 22.5M training tokens. Dollar cost = H100-hours × hourly rate.

**Llama 4 70B LoRA**: 6-8 H100-hours. At $2.50/hr spot (Lambda/RunPod): $15-20. At $4.50/hr on-demand: $27-36. At $5.99/hr spot AWS p5: $36-48.

**Llama 4 70B QLoRA**: 3-4 H100-hours (single GPU). At $2.50/hr spot: $8-10. At $4.50/hr on-demand: $14-18.

**Llama 4 8B LoRA**: 0.5-1 H100-hour. At $2.50/hr spot: $1.25-2.50. At $4.50/hr on-demand: $2.25-4.50.

**Llama 4 405B LoRA**: 40-60 H100-hours (8x cluster). At $2.89/hr cluster spot: $115-174. At $4.49/hr cluster on-demand: $180-270.

**Mistral 8x22B LoRA**: 6-8 H100-hours. At $2.50/hr spot: $15-20.

**Qwen 2.5 32B LoRA**: 2-3 H100-hours. At $2.50/hr spot: $5-7.50.

**Comparison to hosted platforms**: Together AI charges $1.20/1M tokens for Llama 4 70B LoRA = $27 for the same job, slightly more expensive than the cheapest self-hosted spot ($15-20) but less than on-demand or AWS p5. Self-hosting saves 30-50% on training cost but adds engineering effort: cluster setup, environment configuration, monitoring, and recovery from spot interruption.

H100 capacity comes in three tiers and the right choice depends on workload urgency and engineering tolerance.

**Spot pricing** is 40-50% cheaper than on-demand but can be reclaimed by the cloud at any time, usually with 30-60 seconds notice. For LoRA training jobs that complete in 1-4 hours, spot interruption probability is low (typically <5% per hour on Lambda/CoreWeave). For long jobs (8+ hours), interruption probability compounds and you should checkpoint frequently. With aggressive checkpointing (every 30 minutes), spot interruption is recoverable with minor cost overhead.

**On-demand pricing** is the standard rate with no interruption risk. The right choice for production training jobs where wall-clock time matters or for clusters > 2 GPUs where coordinating recovery from partial interruption is complex.

**Reserved or committed-use pricing** offers 20-50% discounts on multi-month commitments. The right choice for teams running continuous training workloads (model versioning, A/B testing, periodic retraining). Lambda Labs offers month-long reservations at meaningful discounts; CoreWeave 1Y commitments hit 30-40% off list.

**Marketplace pricing (Vast.ai)** is the cheapest at $1.80-2.40/hour for H100 but variable — quality and reliability depend on the individual host. Acceptable for experimental runs and low-stakes training; not recommended for production deployments where consistency matters.

Most LoRA jobs above 70B parameters benefit from multi-GPU scaling. Efficiency is not perfect — there is overhead from gradient communication between GPUs.

**2 GPU scaling** (DeepSpeed Zero-2 or FSDP) on Llama 4 70B LoRA achieves approximately 80-90% scaling efficiency. So 2x H100 produces roughly 1.6-1.8x the throughput of 1x H100.

**4 GPU scaling** achieves approximately 70-80% efficiency on a single node with NVLink (~3-3.2x speedup vs single GPU). Across multiple nodes via InfiniBand or 100GbE, efficiency drops to 50-70%.

**8 GPU scaling** on a single node (8x H100 with NVSwitch) achieves approximately 60-70% efficiency on LoRA (~5-5.5x speedup vs single GPU). Full fine-tuning scales better at 8 GPUs because gradient computation dominates communication; LoRA's much smaller gradient footprint makes the multi-GPU overhead relatively larger.

**The cost implication**: for LoRA on Llama 4 70B, the cheapest configuration is often 1-2 GPUs. Going to 4+ GPUs reduces wall-clock time but increases GPU-hours total because of scaling inefficiency. Pick GPU count by wall-clock requirement, not by minimizing dollar cost.

If wall-clock time is not critical and you want the lowest possible dollar cost, QLoRA on a single H100 80GB is the answer for models up to 70B parameters.

**Llama 4 70B QLoRA on 1x H100 80GB** completes a typical 22.5M-token job in 3-4 H100-hours at $8-15 spot cost. Compared to LoRA on 2x H100 ($15-20 spot), QLoRA on 1x H100 is ~50% cheaper.

**The trade-off**: QLoRA quality is typically 1-2 percentage points below full-precision LoRA on standard benchmarks because of the 4-bit quantization of the base model. For most production use cases, this gap is acceptable given the cost savings. For quality-critical workloads, run LoRA on 2 GPUs.

**Provisioning**: QLoRA on a single H100 80GB has ~30 GB of VRAM headroom after model + adapter + optimizer + activations, enough for batch size 2-4 at 8K context. For larger batch sizes or longer context, either move to 2 GPUs (full LoRA) or stay on QLoRA at smaller batch.

The cost difference between self-hosted H100 training and hosted platforms (Together, Fireworks, Replicate) is real but not large. The decision is more about engineering complexity than pure dollar cost.

**Self-host on rented H100s** when: you have or can build the training infrastructure (axolotl, accelerate, DeepSpeed setup); you are running 10+ training jobs/month and the per-job savings compound; you need configurations (DPO, ORPO, continued pre-training) that hosted platforms do not support; or you want full control over the training environment for reproducibility or compliance.

**Use a hosted platform** when: you are running 1-5 training jobs/month and engineering setup time exceeds the per-job savings; you need standard SFT or LoRA configurations that hosted platforms cover well; you do not have GPU access via cloud accounts; or you want a faster path from data to trained model (typical hosted-platform UX is 30-60 minutes from jsonl upload to deployed model).

**Cost crossover math**: typical hosted-platform markup is 30-50% over self-hosted spot. If you save $10/job on self-hosting and you run 20 jobs/month, that is $200/month in savings. If self-hosting adds 4 hours/month of engineer time at $100/hour engineering cost, the engineering cost ($400) exceeds the savings ($200) — hosted wins. For higher-volume training programs (50+ jobs/month), self-hosting starts to win on TCO.

If you decide to self-host, here is the practical setup that produces the cheapest reliable LoRA training runs in 2026.

**Cloud**: Lambda Labs spot or RunPod community spot for one-off jobs; CoreWeave or Lambda Labs reserved for continuous workloads. Avoid hyperscalers (AWS p5, GCP A3) unless you specifically need enterprise networking — they are 2-3x more expensive than specialized GPU clouds.

**Framework**: axolotl (https://github.com/OpenAccess-AI-Collective/axolotl) is the most popular and well-supported framework in 2026 for LoRA on Llama 4 and similar models. unsloth is faster (~30% throughput uplift on single GPU) but supports fewer models. LLaMA-Factory is a good alternative with broad model support and a config-driven workflow.

**Storage**: cloud-native storage (S3, GCS, R2) for training data and checkpoints — cheap and fast enough for typical LoRA workflows. Avoid local-disk-only setups where job failure loses checkpoints.

**Monitoring**: Weights & Biases (https://wandb.ai/) free tier covers most personal training; paid tiers for team workflows. wandb tracks loss curves, eval metrics, and GPU utilization — useful for debugging slow training jobs and validating that your throughput numbers match expectations.

**Checkpointing**: save adapter + optimizer state every 30 minutes for spot-resilient training. Keep at least the last 3 checkpoints for rollback. axolotl handles this with a few config flags.