Purchase-model guide
Spot or interruptible GPU capacity can be economically attractive when a workload can survive eviction, restart cleanly, and tolerate uncertain scheduling. On-demand capacity is usually the safer baseline when continuity, deadlines, or operational simplicity matter more than the lowest published hourly rate.
The economic question
A lower spot rate creates value only when the expected savings exceed the cost of interruptions and the extra engineering required to handle them. That cost can include lost progress since the last checkpoint, restart time, data reloads, queue delay, re-provisioning, operator attention, and the business cost of missing a deadline.
Low interruption cost
Stateless batch jobs, embarrassingly parallel work, checkpointed training, offline inference, rendering, or jobs with flexible completion windows can often absorb interruptions without turning the discount into hidden operational cost.
High interruption cost
Stateful services, latency-sensitive inference, tightly scheduled production jobs, poorly checkpointed training, or workloads with expensive warm-up and recovery can lose more from eviction than they save on the hourly rate.
Six-factor framework
A purchase model should be selected after the recovery behavior is known. If the workload cannot explain what happens when a GPU disappears, a spot price is not yet comparable with an on-demand price.
1 · Checkpointing
Estimate the work between durable checkpoints and the time required to write and restore them. A 30% rate discount can disappear quickly if each eviction destroys hours of expensive progress.
2 · Restart cost
Include environment startup, data staging, model loading, distributed rendezvous, cache warm-up, and operator intervention. Recovery cost belongs in the workload model even when the provider does not charge directly for every part of it.
3 · Deadline
Spot is more attractive when the finish time is flexible. If a training result, render, or batch output must be delivered by a fixed time, the value of predictable capacity rises even when the on-demand hourly rate is higher.
4 · Parallelism
Independent tasks can often be retried cheaply. A tightly synchronized multi-GPU or multi-node job may be more fragile because one interrupted worker can affect the whole group and increase wasted GPU-hours across nodes that were not themselves evicted.
5 · Capacity flexibility
A workload that can move across equivalent GPU types or regions has more ways to recover from thin spot supply. Hard requirements for a specific topology, region, memory size, or GPU variant reduce that flexibility.
6 · Operational maturity
Spot economics improve when retries, checkpoints, scheduling, monitoring, and fallback policies are already part of the platform. Building that machinery only for a small discount may cost more engineering time than it saves in compute.
Effective-cost model
The useful comparison holds the workload constant. Start with the same amount of useful work, then change the purchase model and add the expected interruption overhead. Do not compare a successful on-demand run with an idealized spot run that assumes zero evictions.
GPU hourly rate × GPU count × productive runtime.
Lost progress, failed workers, checkpoint interval, and any GPU time consumed during recovery.
Restart, re-provisioning, data loading, queue delay, and other time before useful work resumes.
Deadline risk, service availability, engineer time, and the cost of an unpredictable finish time when those factors are material.
Workload patterns
Often spot-friendly
Offline inference queues, rendering, independent simulation tasks, fault-tolerant batch pipelines, and checkpointed training can often make productive use of interruptible capacity when schedulers and retries are designed for it.
Often on-demand-friendly
Production serving, interactive systems, strict-SLA jobs, fragile distributed training, and workloads that require a specific scarce topology generally place more value on continuity and predictable scheduling.
Hybrid pattern
Some teams can keep critical baseline capacity on-demand while sending flexible queue depth or retryable jobs to spot. The right split depends on how quickly work can move and whether the application can tolerate mixed capacity pools.
Evidence boundary
Purchase model and availability remain separate. A provider can publish an interruptible rate while capacity is thin or unavailable in the region, GPU count, or topology you need. Check current evidence before converting a theoretical spot saving into a deployment plan.