Google Cloud urges fallback planning for AI workloads

Google Cloud has issued new guidance on dynamic capacity management for AI and enterprise workloads, centred on reserving resources in advance and automating fallback options when preferred infrastructure is unavailable.

The guidance forms part of a broader push to give customers more control over AI spending as demand grows for systems that support large numbers of software agents, data-intensive applications and specialist computing hardware.

Google Cloud recommends a two-pronged approach to infrastructure planning. First, organisations should secure resources for predictable demand, such as product launches, migrations, offline training runs or seasonal traffic peaks. Second, they should build automated responses for unexpected spikes in activity, including surges driven by news events or market changes.

At the centre of the approach is Dynamic Workload Scheduler, which Google Cloud presents as a way to reserve access to scarce computing resources, including GPUs, TPUs and some CPU-based virtual machine families. The system offers a calendar mode for time-bound events and a flex-start mode for workloads that can wait until capacity becomes available.

Calendar mode is intended for customers that need resources at a fixed time for a defined period. Flex-start mode is designed for latency-tolerant jobs such as batch processing, model training and offline fine-tuning, where work can be queued until hardware becomes available.

Fallback planning

A second part of the guidance focuses on reducing dependence on a single hardware configuration. Businesses are advised to create ranked lists of acceptable alternatives so applications can continue running if a preferred machine type, size or zone is unavailable.

For Compute Engine users, that means using instance flexibility in managed instance groups and bulk virtual machine creation. Instead of tying workloads to one machine type, operators can define several compatible options and let the system choose among them based on real-time availability.

This can also support gradual hardware refresh cycles by allowing newer virtual machine generations to take priority while older systems remain available as backups. Using multiple zones within a region can also improve the chances of successful provisioning.

Spot virtual machines are part of the strategy as well. Compute Engine can use Spot capacity signals and favour machine types expected to offer longer uptime with lower risk of pre-emption.

Storage is included in the fallback strategy. Customers can rely on baseline disk defaults for many short-lived instances, while using Hyperdisk for data disks that need to outlive associated virtual machines across different machine generations.

Kubernetes focus

Google Cloud also gave particular attention to Google Kubernetes Engine as the layer that can automate these decisions from what it describes as a single control plane. Through custom ComputeClasses, platform teams can define combinations of machine families, sizes, performance ratios, pricing models and zones, then assign priorities among them.

When a preferred node configuration is unavailable, the system can move to the next approved option. If active migration is enabled, workloads can later shift back to higher-priority configurations when capacity returns.

Google Cloud also highlighted dynamic resource allocation in Google Kubernetes Engine, which allows applications to request part of a GPU or TPU rather than an entire device. In practice, developers can specify needs such as memory or core counts, with the platform assigning a matching slice of hardware.

The aim is to address a long-running problem in AI infrastructure: expensive accelerators often sit underused because they are allocated in large blocks rather than in portions that more closely match application requirements. By aligning consumption more closely with need, operators may be able to increase utilisation and reduce idle spend.

Pressure on infrastructure

The recommendations reflect the strain AI workloads are placing on infrastructure teams. These workloads are often uneven in demand, resource-intensive and dependent on specialist hardware that can be difficult to obtain at short notice.

This creates pressure at several layers of the stack, especially as companies try to run both AI systems and conventional business applications on the same underlying estate. Google Cloud argues that hardware alone is not enough and that stronger operational controls are needed to avoid overprovisioning, stranded capacity and bottlenecks.

It also pointed to a gap between AI deployment ambitions and confidence in current infrastructure readiness. Many enterprises want to deploy software agents over the next few years, yet far fewer technology leaders believe their existing environments can cope with the resulting demand.

In response, Google Cloud advises customers to review workloads that are tightly tied to one virtual machine family, machine type or availability zone. It also urges them to map alternative hardware options, use automated fallback policies through managed instance groups or custom ComputeClasses, and consider committed spending models that offer discounts while preserving flexibility across machine types and locations.

The guidance underlines how cloud providers are shifting from simply offering access to computing resources to helping customers manage scarcity, volatility and cost in AI-heavy environments, particularly where accelerators and containerised workloads have become central to day-to-day operations.