By IT Brew Staff
less than 3 min read
Definition:
Capacity planning is critical for a business to meet current and future demand—and having the required resources to stay up and running.
For IT professionals, though, the process gets complicated when their business requires an emerging technology like AI—and the unpredictable, compute-intensive hardware essentials to run it effectively.
Traditional capacity planning—for example, at a manufacturing plant—may call for business leaders to calculate if they have enough machinery, employees, and operating time to get their current and prospective work done. The process frequently involves phases like a self-assessment of resources, forecasting of needs, and identification of constraints.
IT pros have to plan for capacity, too, and understand the organization’s required infrastructure and how its services and components (like networks, servers, databases, and hardware) can handle expected and unexpected workloads.
The cloud put a different spin on capacity planning, as it gave companies the option to add data-center and server resources—including storage and compute—quickly and on-demand. Companies, in effect, handed much of the capacity problem to cloud providers and focused less on their own hosted hardware, like data centers and servers.
With AI pushing compute demand across industries, capacity planning has become more complex again:
- Training and serving large language models (LLMs) require fast-calculating graphics processing units (GPUs) that can prove expensive to source.
- Organizations may have to decide on which jobs require a sophisticated, costlier model, and which ones require cheaper, basic LLMs.
- Migrating data sets into cloud environments for AI processing has its costs and requires ample storage.
- Overprovisioning wastes money, and AI tool usage has led to unpredictable token usage and costs.
Capacity planning now includes understanding AI adoption patterns and asking new questions like: Do we have the necessary resources to integrate automation into our workflows, including the right models and the most cost-effective use of tokens?
