The data center industry spent decades optimizing for one set of constraints. AI has changed almost all of them.
When companies first started building out data centers, the unit of measure was the rack. A metal cabinet, roughly the height of a refrigerator, holding servers and networking gear in a standardized format. For most of the industry’s history, a rack drawing 10 kilowatts was considered fairly dense. Many drew considerably less. Facilities were designed around those expectations: cooling systems, power distribution, and floor ratings all calibrated to that range.
AI infrastructure has broken that frame entirely. The 100-kilowatt rack is not on the horizon. It has already been passed. Current AI rack-scale systems operate at roughly 120 to 130 kilowatts per rack, next-generation systems are specified at approximately 190 to 230 kilowatts, and published hardware roadmaps for 2027 point toward 600-kilowatt rack designs. That is not an incremental change. It is a fundamental shift in what a data center has to be able to do, and many existing facilities are not built to handle it.
Why AI Workloads Draw So Much Power
The chips that power artificial intelligence are designed to perform enormous numbers of calculations simultaneously. That parallel computing capability is exactly what makes them useful for AI, and it is also what makes them draw so much power.
A conventional server running business software might draw a few hundred watts. A server built around high-end AI accelerators can draw several thousand watts for the chips alone, before accounting for memory, storage, networking, and power conversion. Pack eight of those servers into a rack and the numbers climb fast.
Unlike general computing workloads that fluctuate based on user activity, AI training and inference at scale runs chips at or near full utilization for sustained periods. The peak draw is closer to the average draw than in traditional environments. That sustained demand is what pushes rack power into territory that conventional facilities were never designed for.
Why Air Cooling Stops Working
Heat is the direct consequence of high power draw. Every watt that flows into a rack eventually becomes heat that has to be removed. Traditional data centers handle this by pushing cold air through the facility: hot air rises, gets captured, gets cooled, and recirculates.
This works well at densities of 10 to 15 kilowatts per rack. As density climbs above that threshold, the physics begin to work against you. Air does not carry heat efficiently. Moving enough cold air to keep a 60-kilowatt rack within safe operating temperatures requires fans running at high speed, which creates noise, consumes significant electricity, and still struggles to cool precisely where the hottest components are. At 100 kilowatts and beyond, air cooling is not a difficult engineering challenge. It is simply insufficient.
Liquid carries heat roughly 3,500 times more efficiently than air of equivalent volume. That is why the industry has moved decisively toward liquid cooling as AI infrastructure densities have increased, and why the latest generation of rack-scale AI systems is designed exclusively for liquid cooling, with no air-cooled configuration offered at all.
How Liquid Cooling Works
There are two main approaches, and they address the heat problem in different ways.
Direct-to-chip cooling uses metal plates placed in direct contact with the hottest components inside a server. Cooled water circulates through those plates, absorbs heat at the source, and carries it to a heat exchanger outside the facility. This approach can be integrated into existing facilities with less disruption than full immersion, and it targets the primary heat sources directly.
Immersion cooling takes a more comprehensive approach. Servers are submerged entirely in tanks filled with a non-conductive liquid, either a mineral oil-based fluid or a synthetic engineered coolant. The liquid surrounds every component uniformly, making it extremely effective at managing very high heat densities. It also requires specialized tanks, compatible hardware, and meaningful changes to how teams operate and maintain equipment.
Both approaches require infrastructure that most standard facilities do not include: piping systems, heat exchangers, pumps, fluid management, and in the case of immersion, the tanks themselves. Retrofitting these systems into a facility that was not designed for them is possible, but it is expensive and disruptive.
The Operational Changes That Come With High-Density AI
Cooling is the most visible infrastructure change that high-density AI racks require, but it is not the only one.
Power distribution systems need to be redesigned for higher density. Delivering 100 kilowatts or more to a single rack requires thicker cables, higher-capacity circuit breakers, and power distribution units rated for the load. Facilities originally built for 10-kilowatt racks often lack the electrical infrastructure to support even a fraction of their rack count running at ten times that density.
Floor loading is a physical constraint that gets overlooked until it becomes a problem. High-density AI racks with associated liquid cooling infrastructure are significantly heavier than conventional server equipment. Data center floors have load ratings set for the equipment they were originally designed to hold. Exceeding those ratings is a structural risk, not just an operational one.
Maintenance procedures change substantially when liquid cooling is involved. Accessing components connected to a liquid cooling loop requires different procedures than working with air-cooled equipment. Immersion cooling adds further complexity: removing and replacing hardware that has been submerged in fluid requires training and tooling that most standard data center teams do not currently have.
What This Means for Infrastructure Planning
The era of the 100-kilowatt rack is not approaching. It has arrived, and the trajectory beyond it is already defined. The hardware operating at those densities is deployed today, with more capable and more power-hungry generations already in production.
Infrastructure decisions made today need to account for density requirements that may not apply to current deployments but will apply to future ones. A colocation facility that cannot support liquid cooling and high-density power distribution is not just a constraint for today’s AI hardware. It is a constraint on everything that follows over the next two to three years.
The question worth asking before committing to any AI infrastructure environment is not just whether it handles current requirements. It is whether the physical design and operational capabilities are built for where AI hardware density is heading, not just where it is today.
This is the standard worth designing against: power, cooling, and floor infrastructure specified for the densities AI hardware is moving toward, not the ones it has already left behind. Facilities built to that standard give organizations room to grow into the next hardware generation. Facilities built to yesterday’s standard become the bottleneck.
