Tech
Lambda Gets 24% More AI Tokens From Smarter Power Management
Lambda’s NVIDIA validation delivered 24% more token throughput within the same fixed power budget.
What happened
Cloud provider Lambda tested NVIDIA’s DSX MaxLPS software on a five-rack, 19-node cluster using HGX B200 GPU servers. By running 19 nodes within the power budget previously used by 16 nodes at full power, Lambda increased cluster-wide token throughput from roughly 4 million to 5 million tokens per second. Performance per watt improved 23%. Separately, Emerald AI’s Conductor platform reduced an AI factory’s power demand from four megawatts to three when Silicon Valley Power sent a grid signal, while higher-priority inference continued running. The utility has sent more than 200 such signals, and the system responded successfully each time.
Why it matters
The results show that software-based power allocation can increase AI compute output without expanding the facility’s fixed power budget. For data-center operators, that means more usable capacity from existing infrastructure and improved performance per watt.
Source: NVIDIA Blog