The Road to Data Center Power Efficiency

Single threaded performance has saturated and the generational improvement in frequency from process technology is far less than what is required to keep the pace with the data centers need for performance. To compensate for that, the industry has moved to heterogenous integration of general-purpose CPUs, memory, workload specific accelerators such as GPUs, and other devices on the same package. In parallel, the client world, over the last 20 years, has resorted to clever power management schemes to improve the power efficiency of devices such as phones, tablets, and laptops. Many of the client techniques were developed for single client products, such as a CPU in a laptop or a phone. In data centers however, the situation is more complex, and it is very typical for multiple clients to be running workloads on the same product, such as a multicore CPU or GPU. As has been demonstrated with AMD’s pioneering chiplet products, the move to chiplet technology significantly improves cost of compute for the data centers but can increase power due to the addition of compute chiplet-to-chiplet interconnects. In this presentation, we combine the learnings from both, the client, and the data center worlds. To this end, we review the energy efficiency progress of the mobile world over the last 20 years and show how the chiplet technology can be used to leverage many of the techniques, developed for the single client mobile world, to improve the overall power efficiency of the data center.