Measuring CPU Power Consumption

If you search online for keywords such as “Measure CPU power consumption in Linux,” you may end up in the same position as me – It is quite easy to acquire CPU temperature and core voltage, but not that easy to get a power reading from the CPU, whether package power or core power. Most answers will just tell you it is not possible; instead, you should use a Kill-A-Watt and measure the entire system.

However, we all know that software like HWiNFO has some way of measuring CPU wattage, sometimes both package power and per-core power. How do we achieve this? It turned out that many processors support measuring core power through Model Specific Registers (MSRs,) but they were exposed to the system in different ways.

This article summarizes the programmatic ways to read core and package wattage. That is, we look for ways that a program can read core or package wattage through something like a function call or a POSIX transaction.

Windows

It turns out that many tools are developed for Windows that we can immediately take advantage of.

General

If you are using Windows, HWiNFO SDK might be the best answer for you. It packaged all HWiNFO functionalities into a dll file, which you can call.

AMD Processors

AMD provides the Ryzen Master SDK, which provides ways to read current and power. It is only available for Windows, though.

Intel Processors

Intel provided Intel Power Gadget to read CPU power. However, it only has a GUI interface, and there is no programmatic or CLI access.

Linux / Other OS / No OS

Linux has fewer out-of-the-box tools for reading CPU power, and we need to dig deeper into the system.

RAPL (Intel)

RAPL stands for Running Average Power Limit, a term used by Intel (AMD has similar concepts under a different name.) It is described in Intel 64 and IA-32 Architectures Software Developer’s Manual, Volume 3, Section 15.10.

This system enforces power limits on the CPU, and it has an Energy Status interface, which includes a counter that accumulates power consumed and stores it into MSR_PKG_ENERGY_STATUS. The unit of this MSR is stored in MSR_RAPL_POWER_UNIT[12:8] (denoted ESU). Each increment of MSR_PKG_ENERGY_STATUS represents 0.5^ESU Joules. The default value of ESU is 10000b (=16), which means one increment of MSR_PKG_ENERGY_STATUS is 15.26mJ. This MSR is read-only, so there is no way of tuning the precision of this measurement. MSR_PKG_ENERGY_STATUS has a length of 32 bits and updates about every millisecond.

This code shows how to use these counters: https://github.com/tfett/RAPL

RAPL? (AMD, official)

AMD is widely known for having terrible documentation, performance counters, and analysis tools compared to Intel. Measuring power is no exception.

An official way of measuring energy (thus power) is documented in AMD64 Architecture Programmer’s Manual, Volume 2: System Programming, Section 17.5, Processor Core Power Reporting.

Some AMD processors support Processor Power Reporting, which can be checked with CPUID Fn8000_0007_EDX[ProcPowerReporting] = 1. This flag can be checked with Linux cpuid utility.

On supported CPUs, there is a counter called CpuSwPwrAcc, with a maximum recordable value of MaxCpuSwPwrAcc before it wraps to 0. The documentation did not provide its MSR Address, nor is it listed in the MSR Appendix. Its specific address might be revealed with cpuid, but my CPU (7950X) does not support this feature from the CPUID report.

The unit of this counter is correlated with clock ticks and CpuPwrSampleTimeRatio, stored at CPUID Fn8000_0007_ECX[31:0]. The algorithm is documented in Section 17.5.2 as follows:

• Determine the value of the ratio of Tsample to the TSC period (CpuPwrSampleTimeRatio) by executing CPUID Fn8000_0007. Call this value N.
  N = CPUID Fn8000_0007_ECX[31:0].
• Read the full range of the cumulative energy value from the register MaxCpuSwPwrAcc.
  Jmax = value returned from RDMSR MaxCpuSwPwrAcc.
• At time x, read CpuSwPwrAcc and the TSC
  Jx = value returned by RDMSR CpuSwPwrAcc
  Tx = value returned by RDTSC
• At time y, read CpuSwPwrAcc and the TSC again
  Jy = value returned by RDMSR CpuSwPwrAcc
  Ty = value returned by RDTSC
• Calculate the average power consumption for the processor core over the measurement interval
  TM =(Ty – Tx).
• If (Jy < Jx), rollover has occurred; set Jdelta = (Jy + Jmax) – Jx; else Jdelta = Jy – Jx
  Note: This does not account for multiple rollovers. x and y must not be too far apart.
• PwrCPUavg = N * Jdelta / (Ty – Tx)

The final value PwrCPUavg is in milliWatts.

RAPL (AMD, unofficial)

This GitHub repository shows that AMD has some MSRs, similar to Intel ones, that function as an energy counter: https://github.com/djselbeck/rapl-read-ryzen

It shows that AMD processors potentially have the following MSRs:

#define AMD_MSR_PWR_UNIT 0xC0010299
#define AMD_MSR_CORE_ENERGY 0xC001029A
#define AMD_MSR_PACKAGE_ENERGY 0xC001029B

And the AMD_MSR_PWR_UNIT MSR has the following bit layout:

#define AMD_TIME_UNIT_MASK 0xF0000
#define AMD_ENERGY_UNIT_MASK 0x1F00
#define AMD_POWER_UNIT_MASK 0xF

All these units are also represented in “power of 0.5,” similar to Intel.

These MSRs do not even exist in the AMD documentation, but they work quite well. See the GitHub repo for details.

General

Linux perf tool sometimes provides power measurement through RAPL. For example, the command below measures the total system power consumption during the execution of the workload. Notice that even if we had the workload specified, the tool would not isolate the power consumption of that specific process.

$ sudo perf stat -e power/energy-pkg/ /usr/bin/stress --cpu 4 --timeout 10s
stress: info: [2435282] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
stress: info: [2435282] successful run completed in 10s

 Performance counter stats for 'system wide':

            493.55 Joules power/energy-pkg/

      10.002243108 seconds time elapsed

This is also less of a programmatic way because we are not getting real-time readings.

Intel

A tool called turbostat, said to be developed by Intel engineers, uses the aforementioned MSRs to monitor CPU turboing status.