GPU使用状态监控

Published on Aug. 22, 2023, 12:09 p.m.

GPU usage monitoring (CUDA)

For Nvidia GPUs there is a tool nvidia-smi that can show memory usage, GPU utilization and temperature of GPU. 0.5是刷新间隔

watch -n 0.5 nvidia-smi

结果如下


Sun May 13 20:02:49 2012 
++ 
| NVIDIA-SMI 3.295.40 Driver Version: 295.40 | 
|-+-+-+ | Nb. Name | Bus Id Disp. | Volatile ECC SB / DB | | Fan Temp Power Usage /Cap | Memory Usage | GPU Util. Compute M. | |===============================+======================+======================| | 0. GeForce 9600 GT | 0000:01:00.0 N/A | N/A N/A | | 0% 51 C N/A N/A / N/A | 90% 459MB / 511MB | N/A Default | |-+-+-| | Compute processes: GPU Memory | | GPU PID Process name Usage | |=============================================================================| | 0. Not Supported | +--+