MPSTAT

Another useful command to get CPU related stats is mpstat. Here is an example output:

# mpstat -P ALL 5 2
Linux 2.6.9-67.ELsmp (oraclerac1)       12/20/2008
 
10:42:38 PM  CPU   %user   %nice %system %iowait    %irq   %soft   %idle    intr/s
10:42:43 PM  all    6.89    0.00   44.76    0.10    0.10    0.10   48.05   1121.60
10:42:43 PM    0    9.20    0.00   49.00    0.00    0.00    0.20   41.60    413.00
10:42:43 PM    1    4.60    0.00   40.60    0.00    0.20    0.20   54.60    708.40
 
10:42:43 PM  CPU   %user   %nice %system %iowait    %irq   %soft   %idle    intr/s
10:42:48 PM  all    7.60    0.00   45.30    0.30    0.00    0.10   46.70   1195.01
10:42:48 PM    0    4.19    0.00    2.20    0.40    0.00    0.00   93.21   1034.53
10:42:48 PM    1   10.78    0.00   88.22    0.40    0.00    0.00    0.20    160.48
 
Average:     CPU   %user   %nice %system %iowait    %irq   %soft   %idle    intr/s
Average:     all    7.25    0.00   45.03    0.20    0.05    0.10   47.38   1158.34
Average:       0    6.69    0.00   25.57    0.20    0.00    0.10   67.43    724.08
Average:       1    7.69    0.00   64.44    0.20    0.10    0.10   27.37    434.17

It shows the various stats for the CPUs in the system. The –P ALL options directs the command to display stats for all the CPUs, not just a specific one. The parameters 5 2 directs the command to run every 5 seconds and for 2 times. The above output shows the metrics for all the CPUs first (aggregated) and for each CPU individually. Finally, the average for all the CPUs has been shown at the end.

Let’s see what the column values mean:

 

%user
 Indicates the percentage of the processing for that CPU consumes by user processes. User processes are non-kernel processes used for applications such as an Oracle database. In this example output, the user CPU %age is very little.
 

%nice
 Indicates the percentage of CPU when a process was downgraded by nice command. The command nice has been described in an earlier installment. It brief, the command nice changes the priority of a process.
 
%system
 Indicates the CPU percentage consumed by kernel processes
 
%iowait
 Shows the percentage of CPU time consumed by waiting for an I/O to occur
 
%irq
 Indicates the %age of CPU used to handle system interrupts
 
%soft
 Indicates %age consumed for software interrupts
 
%idle
 Shows the idle time of the CPU
 
%intr/s
 Shows the total number of interrupts received by the CPU per second
 

You may be wondering about the purpose of the mpstat command when you have vmstat, described earlier.

There is a huge difference:

mpstat can show the per processor stats, whereas vmstat shows a consolidated view of all processors.

So, it’s possible that a poorly written application not using multi-threaded architecture runs on a multi-processor machine but does not use all the processors. As a result, one CPU overloads while others remain free. You can easily diagnose these sorts of issues via mpstat.

 

Usage for Oracle Users
Similar to vmstat, the mpstat command also produces CPU related stats so all the discussion related to CPU issues applies to mpstat as well. When you see a low %idle figure, you know you have CPU starvation. When you see a higher %iowait figure, you know there is some issue with the I/O subsystem under the current load. This information comes in very handy in troubleshooting Oracle database performance.

 
(Extracted from oracle technet notes author Arup Nanda)