Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 

Utilities to Measure the Performance of an Existing System

 

This section describes the following utilities available in the Solaris 10 operating system used to measure and view the performance of an existing system:

prstat

prstat

The prstat or top utility shows the current active processes information on the system and reports statistics based on the selected output mode and sort order. The following is a sample output with no arguments for an SBRC system under load for reference:

The command on Linux is similar:

Table 9 describes the key fields of the prstat utility output.

Table 9: prstat Output Description

Field

Description

CPU

Shows the percentage of in-use CPU across the entire machine. The CPU time is regressed over the number of seconds, so this value does not show spike utilization, but is averaged over time.

SIZE

Shows the total working-set virtual memory size of the process (on some versions of Solaris, this reads as SWAP). In general, this should be less than the system memory size, and for the SBRC front-end application, as a 32-bit application, must be less than 4G.

RSS

The resident set size (RSS) shows the amount of memory that is paged in. Some paging out of SBRC is tolerable, but if this value is less than the working-set size, check for other parts of the system processes using too much memory.

NLWP

Number of light-weight processes (or threads).

TIME

Shows the amount of CPU time that the process has been executing since the start.

STATE

The information is not very useful in a multithreaded application in the thread roll-up display, but in prstat –L, it will show the state of the thread.

PRI

The information is not very useful in a multithreaded application in the thread roll-up display, but in prstat –L, it will show the thread priority. See Thread Priority for more information.

The following is an output of prstat –L showing per-thread utilization:

Size and RSS refer to the process and not to the specific thread. STATE, CPU and Time refer to the specific thread.

Note

The maximum CPU for a given thread is equal to 100 divided by the number of virtual CPUs active on the system. In this case, since SBRC runs on an M3000 with 8 virtual CPUs enabled, the maximum would be 12.5 percent. Since all CPU utilization is less than 12.5 percent and evenly distributed across multiple threads, you will get more than 90 percent of the total CPU on the system. In this case, SBRC works well under the load.

For more general usage information on the prstat utility, see the man page at http://download.oracle.com/docs/cd/E19253-01/816-5166/6mbb1kqdk/index.html.

On Linux, the top command accepts the uppercase “H” subcommand to switch between per-process (the default) and per-thread reporting:

Unlike Solaris, the default %CPU on Linux totals to 100% times the number of processors. Thus, 100% utilization for a thread corresponds to one virtual CPU. It does not appear to be the case that 100% CPU utilization is realistic on a hyper threaded system. Further, in this calculation, %System time is not accounted for.

It is safe to assume, given that the highest CPU utilization threads for SBR are generally those involved with network I/O (which takes the bulk of the %SY time), that if the highest CPU utilization thread plus the overall %Sys time approximates to 100%, then that thread is at its maximum.

sar

sar

This is a system activity reporter utility. This Solaris command shows the overall activity of the system. The following is a sample output:

# sar 1 10 [every 1 second, display the results 10 times]

For more general usage information on the sar utility, see the man page at http://download.oracle.com/docs/cd/E19253-01/816-5166/6mbb1kqgf/index.html.

The following is a sample output:

# sar –A 1 10 [every 1 second display all the results 10 times]

The sar utility runs in the background to display metrics on the system, with an interval of every few minutes on a nominal system, or every 5 or 10 seconds on a system that is under examination or undergoing troubleshooting to help diagnose system anomalies that impact SBRC (for example, to identify periods of spikiness).

iostat

iostat

The iostat utility reports I/O statistics such as terminal, disk, and tape I/O activity, as well as CPU utilization. The following is a sample output:

# iostat 1 10

If the CPU utilization from prstat is relatively low but the I/O utilization (kps on any device) is nearing the I/O bandwidth utilization, you may have a problem with I/O.

For more general usage information on the iostat utility, see the man page at http://download.oracle.com/docs/cd/E19253-01/816-5166/6mbb1kq4q/index.html.

vmstat

vmstat

The vmstat utility shows virtual memory usage statistics regarding kernel thread, virtual memory, disk, trap, and CPU activity. The following is a sample output:

# vmstat 1 10

The vmstat utility is useful to determine if the memory utilization is too high. For example, any high pi/po values indicate memory thrashing (that is, more memory is in use by the working set than is available on the system), and you should diagnose for stale sessions, leaks, or add more memory.

For more general usage information on the vmstat utility, see the man page at http://download.oracle.com/docs/cd/E19253-01/816-5166/6mbb1kqms/index.html.

prtdiag and sosreport

prtdiag and sosreport

The prtdiag and sosreport utilities displays system configuration and diagnostic information. The following is a sample output:

# prtdiag

For more general usage information on the prtdiag utility, see the man page at http://download.oracle.com/docs/cd/E19253-01/816-5166/6mbb1kqdn/index.html.

An equivalent of prtdiag on Linux is a sosreport (which may require an additional package, which can be installed with “yum install sosreport”). The sosreport generates a package of files in /tmp, which can be inspected or forwarded to JTAC.

sosreport

The report can be viewed as follows:

# xz -d sosreport-juser.12345-20120312121724-8658.tar.xz
# tar xf sosreport-juser.12345-20120312121724-8658.tar
# cd %ltmachinename%gt-%ltdatetime%gt; ls

The preceding generated trees may be inspected freely and remotely, and are very valuable for debugging issues.

psrinfo

psrinfo

The psrinfo utility displays information about virtual processors. The following is an example output:

# psrinfo

For more general usage information on the psrinfo utility, see the man page at http://download.oracle.com/docs/cd/E19253-01/816-5166/6mbb1kqdt/index.html.

psradm

psradm

The psradm utility changes the status of the virtual processors. The following is a sample output:

# psradm -f 1
# psrinfo
# psradm -n 1
# psrinfo

For more general usage information on the psradm utility, see the man page at http://download.oracle.com/docs/cd/E19253-01/816-5166/6mbb1kqds/index.html.

cpuinfo on Linux

cpuinfo on Linux

On Linux, the equivalent of psrinfo is /proc/cpuinfo, which returns information like:

# more /proc/cpuinfo

Please see the man page for “proc” for more information.

Summary information can be seen via the lscpu command.

# lscpu

Please see the man page for lscpu for more information.

Taskset on Linux

Taskset on Linux

Taskset is used to set or retrieve the CPU affinity of a running process, given a PID or a command. This could be used to increase the performance in certain instances.

taskset –p –c <cpuset> <PID>

Example:

Where –p option indicates the last argument is a PID, the –c option indicates the second-to-last argument is a list of CPU IDs (rather than, the default, a bitmap representing which CPUs). The effect of this command would be to make process 20320 bias to the first 8 cores.

Disabling a CPU on Linux

Disabling a CPU on Linux

To temporarily disable a CPU on Linux, use the following command:

# echo 0 >> /sys/devices/system/cpu/cpu[#]/online

Where [#] is the processor ID of the core to disable. It appears that disabling CPU 0 is not permitted. To enable, echo the value 1. It does not appear to persist between reboots.

System Statistics

System Statistics

You use the Statistics page in Web GUI to display summary statistics for authentication, accounting, and proxy forwarding transactions. You can measure the current, average, and peak TPS values of authentication and accounting transactions.

Authentication statistics summarize the number of authentication acceptances and rejections, with summary totals for each type of rejection or retry (see Figure 2).

Figure 1: Viewing Authentication Statistics
Viewing
Authentication Statistics
Figure 2: Viewing Authentication Statistics
Viewing
Authentication Statistics

Accounting statistics provide information such as the number of transaction starts and stops and the reasons for rejecting attempted transactions. The start and stop numbers rarely match, because many transactions can be in progress at any given time (see Figure 4).

Figure 3: Viewing Accounting Statistics
Viewing Accounting Statistics
Figure 4: Viewing Accounting Statistics
Viewing
Accounting Statistics