iostat I/O Statistics

What it is

iostat is a system monitoring tool that reports CPU statistics and input/output statistics for devices and partitions. You reach for it when you suspect disk I/O is a bottleneck in your system performance.

Installation

Linux: iostat is typically part of the sysstat package.

# Debian/Ubuntu
sudo apt update
sudo apt install sysstat

# Fedora/CentOS/RHEL
sudo dnf install sysstat
# or
sudo yum install sysstat

macOS: iostat is a built-in Unix command and is available by default.

Windows: iostat is not a native Windows command. For similar functionality, consider using perfmon (Performance Monitor) or third-party tools like Resource Monitor.

Core Concepts

• Device: Refers to a physical disk (e.g., sda, nvme0n1) or a logical partition on a disk (e.g., sda1, nvme0n1p1).
• CPU Utilization: Reports on how the CPU is being used, broken down by user, system, idle, etc.
• I/O Statistics: Measures the rate of read/write operations, data transfer, and wait times for devices.
  • r/s, w/s: Reads/writes per second.
  • rkB/s, wkB/s: Kilobytes read/written per second.
  • await: The average time (in milliseconds) for I/O requests issued to the device to be served. This includes the time spent by the requests in queue and the time spent servicing them.
  • svctm: The average service time (in milliseconds) for I/O requests issued to the device. (Note: svctm is often considered unreliable and may not be present or accurate on all systems, especially modern Linux kernels. await is generally preferred.)
  • %util: Percentage of CPU time during which I/O requests were issued to the device (bandwidth utilization for the device). When this value is close to 100%, the device is saturated.

Commands / Usage

Basic Usage: Display CPU and device utilization.

iostat

Output shows CPU utilization since the last reboot.

Interval Reporting: Report statistics every 2 seconds, 5 times.

iostat 2 5

The first line shows statistics since boot, subsequent lines show statistics for the preceding 2-second interval.

Continuous Reporting: Report statistics every 2 seconds indefinitely.

iostat 2

Press Ctrl+C to stop.

Showing All Devices: Display statistics for all block devices, including loop devices.

iostat -a

Showing Disk Statistics: Display I/O statistics for all disks.

iostat -d

Showing Extended Disk Statistics: Display extended disk statistics, including %util, await, svctm.

iostat -dx

Showing Per-CPU Statistics: Display CPU utilization broken down by each CPU core.

iostat -p

Showing Per-CPU Extended Statistics: Display extended CPU utilization broken down by each CPU core.

iostat -px

Reporting on a Specific Device: Display statistics for the sda device every 2 seconds.

iostat -d /dev/sda 2

Reporting on Specific Devices: Display statistics for sda and sdb every 5 seconds.

iostat -d /dev/sda /dev/sdb 5

Showing File System Statistics (Linux): Display statistics for mounted file systems.

iostat -N

Showing Network File System Statistics (Linux): Display statistics for network file systems.

iostat -n

Showing Kernel I/O Statistics (Linux): Display kernel I/O statistics.

iostat -k

This is often the default, reporting in KB.

Showing Megabyte Block Sizes: Report statistics in Megabytes instead of Kilobytes.

iostat -m

Showing Device Utilization (Linux): Display device utilization, similar to -dx but focusing on utilization metrics.

iostat -x

Showing Timestamp: Prefix each output line with the current time.

iostat -t

Customizing Output Fields: Display only r/s, w/s, and %util for sda every 3 seconds.

iostat -d -x sda 3 'r/s' 'w/s' '%util'

(Note: Custom field selection is less common and can be complex. The -x flag usually provides sufficient detail.)

Combining Flags: Report extended disk statistics for all devices every 5 seconds, showing timestamps.

iostat -xt 5

Common Patterns

Identifying Disk Bottlenecks: Run iostat -xt 5 and watch the %util column. If it’s consistently near 100% for a specific device, that device is likely a bottleneck. Also, observe await – high values indicate requests are waiting a long time.

iostat -xt 5

Monitoring a Specific Disk: Focus on the nvme0n1 drive while it’s under heavy load.

iostat -xt /dev/nvme0n1 2

Comparing CPU and Disk I/O: Run iostat -p 2 to see if high CPU usage correlates with high disk I/O, or if the CPU is mostly idle while disks are busy.

iostat -p 2

Logging I/O Statistics: Save iostat output to a file for later analysis.

iostat -xt 10 > iostat_log.txt

Press Ctrl+C to stop logging.

Checking I/O after a specific event: Start iostat logging, perform an action (e.g., database import), then stop logging.

iostat -xt 1 > iostat_before_action.log &
# Perform action
pkill iostat
iostat -xt 1 > iostat_after_action.log

Gotchas

• svctm Unreliability: As mentioned, the svctm (service time) metric is often inaccurate on modern Linux systems and should generally be disregarded in favor of await.
• %util Interpretation: 100% utilization doesn’t always mean a problem, especially for SSDs where latency might still be low. However, consistently high %util on HDDs is a strong indicator of saturation. High await is a more universal indicator of I/O performance issues.
• Reporting Intervals: When using intervals, the first line of output represents statistics since the last reboot. Subsequent lines represent the statistics for the preceding interval.
• Device Naming: Device names (e.g., sda, nvme0n1) can vary. Use lsblk or fdisk -l to identify your specific devices.
• Permissions: On some systems, you might need root privileges (sudo) to get detailed statistics, especially for specific devices.
• sysstat Service: On Linux, the sysstat package often includes a service (sysstat or sadc) that collects historical data. iostat can read from this historical data if configured, but the commands above typically query real-time kernel data.