In the second part of this series, we are going to target 3 different types of potential bottlenecks that a system might suffer from. Specifically these are:
- IO Bound
- CPU Bound
- Memory Bound
Most performance problems can be placed into one of these categories. By using what we learned in Part 1, we can use the information from vmstat to provide more information about what aspect of a system is limiting it from handling a greater load….and therefore what should be upgraded to provided greater capacity.
IO Bound System
An IO bound system is a machine that is usually suffering from performance limitations due to disk speed. While there are other types of IO a server could be waiting on, usually it is disk and disk speed that are the problems. This can be culled from reviewing vmstat by looking at both the uninterruptable sleep column (b column) and a high value in the wa column. An example from vmstate would look as follows:
#vmstat 5
procs ———–memory———- —swap– —–io—- –system– —–cpu——
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 8 5888 140136 167700 1506440 0 0 6783 17333 1738 16648 6 2 9 83 0
1 6 5888 140136 167700 1506440 0 0 6885 18353 1831 18658 8 4 9 77 0
1 6 5888 140136 167700 1506440 0 0 6482 12323 1534 17632 16 12 9 63 0
In the above, you can clearly see both issues at work as predicted above. The wa is quite high 63-83 and the number of processes in uninterpretable sleep is between 6 and 8.
Memory Bound – Swapping
The second type of system we will look at is a system that is attempting to allocate more memory than physically exists in the system. It is normal for a linux/unix system to use swap to augment its available memory. As such, any system can show some amount of memory allocated to swap. However, this only becomes a problem when the system starts waiting for memory to be paged in and out of swap to satisfy its processes. From a vmstat perspective, this would appear in 2 ways. First is the swpd column. However, the presence of large values in the swpd column does not, in and of itself, constitute swapping. The more important demonstration of this is high values in si and so columns. Here is a sample vmstat that shows some swapping.
#vmstat 5
procs ———–memory———- —swap– —–io—- –system– —–cpu——
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 18 3794291 140136 167700 1506440 19851 9780 16783 27333 1738 1648 6 2 29 63 0
1 16 3797936 140136 167700 1506440 15913 30807 38353 28331 1658 3342 8 14 19 57 0
1 10 3847364 140136 167700 1506440 18334 22355 26482 32323 1534 1732 16 12 29 33 0
CPU Bound Machine
The last type of system we will review is a CPU-Bound machine. A server that is experiencing this problem simply does not have enough processing power to properly perform its operations. Usually this system will show a high number in the us column. Again, remember the us column shows us the amount of time spent processing non-kernel code. It is also quite normal in this scenario to have a larger number in the r column as they are queued up waiting for cpu time.
procs ———–memory———- —swap– —–io—- –system– —-cpu—-
r b swpd free buff cache si so bi bo in cs us sy id wa
5 1 648 755300 31460 2095012 0 0 76 3458 2013 1369 95 5 0 0
5 0 648 741196 31568 2103336 0 0 24 1852 2063 1481 88 12 0 0
8 0 648 706828 31616 2115936 0 0 50 2590 2118 1609 90 10 0 0
14 0 648 700628 31664 2133024 0 0 117 4320 2138 1290 90 10 0 0