virtual machine performance – high cpu ready lab

Application issues, undersizing a virtual machine, or applying a restrictive cpu limit can all result in cpu ready

CPU Ready is the primary indicator of a cpu performance issue.
Although it’s possible to create a cpu-ready alarm, every VMware admin needs to be able to recognize what the values mean.

 

How much cpu ready is ok
It is generally accepted that over 5% of cpu ready is not ideal, over 10% is serious.

 

But how do you convert cpu summation in milliseconds to percentages?
Note: It will be better if you do this, rather than just read – Create a vm and upload some cpu stress tools.
(for example for a windows machine I’ve used load storm)
Set the load to 60% or 70%, configure a cpu limit to less than the demand – see this if you need help.

This PowerCLI one liner collects the cpu ready stat, sorts the highest value, selects the first 5 values.
The polling interval is set to 300 seconds.

PerfTest-1 is my stressed vm

PowerCLI > $vm = get-vm PerfTest-1

PowerCLI > Get-Stat -entity $vm -IntervalSecs 300 -stat ‘cpu.ready.summation’ | sort-object value -descending | Select-Object -first 5 | format-table -auto

MetricId                         Timestamp                   Value Unit
——–                             ———                          —– —-
cpu.ready.summation 04/01/2016 13:45:00 166420 millisecond
cpu.ready.summation 03/01/2016 13:45:00 165325 millisecond
cpu.ready.summation 02/01/2016 13:45:00 165035 millisecond
cpu.ready.summation 05/01/2016 13:45:00 146522 millisecond
cpu.ready.summation 04/01/2016 17:10:00  92956 millisecond

 

The “chart default update interval in seconds” will affects data collection.
Focus on the -IntervalSec variable, to calculate cpu ready in percentage we need to divide the milliseconds

ie: (166420/300 * 1000)*100
(166420/300,000)*100 = 55,47 percent

You can use vmcalc to check your math, but you need to know the chart type (days)

 

Why Statistics ‘Interval Duration’ and ‘Save For’ matter

The powerCLI one liner can be run with different Interval Durations.
In real time the values are every 20 seconds, but these are only kept 2 hours,
The vSphere database rolls or aggregates the statistics after that, real time is rolled up into a 5 minute aggregate, 5 minute into 30 minutes…etc
For example a 30 minute interval is an aggregate of six 5 minute values, and this could be flattening the values.

 

How long does vCenter keep 5 minutes statistics?

vcenterStats-IntervalDuration

 

Above we see vCenter has an Interval Duration for 5 minutes (300 seconds) and Save For of 3 days.
Today is 05/01/2016 that is why the output above goes back to 02/01

If we change the ‘Save For’ we can go back further, the default is one day, the maximum five days.

The following table shows seconds/milliseconds – the 300 seconds or 5 minute interval is the only variable*

vcenterStats-table

*Be aware of how this affects database size when changing values

 

Ok hope you got that, by modifying the “Save For” of the 5 minute interval period we can have a reasonably granular collection for up to 5 days.

 

Check historical cpu ready values
To dig out the asthmatic virtual machines we need some beefy PowerCLI scripting.
See powercli-get-vmreadyspikesforxdays

The interval duration is set to 300 seconds:
Once you run it you can go to the performance monitor in vsphere and compare peak values for past day.
Feed those values into http://www.vmcalc.com/ and check the percentages are correct.

cpu-ready-output-1

 

If we look at vSpshere for this machine we see at 19.55,  ready values of 86353

cpu-ready-output-2

 

Put those values in vCalc and compare the percentage  with the grid-view above

 

cpu-ready-output-3

 

Bingo spot on 28.7 %…

 

hope you found this helpful

 

performance snorkel dives home

 


 

 

Setting a custom vcenter alarm to detect high ready issues,
How cpu limits disable cpu usage monitoring

Configure high cpu ready alarm

virtual machine cpu limits affect cpu usage monitoring

 

Leave a Reply

Your email address will not be published. Required fields are marked *