Losing one’s mind over memory choices

Losing one’s mind over memory choices

Capacity Optimisation, Predictive Analytics and Memory Modelling

We’re often asked by customers, “Why does my memory show 99% used? That’s a capacity issue, right?”

Well, the answer is typically, “not necessarily” and as with most memory-related questions, there is usually a bit of further investigation required into the context. Invariably, the question turns out to be in relation to a Unix server, and the metric being supplied by the server is simply stating the total memory used by the OS.

However, many flavours of Unix OS are configured to use all the memory that is available, on the basis that memory unused is memory wasted. Instead they will use any memory not needed by the application for caching and for buffering disk I/O, which can improve the performance. Give the server twice the memory and the usage will still show 99% – the buffer and cache will just be even bigger!  If the application itself needs more memory, the OS simply re-assigns some pages of cache memory that have not been used recently to the application.

So how do we know if the server really is short of memory?  On servers configured in this way, a better measure of memory usage is one that excludes the memory used for buffer and cache from the used total.  Buffer and cache memory can be freed up for use by the application with little overhead, and so effectively it counts as “free”.  This measure can then be monitored against the total memory capacity or against the capacity minus a reserve headroom (we may want to define a reserve so that the server has some memory set aside available to be used for the cache).

But the choice of memory metric doesn’t end there. On VMware vCenter virtualisation platform for example, there are two key memory metrics available for both virtual servers and hosts; memory consumed and active memory.

Memory consumed measures the actual amount of physical memory that is being used by each server. So that’s the one to use, right? Well, maybe! When a virtual server needs memory, the hypervisor allocates physical memory pages to it. However, if the virtual server OS then deallocates that memory, the host does not free the memory for use elsewhere (it can’t see the guest OS de-allocation operation). So consumed memory tends to increase over time, and rarely decrease. It is a high-water mark, a measure of how much memory the virtual server has used at any time in the past.

Only when the host gets close to full physical memory usage does it attempt to reclaim some of that memory from the virtual server, and it does so using techniques like ballooning that are beyond the scope of this article.

An alternative on VMWare vCenter is the active memory metric. vCenter measures active memory by looking at which of the consumed memory pages have actually been used during the collection interval.  By measuring a server’s memory usage based on its peak active memory, we ensure that the memory is big enough at least to hold the maximum number of actively used pages at any time. We do need to bear in mind that memory paging will still occur if the actual set of pages in use is changing over time (the new ones are paged in to replace older ones that have not been used in a while, and hence are no longer deemed “active”).  But this is the kind of normal paging activity that is typical of any virtual memory system.

Figure: A typical summary view of memory usage in Capacity Planner – Memory demand (active memory) shows as much lower than consumed memory (mem.consumed.average).

So which of these should we use? Well, we have clients that use both! Those who have very high performance requirements for their applications may choose not to over-allocate memory on vCenter at all.  In this case, all virtual server memory is mapped to physical memory, and memory consumed gives a good reflection of the “maximum footprint” of the server over time.

For clients who wish to use vCenter’s memory over-allocation capabilities to share the physical memory resource between many virtual servers, then active memory is a useful metric. It allows us to determine how much memory is needed to allow all active pages at any time to fit into physical memory. That means the application will still perform well, but we can fit more virtual servers on each physical host than if we sized based on consumed memory.

Capacity Planner provides the flexibility to view capacity on your terms, combining your knowledge of how applications are best served using your preferred metrics with unprecedented visibility of headroom for growth across your estate.  Predictive forecasting on these metrics enable informed decisions on when and how much to invest to minimise risk and ensure availability.


By | 2017-12-05T15:03:34+00:00 December 5th, 2017|