We have been engaged with a customer to do a performance review, and they had no apparent issues of performance. We set up performance counters on all machines and they were all within threshold. However the backend role machine showed interesting behavior. It had “Available memory” within threshold while “pages/ sec” far above threshold, other hardware metrics were also within threshold.
Available MBytes is the amount of physical memory, in Megabytes, immediately available for allocation to a process or for system use.
Pages/sec is the rate at which pages are read from or written to disk to resolve hard page faults. This counter is a primary indicator of the kinds of faults that cause system-wide delays
Investigating further it turned out this machine is handling a pretty good load of plugins, MSCRMAsync services and it is virtualized (VMware) as well.
Virtualization is supported as stated in Software requirements for Microsoft Dynamics CRM Server 2013
Checking the settings of the virtual machine it turned out it doesn’t have “resource reservation” at all, which is contrary to Microsoft recommendations. Microsoft recommends using fully reserved memory to avoid all ballooning, swamping and “Double paging” problems (described in VMware white paper).
If the virtual machine memory size is too small, guest-level Paging is inevitable:
Ref: http://www.vmware.com/files/pdf/perf-vsphere-memory_management.pdf
Hardware virtualization technologies can provide significant advantages over traditional hardware-based implementations. However, software that is running in a virtualized hardware environment is affected by intrinsic
limitations, especially performance limitations.
Ref : http://support.microsoft.com/kb/957054/en-us
Having these settings in place stabilized the system and added to the general performance and stability of the system. So our advice here if your Dynamics CRM server machines are virtualized make sure you familiarize yourself with the best practices of the virtualization provider, and you understand their best practice for memory management. And of course “performance and scalability considerations” from kb 957054
By: Julien Clauzel, Fadi EL-Hennawy