What on earth is a Benchmark? The purpose of this blog is to explain the conducting of a Microsoft Dynamics CRM Benchmark. This entry will serve as a Benchmarking 101, and try to lay out the basics of what it all entails. To try and illustrate it, I’ll use the example that was given to me by Grant Geiszler (PFE team). He said that while it’s not a perfect example, it’s a bit like taking a car for a test drive. A person gets to see what they might buy and how it handles in the environment.
A benchmark can be done in two different scenarios: first, they already have their own hardware and infrastructure, and we replicate that configuration in our labs. The second scenario is if the customer does not have the hardware and infrastructure and want to know what system requirements they would need to optimize their environment. Once the situation has been determined, we move on to test cases.
A test case simulates users by executing a series of real-world business transactions; each transaction is composed of a set of discrete user interactions with the Microsoft CRM system. When these test cases have all been created and the environment is in place, we can do a run in our lab using settings matched to the customer’s specific environment (expected number of users, transaction volume placed on the system, etc.). We’ll analyze what tweaks we can make to optimize the system and then repeat the tests. This cycle repeats itself until we believe we have reached optimal performance.
When we share the results of the benchmark, we highlight the expected response times on multiple runs of the performance simulator as well as any recommended hardware configuration, and any optimization tips. So this is the part where I can draw a line back to my example about Benchmarking being a bit like test driving a car--the customer gets to see what kind of results they can expect without having to actually buy the car.
Why would you want to do a benchmark? It gives you the ability to see the future. Now I’m half kidding, but it does allow customers to anticipate problems before they arise and then deal with those issues by optimizing for performance early on. This gives the customer an infrastructure that is truly good to go. This will translate directly into a significant decrease in risk of downtime or user frustration due to performance issues. Less down time will drive down costly inefficiencies that decrease a business’ overall health and bottom line significantly.
Hopefully, this gave you a better understanding of just what exactly Benchmarking is. It really is a great way for a customer to preview a purchase, anticipate future problems, and fix those problems before they happen so the company can ultimately run much more efficiently.
Over the coming weeks you will see content out here explaining how a benchmark works in detail, and some results from successful benchmarks we’ve conducted, as well as certain scenarios we’ve worked on and vendors we’ve worked with. If you have questions along the way or if there’s some more information you’d like to see, feel free to drop in a comment.