VMmark Scores Investigated: should VMmark be part of your hardware decisions?

Introduction

While the SPEC Virtualization Committee is slowly but steadily developing a new virtualization benchmark called "SPECvirt_sc2009", VMmark remains the only industry standard benchmark to compare hardware for virtualization as all the tier-one hardware vendors have published results for this benchmark. Why would you, the IT professional, care about yet another benchmark? Some professionals will probably point towards the lack of impact (and interest) that TPC and other industry standard benchmarks make when it comes to purchasing hardware decisions, as "performance is only a small part of the decision". We could not agree more when it comes to many industry benchmarks, but a virtualization benchmark is a special case.

We have said this before: virtualization is the killer application of the first decade of the 21^st century. More than half of the servers sold today will end up being host for quite a few virtual machines. Even better, it is a market that is expanding even in these times of economic crisis. From the 591 IT professionals that took the Data Center Decisions 2008 Purchasing Intentions survey, only 2% indicated that the budget for virtualization related purchases would decrease, while 56% indicated that this budget would increase. That means scoring well in the VMmark benchmark is starting to get crucial for the server hardware vendors.

Higher virtualization performance means higher consolidation ratio, and thus translates in most cases in immediate cost savings: you need fewer servers, which allows you to save on energy, hardware costs, and manpower to maintain and install those servers. Performance is a much more important factor in the decision process when it comes to virtualizing the datacenter. So while a higher TPC or SPECjbb score will seldom result in happier users and cost savings, a higher VMmark score should almost always do that… in theory.