Virtualization (ESX 3.5 Update 2/3)

As we discussed in the first page of this article, virtualization will be implemented on about half of the servers bought this year. Virtualization is thus the killer application and the most important benchmark available. Since we have not been able to carry out our own virtualization benchmarking (due to the BIOS issues described earlier), we turn to VMware's VMmark. It is a relatively reliable benchmark as the number of "exotic tricks hardly used in the real world" (see SPECjbb) are very limited.

VMware VMmark is a benchmark of consolidation. Several virtual machines performing different tasks are consolidated together and called a tile. A VMmark tile consists of:

  • MS Exchange VM
  • Java App VM
  • An Idle VM
  • Apache web server VM
  • MySQL database VM
  • A SAMBA fileserver VM

The first three run on a Windows 2003 guest OS, the last three on SUSE SLES 10.

To understand why this benchmark is so important just look at the number of tiles that a certain machine can support:

VMware VMmark number of Tiles

The difference between the Opteron 8360 SE and the 8384 "Shanghai" is only 200MHz and 4MB L3 cache. However, this small difference allows you to run 18 (!) extra virtual machines. Granted, it may require that you install more memory, but adding memory is cheaper than buying a server with more CPU sockets or adding yet another server. Roughly calculated you could say that the new quad-core Opteron allows you to consolidate 27% more virtual servers on one physical machine, which is a significant cost saving.

Of course, the number of tiles that a physical server can accommodate provides only a coarse-grain performance measure, but an important one. This is one of the few times where a higher benchmark score directly translates to a cost reduction. Performance per VM is of course also very interesting. VMware explains how they translate the performance of each different workload in different tiles into one overall score:

After a benchmark run, the workload metrics for each tile are computed and aggregated into a score for that tile. This aggregation is performed by first normalizing the different performance metrics such as MB/second and database commits/second with respect to a reference system. Then, a geometric mean of the normalized scores is computed as the final score for the tile. The resulting per-tile scores are then summed to create the final metric.

Let us take a look:

VMware VMmark

Thanks to the lower world switch times, higher clock speed, and larger cache, the new "Shanghai" Opteron 8384 improves the already impressive scores of the AMD "Barcelona" 8356 by almost 43%. The only Intel that comes somewhat close is the hex-core behemoth known as the Xeon X7460, which needs a lot more power. IBM is capable of performing a tiny bit better than Dell thanks to its custom high performance chipset.

It is clear that the Xeon 7350 is at the end of its life: it offers a little more than 2/3 of the performance of the best Opteron while using a lot more power. Even the latest improved stepping, the Xeon X5470 at 3.33GHz, cannot keep up with the new Opteron quad-core. The reason is simple: as the number of VMs increase, so do the bandwidth requirements and the amount of world switches. That is exactly where the Opteron is far superior. It is game over here for Intel… until the Xeon 5570 2.93GHz arrives in March.

Other (Windows 2003 64-bit) Pricing


View All Comments

  • Bruce Herndon - Tuesday, December 23, 2008 - link

    I'm surprised by your comments. You claim that VMmark is a CPU/memory-centric benchmark. If I look at the raw data in the VMmark disclosure for Dell's R905 score of 20.35 @ 14 tiles, I see that the benchmark is driving 250-300 MB/s of disk IO across several HBAs and storage LUNs. This characteristic scales with the various systems mentioned in the article.

    As a designer of VMmark, I happen to know that both storage bandwidth (for the fileserver) and latency (for mail and database)are critical to acheiving good VMmark scores. Furthermore, the webserver drives substantial network IO. The only purely CPU-centric component to VMmark is the javaserver. Overall, the benchmark does exercise the entire virtualization solution - hypervisor, CPU, memory, disk, and network.
  • cdillon - Tuesday, December 23, 2008 - link

    While SAS and Infiniband share some connectors and obtain similar data rates, they are incompatible technologies with two different purposes. Infiniband can be used for disk shelf connections, but it is less common and definitely not the case here. You should not call the connection between the Adaptec 5805 controller and the disk shelf an "Infiniband connection", even if it is using Infiniband connectors and cables, it is simply an SAS connection.

  • JohanAnandtech - Tuesday, December 23, 2008 - link

    Well, the physical layer is Infiniband, the used protocol is SCSI. I can understand calling it an "infiniband connection" maybe confusing, but the cable is an infiniband cable. Reply
  • shank15217 - Friday, December 26, 2008 - link

    Anand, I think the above poster is right. The Adaptec RAID 5805 uses SFF-8087 connectors but the protocol is SSP (Serial SCSI Protocol). Infiniband is a physical layer protocol that shares the same connector as SAS but they are not the same. Nothing in the Adaptec RAID 5805 spec mentions Infiniband as a supported protocol.">
  • niva - Tuesday, December 23, 2008 - link

    I'm not sure you can run your same ol benchmark for rendering, and I'd really like more insight into what you guys are rendering and if it's indeed using all 16/24(six core 4 point system)/32(hyperthreading) cores on the system.

    What renderer, what scene, details details...

    These chips get gobbled up by render farms and this is indeed where they can really flex their muscles to the fullest.
  • JohanAnandtech - Tuesday, December 23, 2008 - link

    Just click on the link under "we have performed so many times before" :-) Reply
  • akinneyww - Tuesday, December 23, 2008 - link

    I read DailyTech and to keep up with the latest in IT. I appreciate the thought that has gone into putting together this article. I would like to see more articles like this one. Reply
  • Jammrock - Tuesday, December 23, 2008 - link

    The VMware results shocked me the most. I know AMD has been working hard on the virtualization sector and it looks like their work has paid off. Reply
  • classy - Tuesday, December 23, 2008 - link

    With the rapid increase of virtualization, AMD is looking really strong. We have begun using 3.5 Vmware and are expanding the use of it. Virtualization is truly becoming a big thing in server choice. Reply

Log in

Don't have an account? Sign up now