Application Benchmarks

With a complex multi-layer storage system like the Intel Optane Memory H10, the most accurate benchmarks will be tests that use real-world applications. BAPCo's SYSmark 2018 and UL's PCMark 10 are two competing suites of automated application benchmarks. Both share the general goal of assigning a score to represent total system performance, plus several subscores covering different common use cases. PCMark 10 is the shorter test to run and it provides a more detailed breakdown of subscores. It is also much more GPU-heavy with 3D rendering included in the standard test suite and some 3DMark tests included in the Extended test. SYSmark 2018 has the advantage of using the full commercial versions of popular applications including Microsoft Office and Adobe Creative Suite, and it integrates with a power meter to record total system energy usage over the course of the test.

The downside of these tests is that they cover only the most common everyday use cases, and do not simulate any heavy multitasking. None of their subtests are particularly storage-intensive, so most scores only vary slightly when changing between fast and slow SSDs.

BAPCo SYSmark 2018

BAPCo's SYSmark 2018 is an application-based benchmark that uses real-world applications to replay usage patterns of business users, with subscores for productivity, creativity and responsiveness. Scores represnt overall system performance and are calibrated against a reference system that is defined to score 1000 in each of the scenarios. A score of, say, 2000, would imply that the system under test is twice as fast as the reference system.

BAPCo SYSmark 2018 Scores
Creativity Productivity Responsiveness Overall

The Kaby Lake desktop and Whiskey Lake notebook trade places depending on the subtest; sometimes the notebook is ahead thanks to its extra RAM, and sometimes the desktop is ahead thanks to its higher TDP. These differences usually have a bigger impact than choice of storage, though the Responsiveness test does show that a hard drive alone is inadequate. The Optane Memory H10's score with caching on is not noticeably better than when using the QLC portion alone, and even the hard drive with an Optane cache is fairly competitive with the all-solid state storage configurations.

Energy Usage

The SYSmark energy usage scores measure total system power consumption, excluding the display. Our Kaby Lake test system idles at around 26 W and peaks at over 60 W measured at the wall during the benchmark run. SATA SSDs seldom exceed 5 W and idle at a fraction of a watt, and the SSDs spend most of the test idle. This means the energy usage scores will inevitably be very close. The notebook uses substantially less power despite this measurement including the display. None of the really power-hungry storage options (hard drives, Optane 900P) can fit in this system, so the energy usage scores are also fairly close together.

BAPCo SYSmark 2018 - Energy Consumption

The Optane Memory H10 was the most power-hungry M.2 option, and leaving the Optane cache off saves a tiny bit of power but not enough to catch up with the good TLC-based drives. The Optane SSD 800P has better power efficiency than most of the flash-based drives, but its low capacity is a hindrance for real-world use.

 

UL PCMark 10

PCMark 10 scores
Subscore:

The Optane cache provides enough of a boost to PCMark 10 Extended scores to bring the H10 into the lead among the M.2 SSDs tested on the Whiskey Lake notebook. The Essentials subtests show the most impact from the Optane storage while the more compute-heavy tasks are relatively unaffected, with the H10 performing about the same with or without caching enabled.

Test Setup Cache Size Effects
Comments Locked

60 Comments

View All Comments

  • Flunk - Monday, April 22, 2019 - link

    This sounded interesting until I read software solution and split bandwidth. Intel seems to be really intent upon forcing Optane into products regardless if they make sense or not.

    Maybe it would have made sense with SSDs at the price points they were this time last year, but now it just seems like pointless exercise.
  • PeachNCream - Monday, April 22, 2019 - link

    Who knew Optane would end up acting as a bandage fix for QLC's garbage endurance? I suppose its better than nothing, but 0.16 DWPD is terrible. The 512GB model would barely make it to 24 months in a laptop without making significant configuration changes (caching the browser to RAM, disabling the swap file entirely, etc.)
  • IntelUser2000 - Monday, April 22, 2019 - link

    The H10 is a mediocre product, but endurance claims are overblown.

    Even if the rated lifespan is a total of 35TB, you'd be perfectly fine. The 512GB H10 is rated for 150TB.

    The amount of users that would even reach 20TB in 5 years are in the minority. When I was actively using the system, my X25-M registered less than 5TB in 2 years.
  • PeachNCream - Monday, April 22, 2019 - link

    Your usage is extremely light. Endurance is a real-world problem. I've already dealt with it a couple of times with MLC SSDs.
  • IntelUser2000 - Monday, April 22, 2019 - link

    SSDs are over 50% of the storage sold in notebooks. It's firmly reaching mainstream there.

    I would say instead I think most of *your* customers are too demanding. Vast majority of the folks would use less than me.

    The market agrees too, which is why we went from MLC to TLC, and now we have QLCs coming.

    Perhaps you are confusing write-endurance with physical stress endurance, or even natural MTBF related endurance.
  • PeachNCream - Monday, April 22, 2019 - link

    I haven't touched on any usage but my own so far. The drives' own software identified the problems so if there is confusion about failures, that's in the domain of the OEM. (Note, those drives don't fail gracefully either so that data can be recovered. It's a pretty ugly end to reach.) As for the move from MLC to TLC and now QLC -- thats driven by cost sensitivity for given capacities and ignores endurance to a great extent.
  • IntelUser2000 - Monday, April 22, 2019 - link

    I get the paranoia. The world does that to you. You unconsciously become paranoid in everything.

    However, for most folks endurance is not a problem. The circuit in the SSD will likely fail of natural causes before write endurance is reached. Everything dies. But people are just excessively worried about NAND SSD write endurance because its a fixed metric.

    It's like knowing the date of your death.
  • PeachNCream - Friday, May 3, 2019 - link

    That's not really a paranoia thing. You're attempt to bait someone into an argument where you can then toss out insults is silly.
  • SaberKOG91 - Monday, April 22, 2019 - link

    That's a naive argument. Most SSDs of 250GB or larger are rated for at least 100TBW on a 3 year warranty. 75TBW on a 5 year warranty is an insult.

    I think you underestimate how much demand the average user makes of their system. Especially when you have things like anti-virus and web browsers making lots of little writes in the background, all the time.

    The market is going from TLC to QLC because of density, not reliability. We had all the same reliability issues going from MLC to TLC and from SLC to MLC. It took years for each transition for manufacturers to reach the same durability level as the previous technology, all while seeing the previous generation continuing to improve even further. Moving to denser tech means smaller dies for the same capacity or higher capacity for unit area which is good for everyone. But these drives don't even look to have 0.20DWPD or 5 year warranty of other QLC Flash products.

    I am a light user who doesn't have a lot of photos or video and this laptop has already seen 1.3TBW in only 3 months. My work desktop has over 20TBW from the last 5 years. My home desktop where I compile software has over 12TBW in the first year. My gaming PC has 27TBW on a 5 year old drive. So while I might agree that 75TBW seems like a lot, If I were to simplify my life down to one machine, I'd easily hit 20TBW a year or 8TBW a year even without the compile machine.

    That all said, you're still ignoring that many Micron and Samsung drives have been shown to go way beyond their rated lifespan whereas Optane has such horrible lifespan at these densities that reviewers destroyed the drives just benchmarking them. Since the Optane is acting as a persistent cache, what happens to these drives when the Optane dies? At the very least performance will tank. At the worst the drive is hosed.
  • IntelUser2000 - Monday, April 22, 2019 - link

    Something is very wrong with your drive or you are not really a "light user".

    1300GB in 3 months equals to 14GB write per day. That means if you use your computer 7 hours a day you'd be using 2GB/s hour. The computer I had the SSD on I used it for 8-12 hours every day for the two years and it was a gaming PC and a primary one at that.

    Perhaps the X25-M drive I had is particularly good at this aspect, but the differences seem too much.

    Anyways, moving to denser cells just mean consumer level workloads do not need the write endurance MLC needs and lower prices are preferred.

    "Optane has such horrible lifespan at these densities that reviewers destroyed the drives just benchmarking them."

    Maybe you are referring to the few faulty units in the beginning? Any devices can fail in the first 30 days. That's completely unrelated to *write endurance*. The first gen modules are rated for 190TBW. If they played around for a year(which is unrealistic since its for a benchmark), they would have been using 500GB/s day. Maybe you want to verify your claims yourself.

Log in

Don't have an account? Sign up now