Benchmark Overview

2017 CPU

For our review, we are implementing our fresh CPU testing benchmark suite, using new scripts developed specifically for this testing. This means that with a fresh OS install, we can configure the OS to be more consistent, install the new benchmarks, maintain software version consistency without random updates and start running the tests in under 5 minutes. After that it's a one button press to start an 8-10hr test (with a high-performance core) with nearly 100 relevant data points in the benchmarks given below. The tests cover a wide range of segments, some of which will be familiar but some of the tests are new to benchmarking in general, but still highly relevant for the markets they come from.

Our new CPU tests go through six main areas. We cover the Web (we've got an un-updateable version of Chrome 56), general system tests (opening tricky PDFs, emulation, brain simulation, AI, 2D image to 3D model conversion), rendering (ray tracing, modeling), encoding (compression, AES, h264 and HEVC), office based tests (PCMark and others), and our legacy tests, throwbacks from another generation of bad code but interesting to compare.

All of our benchmark results can also be found in our benchmark engine, Bench.

A side note on OS preparation. As we're using Windows 10, there's a large opportunity for something to come in and disrupt our testing. So our default strategy is multiple: disable the ability to update as much as possible, disable Windows Defender, uninstall OneDrive, disable Cortana as much as possible, implement the high performance mode in the power options, and disable the internal platform clock which can drift away from being accurate if the base frequency drifts (and thus the timing ends up inaccurate).

New Tests

SpecWPC v2.1 – A lot of the industry turn to SPEC to produce standard benchmarks suitable for various markets. The latest version of the workstation focused benchmark suite was released this year, and tackles six main areas with over 30 different benchmarks. These include compute, visualization, medical, oil and gas, finance, and typical workstation areas. For consistency we run all the tests (except IOMeter) on Windows 10, using an RX 460 graphics card at 1080p resolution with an MX200 SSD.

PCMark10 – We had several requests to include PCMark10 in our new testing suite. Normally we wait until a new benchmark has most of the problems ironed out, however our initial performance scaling metrics show that PCMark10 is basically there already. The extended suite covers ‘Essential, Productivity and Creativity’ benchmarks such as GIMP, Blender, video editing, conferencing, complex spreadsheets and other tests. We use the subtest values as well as the gaming physics result.

Agisoft PhotoScan 1.3.3 – Again, requests to use a more updated version of Photoscan were also coming through the inbox. Over the older version, Photoscan includes various throughput enhancements to each of the core points of the algorithm. Agisoft also gave us a new larger set of more detailed test images to generate our 3D models, giving a longer benchmark (but results are not comparable to the old data). We’ve run this benchmark on about a dozen CPUs ready for this review.

Office / Professional Tests

PCMark8
Chromium Compile (v56)

Rendering Tests

Corona 1.3
Blender 2.78
LuxMark v3.1 CPU C++
LuxMark v3.1 CPU OpenCL
POV-Ray 3.7.1b4
Cinebench R15 ST
Cinebench R15 MT

Encoding Tests

7-Zip 9.2
WinRAR 5.40
AES Encoding (TrueCrypt 7.2)
HandBrake v1.0.2 x264 LQ
HandBrake v1.0.2 x264-HQ
HandBrake v1.0.2 HEVC-4K

System Tests

PDF Opening
FCAT
3DPM v2.1
Dolphin v5.0
DigiCortex v1.20
Agisoft PhotoScan v1.0

Legacy Tests

3DPM v1 ST / MT
x264 HD 3 Pass 1, Pass 2
Cinebench R11.5 ST / MT
Cinebench R10 ST / MT

A Note on CPU Gaming Tests (Repeat from Page 1)

I know a lot of our readers are gamers, and are interested in seeing how well (or poorly) these massive multi-core chips perform in the latest titles at the highest resolutions. Apologies to disappoint, but I am going to tackle the more traditional consumer tasks in a second review, and which will mean that gaming will be left for that review. For the users that have followed my reviews (and Twitter) of late, I am still having substantial issues with my X299 test beds on the gaming results, with Skylake-X massively underperforming where I would expect a much higher result.

After having to dedicate recent time to business trips (Hot Chips, IFA) as well as other releases (Threadripper), I managed to sit down in the two weeks between trips to figure what exactly what was going on. I ended up throwing out the two X299 pre-launch engineering samples I was using for the Skylake-X testing, and I received a new retail motherboard only a few days before this review.  This still has some issues that I spent time trying to debug, which I think are related to how turbo is implemented, which could either be Intel related or BIOS specific.

To cause insult to injury to everyone who wants to see this data, I have jumped on a plane to travel half-way around the world for a business trip during the week of this launch, which leaves the current results inconclusive. I have reached out to the two other motherboard vendors that I haven’t received boards from; just in case the issue I seem to be having is vendor specific. If I ever find out what this issue is, then I will write it up, along with a full Skylake-X gaming suite. It will have to wait to mid-late October, due to other content (and more pre-booked event travel).

Test Bed and Setup Benchmarking Performance: SPECwpc v2.1
Comments Locked

152 Comments

View All Comments

  • mapesdhs - Tuesday, September 26, 2017 - link

    In that case, using Intel's MO, TR would have 68. What Intel is doing here is very misleading.
  • iwod - Monday, September 25, 2017 - link

    If we factor in the price of the whole system, rather then just CPU, ( AMD's MB tends to be cheaper ), then AMD is doing pretty well here. I am looking forward to next years 12nm Zen+.
  • peevee - Monday, September 25, 2017 - link

    From the whole line, only 7820X makes sense from price/performance standpoint.
  • boogerlad - Monday, September 25, 2017 - link

    Can an IPC comparison be done between this and Skylake-s? Skylake-x LCC lost in some cases to skylake, but is it due to lack of l3 cache or is it because the l3 cache is slower?
  • IGTrading - Monday, September 25, 2017 - link

    There will never be an IPC comparison of Intel's new processors, because all it would do is showcase how Intel's IPC actually went down from Broadwell and further down from KabyLake.

    Intel's IPC is a downtrend affair and this is not really good for click and internet traffic.

    Even worse : it would probably upset Intel's PR and that website will surely not be receiving any early review samples.
  • rocky12345 - Monday, September 25, 2017 - link

    Great review thank you. This is how a proper review is done. Those benchmarks we seen of the 18 core i9 last week were a complete joke since the guy had the chip over clocked to 4.2GHz on all core which really inflated the scores vs a stock Threadripper 16/32 CPU. Which was very unrealistic from a cooling stand point for the end users.

    This review had stock for stock and we got to see how both CPU camps performed out of the box states. I was a bit surprised the mighty 18 core CPU did not win more of the benches and when it did it was not by very much most of the time. So a 1K CPU vs a 2K CPU and the mighty 18 core did not perform like it was worth 1K more than the AMD 1950x or the 1920x for that matter. Yes the mighty i9 was a bit faster but not $1000 more faster that is for sure.
  • Notmyusualid - Thursday, September 28, 2017 - link

    I too am interested to see 'out of the box performance' also.

    But if you think ANYONE would buy this and not overclock - you'd have to be out of your mind.

    There are people out there running 4.5GHz on all cores, if you look for it.

    And what is with all this 'unrealistic cooling' I keep hearing about? You fit the cooling that fits your CPU. My 14C/28T CPU runs 162W 24/7 running BOINC, and is attached to a 480mm 4-fan all copper radiator, and hand on my heart, I don't think has ever exceeded 42C, and sits at 38C mostly.

    If I had this 7980XE, all I'd have to do is increase pump speed I expect.
  • wiyosaya - Monday, September 25, 2017 - link

    Personally, I think the comments about people that spend $10K on licenses having the money to go for the $2K part are not necessarily correct. Companies will spend that much on a license because they really do not have any other options. The high end Intel part in some benchmarks gets 30 to may be 50 percent more performance on a select few benchmarks. I am not going to debate that that kind of performance improvement is significant even though it is limited to a few benchmarks; however, to me that kind of increased performance comes at an extreme price premium, and companies that do their research on the capabilities of each platform vs price are not, IMO, likely to throw away money on a part just for bragging rights. IMO, a better place to spend that extra money would be on RAM.
  • HStewart - Monday, September 25, 2017 - link

    In my last job, they spent over $100k for software version system.

    In workstation/server world they are looking for reliability, this typically means Xeon.

    Gaming computers are different, usually kids want them and have less money, also they are always need to latest and greatest and not caring about reliability - new Graphics card comes out they replace it. AMD is focusing on that market - which includes Xbox One and PS 4

    For me I looking for something I depend on it and know it will be around for a while. Not something that slap multiple dies together to claim their bragging rights for more core.

    Competition is good, because it keeps Intel on it feat, I think if AMD did not purchase ATI they would be no competition for Intel at all in x86 market. But it not smart also - would anybody be serious about placing AMD Graphics Card on Intel CPU.
  • wolfemane - Tuesday, September 26, 2017 - link

    Hate to burst your foreign bubble but companies are cheap in terms of staying within budgets. Specially up and coming corporations. I'll use the company I work for as an example. Fairly large print shop with 5 locations along the US West coast that's been in existence since the early 70's. About 400 employees in total. Server, pcs, and general hardware only sees an upgrade cycle once every 8 years (not all at once, it's spread out). Computer hardware is a big deal in this industry, and the head of IT for my company Has done pretty well with this kind of hardware life cycle. First off, macs rule here for preprocessing, we will never see a Windows based pc for anything more than accessing the Internet . But when it comes to our servers, it's running some very old xeons.

    As soon as the new fiscal year starts, we are moving to an epyc based server farm. They've already set up and established their offsite client side servers with epyc servers and IT absolutely loves them.

    But why did I bring up macs? The company has a set budget for IT and this and the next fiscal year had budget for company wide upgrades. By saving money on the back end we were able to purchase top end graphic stations for all 5 locations (something like 30 new machines). Something they wouldn't have been able to do to get the same layout with Intel. We are very much looking forward to our new servers next year.

    I'd say AMD is doing more than keeping Intel on their feet, Intel got a swift kick in the a$$ this year and are scrambling.

Log in

Don't have an account? Sign up now