TurboWrite wouldn't make any sense on an enterprise drive because all that matters is sustained performance. Client drives are a different case because IO activity tends to happen in bursts, so having a fast buffer is beneficial (and it makes the drive look better in benchmarks).
Yeah i figured as much. Aside from being more suited for short bursts (as you'd said) its really great for marketing department, since they can slap magical 500MB/s+ speeds, that every uninformed buyer is after. 500MB/s sure sells a lot better than 250MB/s (or even less, if we're dealing with lower capacities) :)
Thanks for the article. Would suggest to make a $/manufacturer declared write endurance comparison (PB writen until manufacturer write warranty end). If there's a chance (due to the difficult on testing time) also make a $/tested write life (PB writen until tested dead).
$/endurance is certainly something I will be looking at in the future. This was more of a preview than a full review and in the full review I will have more graphs and comparisons to support the conclusions :)
As for actual endurance testing, that is something I will not do. It took me two weeks of continuos writing to get the rated endurance to drop by 1% in the 845DC PRO, so it would take way too long to wait for the drive to die.
A question/comment, though, do you think it would make more sense to report the stdev measurements as an "error bar" attached to each data point rather than an additional graph with a 2nd y-axis? I think that might be more compact without having to worry about having multiple y-scales to read. Then it might even be possible to plot an average + error bars data set for multiple different SSDs on the same axis w/o having to worry about which curve is the avg, which curve is the stdev, etc.
Another way to present the standard deviation data would be to draw it in two average +/- stdev graphs above and below the average graph. This would better allow visualizing the actual values the average minus stdev has.
Thanks for the suggestion. I will certainly look into alternative ways to present the standard deviation. I need to redo the graphs anyway for our new suite, so while I'm at it I can see if there is a better way to present the data.
I would like to echo this statement. Some of the heavy technical stuff makes my eyes glaze over at times, but this was so well written that I really got into it. Awesome article, Kristian.
"The only difference between the 845DC EVO and PM853T is the firmware and the PM853T is geared more towards sustained workloads, which results in slightly higher random write speed (15K IOPS vs 14K IOPS)."
Chart for PM853T shows 14K for Random Writes, so likely needs to be corrected.
Good catch, I was waiting for Samsung to send me the full data sheet for the PM853T but I never got it, so I accidentally left the 845DC EVO specs there. I've now updated it with the specs I have and with some additional commentary.
The only issue with calculating performance as listed on the first page is that it assumes that the SSD works perfectly in all aspects. No ECC, no wear leveling, no garbage collection. None of these are true. Even without those factors no SSD will ever behave absolutely perfectly in every aspect at all times....anything but. That is why there is so much variation between vendors. It would be impossible to calculate performance using that method with an SSD in the real world.
Of course real world is always different because the transfer size and queue depth are constantly changing and no SSD behaves perfectly. I mentioned that it is a hypothetical SSD and obviously no real drive would have a constant latency of 1ms or perfect transfer size scaling. The goal was to demonstrate how the metrics are related and it is easier with concrete, albeit hypothetical, examples.
I think you nailed it on the theoretical stuff (the relationships between the parameters), and presented it well (easy to understand).
This has been bugging me for a while too, although I won't pretend to have gotten it figured out - it's just that I keep getting pickier in my lab notes and shopping specs about things like queue depth for a given size transfer for the specified performance. Leave any condition or parameter out, and the specs seem kinda useless. Leave everything in, and then I wonder which ones are pertinent to my usage scenarios.
Now, your explanation will have me questioning the manufacturer's motives for the selection of each unit of measure chosen for each listed spec. Who says this field of endeavor doesn't lead to obsession? :)
If any of the specs *are* pertinent to my usage scenarios, then I wonder which ones are *most* pertinent for which of my usage scenarios (laptop, versus general desktop, versus high powered workstation).
Any relationships that you discover or methods that you develop for the charts to help explain it better are most welcome. I know this is an over-simplification, but I would guess that most workstation users want to know:
- Is my drive limiting my performance (why did that operation stutter or lag, or why does it take so long - was it the drive)?
- Is there anything I can do about it that I can afford (mainly, what can I replace it with - new controller card, newer, better designed SSDs, better racks, cables, etc.)?
I am pleased that you are helping us out by further dissecting performance consistency / variation. I suspect that although SSDs are an order of magnitude (or more) better than HDDs at many tasks, the "devil is in the details," and that there is a reason that many SSD equipped machines still "hiccup," fairly frequently (although not nearly as often or as bad as HDD equipped machines). I also suspect that drive sub-systems are still one of the most common weaker links that is responsible for such hiccups.
I am particularly interested in the (usually) brutally difficult small file size tests. These tests seem to be able to bring even the best of machines to a crawl, and any device (ie: SSD) that can help performance on those tests seems to be very likely to be noticeable to the end user.
If you do indeed find "let downs" in performance consistency (or any other drive related performance spec), then maybe the manufacturers will work to improve upon those weaknesses until we get "buttery smooth" performance...
...or at least until we can definitively start looking at other sub-systems (compute, memory, I/O, etc.) to solve the hiccups.
My introductory courses were on 8086 computers running DOS. I don't remember them often stopping to think about anything... until I started "hitting" the disks heavily. The more things change, the more I suspect they stay the same ;)
So I keep allocating more of my budget to disks and disk sub-systems than anything else. AT's articles are thus *very* helpful in "aiming" that budget and I hope you have some revelations for us soon that show which products are worth the money.
I have been wondering for a bit, why hasn't Enterprise switched to some other interconnect? Surely they could do PCI-E have have it directly connected to the CPU. ( Assuming they dont need those for GPU or other things ).
And I have been Samsung being extremely aggressive with the web hosting market. Where Intel is lacking behind.
"real" enterprise has been running SAS and FibreChannel for rather a long time. InfiniBand every now and again. To the extent that enterprise buys X,000 drives to parcel out to offices and such, then that's where the SATA drives go. But that's not really enterprise storage. Real enterprise SSD/flash/etc. doesn't have a list price (well, only in the sense that your car did) and I'd wager that not one of the enterprise SSD/flash companies (and no, Intel doesn't count) has ever offered up a sample to AnandTech.
Fusion-io (and others) have been doing precisely this (PCIe connected flash storage) for a number of years. They are currently producing modules with up to 6TB of storage.
The titles are basically "name of the SSD and its capacity - 4KB Random Write (QD32) Performance". The name of the SSD should change when you select a different SSD but every graph has the "4KB Random Write (QD32) Performance" attached to it.
Hi Kristian, a small suggestion: when talking about worst case IOPS you write that "The blue dots in the graphs stand for average IOPS just like before, but the red dots show the worst-case IOPS for every second." Ok, but I'd write it in the graph legend instead.
I'd suggest the following. Use FIO to do your benchmarking. It supports generating and measuring just about every load you'd care about. You can also use it in a distributed mode, so you can run as many tests as you have hardware to support, at the same time.
Second, don't use logarithmic axes on your charts. The drives you describe here take *huge* dropoffs in performance after their caches fill up and they have to start "working for a living". You are masking this performance drop by not using linear measures.
Third, divide up your time axis into (say) 60 second chunks, and show the min/max/95/99/99.9/99.9 latency marks. Most enterprise customers care about sustained performance and worst case performance. A really slow IO is going to hold up a bunch of other stuff. There are two ways out of that: Speculative IO (wait a little while for success then issue another IO to another device), or manage and interleave background tasks (defrag/garbage collect) very carefully in the storage device. Better yet, don't have the problem at all. The marketing stats on these drives have nothing to do with the performance they exhibit when they are subject to non-stop, mixed loads.
Unless you are a vendor that constantly tests precisely those loads, and ensures they work, stay working, and stay tight on latency.
Hi Kristian. I need to bother you with a question: do you think isit worth it to stick this SSD in a NAS? I have a ''fanless'' QNAP HS-210, 2 bay small form NAS, without drives for the moment, so in order to have a complete zero noise and time ''resistence'' to go for SSDs. But I was forgoten what was mentioned here "no wear leveling, no garbage collection'', so I'm wondering if in time the performances will decrease dramatically I'm thinking that the OS of NAS is not knowing to do such ''treatments'' over SSDs for maintaining performances, no? It's not in my intention to do operations over operations on NAS but I would like to know that my data will be ''safe'' and easy ''accesible'' over long time, OK? Very appreciated your oppinion. Thanks, Cristian
Ok, this doesn't clear anything up. If my manufacturer is lying about the IOPS of my SSD, how do I figure out the real value?
If they aren't lying about the 97,000 random read IOPS, how many megabytes per second is this for 4KB?
What exactly is the formula? You never elaborate on this in the article beyond senseless ramble.
First of all, how am I gonna know what the queue length even is? The way I see it on benchmark tests, low queue depths have lower MB/s than higher ones so this confuses the hell out of me.
But okay, I wanna know how my 840 Evo will perform random reads in the worst case scenario and QD1 seems to be the worst case scenario according to all the benchmark evidence.
In that case, I must time the QD by the latency. My latency I believe is 1 ms so my result is 1000. 1000 IOPS? Okay, so my 4KB random read speed will be 4 MB/s? That's nowhere close to the real result CrystalDiskMark shows.
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
31 Comments
Back to Article
hojnikb - Wednesday, September 3, 2014 - link
Looks like, they ain't doing Turbowrite on TLC models :)Kristian Vättö - Wednesday, September 3, 2014 - link
TurboWrite wouldn't make any sense on an enterprise drive because all that matters is sustained performance. Client drives are a different case because IO activity tends to happen in bursts, so having a fast buffer is beneficial (and it makes the drive look better in benchmarks).hojnikb - Wednesday, September 3, 2014 - link
Yeah i figured as much. Aside from being more suited for short bursts (as you'd said) its really great for marketing department, since they can slap magical 500MB/s+ speeds, that every uninformed buyer is after. 500MB/s sure sells a lot better than 250MB/s (or even less, if we're dealing with lower capacities) :)Spirall - Wednesday, September 3, 2014 - link
Thanks for the article. Would suggest to make a $/manufacturer declared write endurance comparison (PB writen until manufacturer write warranty end). If there's a chance (due to the difficult on testing time) also make a $/tested write life (PB writen until tested dead).Kristian Vättö - Wednesday, September 3, 2014 - link
$/endurance is certainly something I will be looking at in the future. This was more of a preview than a full review and in the full review I will have more graphs and comparisons to support the conclusions :)As for actual endurance testing, that is something I will not do. It took me two weeks of continuos writing to get the rated endurance to drop by 1% in the 845DC PRO, so it would take way too long to wait for the drive to die.
hojnikb - Wednesday, September 3, 2014 - link
And there is a good chance, that thing would still go strong after rated endurance would drop to 0% (unless its hardcapped to die after that).Essence_of_War - Wednesday, September 3, 2014 - link
Another excellent article, Kristian.A question/comment, though, do you think it would make more sense to report the stdev measurements as an "error bar" attached to each data point rather than an additional graph with a 2nd y-axis? I think that might be more compact without having to worry about having multiple y-scales to read. Then it might even be possible to plot an average + error bars data set for multiple different SSDs on the same axis w/o having to worry about which curve is the avg, which curve is the stdev, etc.
hulu - Wednesday, September 3, 2014 - link
Another way to present the standard deviation data would be to draw it in two average +/- stdev graphs above and below the average graph. This would better allow visualizing the actual values the average minus stdev has.Kristian Vättö - Thursday, September 4, 2014 - link
Thanks for the suggestion. I will certainly look into alternative ways to present the standard deviation. I need to redo the graphs anyway for our new suite, so while I'm at it I can see if there is a better way to present the data.Essence_of_War - Thursday, September 4, 2014 - link
Best luck then, I'm sure you'll figure out a good way to do it.LiviuTM - Wednesday, September 3, 2014 - link
Great article, Kristian.I enjoyed finding more about latency, IOPS and throughput and the relationship between them.
Keep up the good work :)
Chapbass - Wednesday, September 3, 2014 - link
I would like to echo this statement. Some of the heavy technical stuff makes my eyes glaze over at times, but this was so well written that I really got into it. Awesome article, Kristian.romrunning - Wednesday, September 3, 2014 - link
"The only difference between the 845DC EVO and PM853T is the firmware and the PM853T is geared more towards sustained workloads, which results in slightly higher random write speed (15K IOPS vs 14K IOPS)."Chart for PM853T shows 14K for Random Writes, so likely needs to be corrected.
Kristian Vättö - Wednesday, September 3, 2014 - link
Good catch, I was waiting for Samsung to send me the full data sheet for the PM853T but I never got it, so I accidentally left the 845DC EVO specs there. I've now updated it with the specs I have and with some additional commentary.JellyRoll - Wednesday, September 3, 2014 - link
The only issue with calculating performance as listed on the first page is that it assumes that the SSD works perfectly in all aspects. No ECC, no wear leveling, no garbage collection. None of these are true. Even without those factors no SSD will ever behave absolutely perfectly in every aspect at all times....anything but. That is why there is so much variation between vendors. It would be impossible to calculate performance using that method with an SSD in the real world.Kristian Vättö - Wednesday, September 3, 2014 - link
Of course real world is always different because the transfer size and queue depth are constantly changing and no SSD behaves perfectly. I mentioned that it is a hypothetical SSD and obviously no real drive would have a constant latency of 1ms or perfect transfer size scaling. The goal was to demonstrate how the metrics are related and it is easier with concrete, albeit hypothetical, examples.hrrmph - Wednesday, September 3, 2014 - link
I think you nailed it on the theoretical stuff (the relationships between the parameters), and presented it well (easy to understand).This has been bugging me for a while too, although I won't pretend to have gotten it figured out - it's just that I keep getting pickier in my lab notes and shopping specs about things like queue depth for a given size transfer for the specified performance. Leave any condition or parameter out, and the specs seem kinda useless. Leave everything in, and then I wonder which ones are pertinent to my usage scenarios.
Now, your explanation will have me questioning the manufacturer's motives for the selection of each unit of measure chosen for each listed spec. Who says this field of endeavor doesn't lead to obsession? :)
If any of the specs *are* pertinent to my usage scenarios, then I wonder which ones are *most* pertinent for which of my usage scenarios (laptop, versus general desktop, versus high powered workstation).
Any relationships that you discover or methods that you develop for the charts to help explain it better are most welcome. I know this is an over-simplification, but I would guess that most workstation users want to know:
- Is my drive limiting my performance (why did that operation stutter or lag, or why does it take so long - was it the drive)?
- Is there anything I can do about it that I can afford (mainly, what can I replace it with - new controller card, newer, better designed SSDs, better racks, cables, etc.)?
I am pleased that you are helping us out by further dissecting performance consistency / variation. I suspect that although SSDs are an order of magnitude (or more) better than HDDs at many tasks, the "devil is in the details," and that there is a reason that many SSD equipped machines still "hiccup," fairly frequently (although not nearly as often or as bad as HDD equipped machines). I also suspect that drive sub-systems are still one of the most common weaker links that is responsible for such hiccups.
I am particularly interested in the (usually) brutally difficult small file size tests. These tests seem to be able to bring even the best of machines to a crawl, and any device (ie: SSD) that can help performance on those tests seems to be very likely to be noticeable to the end user.
If you do indeed find "let downs" in performance consistency (or any other drive related performance spec), then maybe the manufacturers will work to improve upon those weaknesses until we get "buttery smooth" performance...
...or at least until we can definitively start looking at other sub-systems (compute, memory, I/O, etc.) to solve the hiccups.
My introductory courses were on 8086 computers running DOS. I don't remember them often stopping to think about anything... until I started "hitting" the disks heavily. The more things change, the more I suspect they stay the same ;)
So I keep allocating more of my budget to disks and disk sub-systems than anything else. AT's articles are thus *very* helpful in "aiming" that budget and I hope you have some revelations for us soon that show which products are worth the money.
iwod - Wednesday, September 3, 2014 - link
I have been wondering for a bit, why hasn't Enterprise switched to some other interconnect? Surely they could do PCI-E have have it directly connected to the CPU. ( Assuming they dont need those for GPU or other things ).And I have been Samsung being extremely aggressive with the web hosting market. Where Intel is lacking behind.
FunBunny2 - Wednesday, September 3, 2014 - link
"real" enterprise has been running SAS and FibreChannel for rather a long time. InfiniBand every now and again. To the extent that enterprise buys X,000 drives to parcel out to offices and such, then that's where the SATA drives go. But that's not really enterprise storage. Real enterprise SSD/flash/etc. doesn't have a list price (well, only in the sense that your car did) and I'd wager that not one of the enterprise SSD/flash companies (and no, Intel doesn't count) has ever offered up a sample to AnandTech.rossjudson - Thursday, September 4, 2014 - link
Fusion-io (and others) have been doing precisely this (PCIe connected flash storage) for a number of years. They are currently producing modules with up to 6TB of storage.Laststop311 - Wednesday, September 3, 2014 - link
Wish the consumer m2 drives would be released already. Samsung sm951 with pcie gen 3.0 x4 controller would be nice to be able to buy.tuxRoller - Wednesday, September 3, 2014 - link
All chart titles are the same on page five (performance consistency average iops).tuxRoller - Wednesday, September 3, 2014 - link
Actually, all the charts carry the same title, but different data.Kristian Vättö - Thursday, September 4, 2014 - link
The titles are basically "name of the SSD and its capacity - 4KB Random Write (QD32) Performance". The name of the SSD should change when you select a different SSD but every graph has the "4KB Random Write (QD32) Performance" attached to it.CountDown_0 - Wednesday, September 3, 2014 - link
Hi Kristian,a small suggestion: when talking about worst case IOPS you write that "The blue dots in the graphs stand for average IOPS just like before, but the red dots show the worst-case IOPS for every second." Ok, but I'd write it in the graph legend instead.
Kristian Vättö - Thursday, September 4, 2014 - link
It's something I thought about and can certainly consider adding it in the future.rossjudson - Thursday, September 4, 2014 - link
I'd suggest the following. Use FIO to do your benchmarking. It supports generating and measuring just about every load you'd care about. You can also use it in a distributed mode, so you can run as many tests as you have hardware to support, at the same time.Second, don't use logarithmic axes on your charts. The drives you describe here take *huge* dropoffs in performance after their caches fill up and they have to start "working for a living". You are masking this performance drop by not using linear measures.
Third, divide up your time axis into (say) 60 second chunks, and show the min/max/95/99/99.9/99.9 latency marks. Most enterprise customers care about sustained performance and worst case performance. A really slow IO is going to hold up a bunch of other stuff. There are two ways out of that: Speculative IO (wait a little while for success then issue another IO to another device), or manage and interleave background tasks (defrag/garbage collect) very carefully in the storage device. Better yet, don't have the problem at all. The marketing stats on these drives have nothing to do with the performance they exhibit when they are subject to non-stop, mixed loads.
Unless you are a vendor that constantly tests precisely those loads, and ensures they work, stay working, and stay tight on latency.
SuperVeloce - Thursday, September 4, 2014 - link
Great review... but dropdown menu for graphs annoys me. ughKristian Vättö - Thursday, September 4, 2014 - link
What do you find annoying in them? I can certainly consider alternative options if you can suggest any.grebic - Thursday, October 2, 2014 - link
Hi Kristian. I need to bother you with a question: do you think isit worth it to stick this SSD in a NAS? I have a ''fanless'' QNAP HS-210, 2 bay small form NAS, without drives for the moment, so in order to have a complete zero noise and time ''resistence'' to go for SSDs. But I was forgoten what was mentioned here "no wear leveling, no garbage collection'', so I'm wondering if in time the performances will decrease dramatically I'm thinking that the OS of NAS is not knowing to do such ''treatments'' over SSDs for maintaining performances, no? It's not in my intention to do operations over operations on NAS but I would like to know that my data will be ''safe'' and easy ''accesible'' over long time, OK? Very appreciated your oppinion. Thanks, CristianInds - Wednesday, February 4, 2015 - link
Ok, this doesn't clear anything up. If my manufacturer is lying about the IOPS of my SSD, how do I figure out the real value?If they aren't lying about the 97,000 random read IOPS, how many megabytes per second is this for 4KB?
What exactly is the formula? You never elaborate on this in the article beyond senseless ramble.
First of all, how am I gonna know what the queue length even is? The way I see it on benchmark tests, low queue depths have lower MB/s than higher ones so this confuses the hell out of me.
But okay, I wanna know how my 840 Evo will perform random reads in the worst case scenario and QD1 seems to be the worst case scenario according to all the benchmark evidence.
In that case, I must time the QD by the latency. My latency I believe is 1 ms so my result is 1000. 1000 IOPS? Okay, so my 4KB random read speed will be 4 MB/s? That's nowhere close to the real result CrystalDiskMark shows.