Update 7/14/2008 - Tomshardware redeems themselves with new flash versus hard drive energy efficiency tests
Lots of people are talking about Tomshardware’s latest claim that SSD (Solid State Disk) flash drives are less energy efficient than a conventional laptop hard drive. The problem is that the analysis is fundamentally shoddy.
Tomshardware’s Patrick Schmid and Achim Roos claims that a computer with a SanDisk SSD5000 SSD drive doesn’t last as long as the same computer running a conventional Hitachi 7K200 2.5″ 7200 RPM hard drive. We can pretty much throw this conclusion out the window because the SanDisk SSD5000 has a PEAK power consumption of 1 watt while the Hitachi 7K200 has an IDLE power consumption of 1.1 watt. So even if we assume that the test they ran was so flawed that it forced the SanDisk to operate at PEAK and permitted the Hitachi to operate at IDLE, the computer running the SanDisk SSD should still consume less power. But the Hitachi 7K200 actually has a PEAK power consumption of 3.2 watts so the odds are it consumed a lot more than 1.1 watt during the tests and the battery drain test run by Schmid and Roos are flawed.
It’s actually quite simple to explain how they might have botched the experiment. The SSD drives are capable of far more performance especially when it comes to random access. By having more storage IO performance, it’s very easy for the CPU and the rest of the system to work harder because they’re not idling as much waiting for data from the storage system. Whenever you’re doing a batter drain test, you have to keep the variables in check. The amount of work done by the two systems must be identical because cranking one system’s CPU to a higher utilization rate will easily cause it to consume more power and run the battery down. The ideal test for a battery drain test is to play back a movie using popular movie formats and movie software. This way, the computational workload on the CPU, graphics, and the storage system is identical across all the systems and you’re not unfairly stressing one system more than the other.
Schmid and Roos are so confident that they state “No, our results are definitely correct” and they have theories to explain why they’re think they’re right. On hard drive power consumption, the Tomshardware duo explains that for sequential access, hard drives should “not require much more than the idle power” because the mechanical actuators aren’t jumping back and forth. That’s an interesting hypothesis, too bad they never tested that theory before they exclaimed to the world that their conclusions are definitive.
Xbit Laboratories did test power consumption between idle, random access, and sequential access but they showed that this theory is wrong. In fact, power consumption between random access and sequential access is almost identical and in some cases, sequential access actually consumed more power. How can this be? Well with random access, the hard drive is putting out maybe 1 Megabyte/second of data which isn’t very much work for the I/O (Input/Output) logic on the storage device. With sequential access, the hard drive is putting out roughly 50 MB/sec which stresses the hell out of the IO logic on the hard drive. This additional power consumption in the IO logic is enough to offset or even exceed the power consumed by the mechanical actuators.However, Schmid and Roos proclaimed in bold subtitles that “Flash SSDs only Know Two Power States” and that SSDs are “active or idle”. Oh really? Is it possible they concluded this because they only tested two working states rather than testing three working states between idle, random access, and sequential access? I’ve been looking and analyzing power consumption characteristics for a long time and I’ve never seen IO logic with only two power states. IO logic like all microprocessors subtly increases in power consumption as you increase workloads. Tomshardware would have you believe that an SSD would consume 0.5 watts in idle and immediately jump to 2.9 watts as soon as you throw any work at it. So they’re claiming that an SSD pushing out data at 1 MB/sec would consume just as much power as it would at 100 MB/sec. This is a preposterous hypothesis and it is laughable.
Tomshardware only tested “load” (whatever that means since they didn’t disclose methodology) and idle power consumption. The problem is that they don’t factor in the fact that the SSDs are being forced to operate at above 100 MB/sec while the hard drive is only operating at around 50 MB/sec. In fact if you divide their Megabyte/second measurements by the wattage (Joules/second) measurements, you can actually get the megabytes per joule (unit energy) number. This is the number of megabytes transferred using a single joule of energy. I made an assumption and used their average sequential throughput divided by load power measurements and produced the following table showing how much more efficient the flash drives are if you account for the amount of work being done.
The proper way to do this test is to run all the devices at equal throughput. So you could simply play a DVD movie off the storage device at 1 MB/sec (8 megabits/sec) or an HDV stream at 3 MB/sec and measure the power consumption at these fixed storage workloads rather than assuming that the SSDs “only know two power states”.
The purpose of my analysis is not to say you should buy SSD drives for your laptop. I simply abhor bad science and bad analysis especially when it gets taken seriously. I’m actually not a big proponent of SSD drives in laptops yet because the economics aren’t there *YET* and the power consumption difference isn’t that great.
At this point in time, SSDs make sense for submarine duty in the US Navy where silence means survival. It might also make sense in ultraportable laptops where you don’t need that much capacity and the SSD saves space and some power. Companies like MSI are actually opting out of SSDs for their 10 inch “Wind” notebook because of the additional storage capacity and the power savings aren’t all that compelling. SSDs also make sense for server duty where random access performance is extremely important.
The reality is that the average SSD drive doesn’t save all that much power compared to a conventional 2.5″ hard drive and it’s quite possible that some 2.5″ hard drives are more energy efficient than some SSDs. But the analysis and conclusions drawn by Tomshardware are not to be taken seriously.