Western Digital RE4-GP

Western Digital released their lower-power “Enterprise” RAID Edition (The RE-GP series) Serial ATA (SATA) drives some time ago, and the 1.5 and 2TB versions (RE4-GP series) last year. They were found to have an issue with certain RAID controllers and received much bad press as a result.

It’s now more than a year on and they have newer firmware, which has been generally well received. I also was toying with reducing the energy footprint of my home-business mass storage needs (not to mention to improve its reliability and performance) so I decided to give them a whirl in my newest server build which has eight Western Digital 2TB RE4-GP's» in RAID-6.

What makes it green?

Western Digital (WDC) has had a “Green” variant of their desktop drive for some time and the key difference is that it nominally runs at a lower rotational speed (5400RPM versus 7200RPM), though in reality it varies the spin rate with load. The RE version follows ostensibly the same idea but incorporates the other RAID-focussed features, such as time-limited error recovery» and a tolerance for vibration.» The area that gives one pause before choosing a drive with a lower rotational speed is it may have a direct impact on the drives data transfer performance. The drive heads can only pass over the physical locations where data is stored as fast as the surface moves beneath them and this translates directly to the rate at which data can be sent to the host. Further, if you want to read data that’s on the other side of the platter, you have to wait for it to arrive – as data density capabilities have increased, so has the volume of data that is more than half a revolution away.

Lower rotational speed has a direct impact on the power required to spin the platters but it comes at the cost of performance. You could go install the 7200RPM SATA drives, or even go with Serially Attached SCSI (SAS) to get 10,000 and 15,000RPM drives – but if they spend any time idle, or less than say 50% busy, you may be wasting energy on it. And they cost more, too.

Most of the literature published by WDC doesn’t explicitly state the range of rotational speeds the RE4-GP drives operate at and most reviewers online either guess a value or have some way of measuring the speed and end up at 5400RPM along with comments like “I’ve not seen it vary with load”. This is disappointing but then I stumbled across a WDC Knowledge Base page which supports the notion that the drives will vary their spin rate between 5400 and 7200RPM. This encouraged me and the fact that all the nay-sayers comments were from last year, and thus probably on old firmware, persuaded me to spin up some RE4-GP.

The Setup

I’ve long been a proponent of home-built machines – you get what you want, sometimes for less cash, with no vendor lock-in. Sometimes, however, I just can’t be bothered with it especially when I just want it to work. In superseding my trusty VMware/NAS host I decided to do something a bit more off-the-shelf. Supermicro and Dell were the obvious candidates for me. I was leery of Dell after reading about vendor lock-in» issues with their RAID controllers. However they didn’t count on a huge customer backlash and quickly backtracked. Now it’s worth mentioning that Dell does offer very good support on the hard drives it supplies, but sometimes I feel that vendors lose sight of the “I” in RAID. Inexpensive.

In the end what sold me, after confirming the current firmware wasn’t vendor crippled, was that Dell had introduced the PowerEdge R515: a dual AMD Opteron box with room for lots of memory and, more importantly, had a 12-drive chassis with an option for two more internal drives. I may not need all of them now, but I like having options down the road. Furthermore, they had some huge discounts for Small Business and for the second processor. It was a bargain. And free shipping too. I ordered it after forcing myself to spend a week considering whether it was a good idea or not.

I had drive trays already and I ordered the front drives from Amazon.com. I kitted the machine out with two 2.8GHz Opterons and 32GB of DRAM. The Dell PERC H700 Raid Controller I ordered with 1GB of NVRAM. Basically, this thing should fly. I ordered two internal 10k RPM SAS drives, from Dell, configured as RAID-1 for the boot device and loaded the eight RE4-GP’s in the front, configured as RAID-6.

This setup has VMware ESXi 4.1 installed on the boot volume, hosted on the internal drives, and a combination of VMFS volumes and LUNs mapped directly into a client VM on the RE4-GP drives.

How hungry are they?

Western Digital specifies the RE4-GP and it’s 7200RPM brother power consumption as follows:

I added the difference column. It shows quite a drop. On a machine with ten drives that would mean at idle the GP uses 44W less than its counterpart and under load the difference is 39W. That’s 1 kWh per day. Where I live, that’s US$75 a year. Add in that the GP model costs less than its friend, there is a decent saving to be had.

Compare this to old NAS where it’s pulling about 150W just for hard drives when busy – and even when idle, they pull more than 110W (whole machine draws about 300 to 350W total). My replacement machine is looking good. In fact, the new Dell servers brag about having more efficient power supplies than previously and include power monitoring sensors, so I can tell you that before I put the RE4-GP drives in it was using a under 200W when I taxed both CPUs. This went up to just over 200W after I inserted them and now, whilst doing some file transfers from another host, the load is hovering at about 230W.

Dell PowerEdge R515 Power Graph

In the graph it is very obvious when the drives are idle or under load. Overall, I’m better off by about 120W. 2.8 kWh per day. About $200 a year. Tangible.

Performance

The crux of the story, however, is what is the penalty of this power reduction? Well the host itself (discounting the hard drives) is using about the same amount of power as the older machine but has more cores (twelve versus four), more memory (32GB DDR3 versus 8GB DDR2) and a better memory management architecture (Opteron memory bandwidth scales with processors and this is a dual-chip board, the older machine was a Core2 Quad chip which is not known for its memory abilities). So we have lots more compute power. What about its storage?

The older machine uses 14 500GB drives (mostly RE2 but some RE3 as they’ve been replaced) drives in RAID-6 on an Areca controller with 1GB of cache memory. This configuration has usually done moderately well, easily reaching 150MBytes/sec on sequential reads once I’ve made sure it’s not fetching from the cache. By itself this is not a useful indicator of overall performance, but indicative of the ability to extract data from the platters which is, as mentioned above, a function of (to a large extent) rotational speed – my minor concern with the RE4-GP series.

This new setup has a major difference from the older one: fewer spindles. Fewer drives means that the load of reading and writing is spread across less hardware. With RAID-5 and RAID-6 setups this decreases some of the work performed by the RAID system when writing (the read-modify-write cycle) but, during reading especially, it means potentially that there is less bandwidth available.

Imagine my surprise, then, when I wrote out a 10 gigabyte file in under a minute. 200MBytes/sec. And reading was no slouch – from the cache it reached 600MBytes/sec at one point (The PERC H700 is a PCI-Express x8 card, so 8Gbits/sec is the physical maximum it can do – 600MBytes/sec is not too far from that limit!) and larger sequential reads ran between 150 and 200MBytes/sec. Note: Why would reads be slower?» Very respectable considering this is from within a guest virtual machine, too. For what it was worth the results were comparable with various file-system types, but ext4 and xfs are what I generally use with Linux. I may later post more comprehensive data from bonnie++.

Final word

It’s only been a handful of days and I’ve not really exercised the new drives, or the new machine, but I’m happy with it so far. It uses less power than the machine it supplants, gives more compute resource, more storage space, more expandability and any concerns I had over the impact of using the low-power version of the hard drives and their indeterminate-but-probably-mostly-less-than-7200RPM rotational speed are diminished.

VMware ESXi Disk Performance Graph

Yes, I’m happy.

Errata

I have posted some post-publication thoughts on the read performance noted above.