I’ve been planning to do some network testing and deploy some new storage for VMware vSphere in the home lab. My Synology NAS boxes (DS1513+ and DS1813+) have good performance but are limited to four 1GbE ports each, and my budget won’t allow a 10GbE-capable Synology this spring. [See below for a note on those $499 Synology systems on Amazon.]
Update 2017-04-30: About two weeks ago Synology released their newest 5- and 8-bay desktop NAS boxes, featuring 1 PCIe slot supporting either a 10GbE or an M.2 card. The DS1517+ (5-bay) is around $750, and the DS1817+ (8-bay) is around $1,000. Easier to afford, but probably late summer at the earliest for my credit card.
So of course, I built my own 10GbE-capable NAS system instead. Here’s what I used, and a bit of why.
With an Amazon gift card I was able to take some of the bite out of a Xeon-D motherboard’s cost, along with a few components around the house from previous Weird Stuff and flea market shopping runs, I built what should be a pretty viable storage server or a vSphere system with local storage and high speed network connectivity.
The Build:
- SuperMicro X10SDV-4C-TLN2F-O motherboard – Xeon D-1521 4-core, 6 SATA ports, dual 10GBase-T plus out-of-band NIC. ($490 at Amazon today)
- Corsair Vengeance LPX 64GB DDR4-2400 RAM (4x 16GB PC4-19200 DIMMs). (2x16GB for $230 at Amazon)
- IcyDock ToughArmor 6 Bay 2.5″ SATA backplane cage. ($83 at Amazon)
- 4x 256GB LiteOn SSD, ($130-250 each at Amazon)
- Four-pack of RAIDMAX 80mm case fans, $8 at Fry’s Electronics (promo code deal, regular price $13), similar to this $14 Apevia kit on Amazon.
- Silverstone Milo ML03 HTPC case, $25 at Weird Stuff ($72 at Amazon)
- Thermaltake TR2 430W power supply, probably about $20 from a thrift store ($35 at Amazon)
- USB 2.0 header to USB 3.0 connector, plus USB 3.0 connector to dual USB port adapter; A direct 10-pin to dual USB 2.0 would be better, (about $8 at Amazon)
- Two Hyundai 16GB USB 2.0 USB flash drives ($5 each)
Total build price on Amazon as of April 10, 2017: $1,692 (plus taxes)
Updated build price on Amazon as of January 2, 2019: $1,365 (plus taxes)
(Motherboard and memory prices went up in the last two years, SSD prices went down, and I switched to Sandisk Ultra Flair USB drives due to availability.)
Not all components were purchased at the listed prices, or even from Amazon; some items came from thrift shops, and others came from Central Computers, a local systems and components shop I’ve frequented for almost two decades, or Weird Stuff, a local recycler/reseller who’s also been on my weekly runs since the late 90s.
Had I been buying everything new at retail/Amazon, I probably would have made some slightly different choices. For example, the Samsung Evo 500 SSD would be cheaper than the LiteOn model I had on hand, if I had to order it new. But you work with what you have, and it often works out.
Component notes:
The case was $25 at Weird Stuff Warehouse, a local “junk shop” and refurbisher of computers and components. I probably got the power supply there or at a local Goodwill. And while the case is compact, it takes a full size ATX power supply, a 5.25″ half height exposed bay (hence the IcyDock), as well as two 3.5″ drive bays. It also supports four 80mm case fans on the right side if you need more cooling, which I did.
The SuperMicro mini-ITX board was similar to some I discussed on the antsle post in March. I chose this one because of onboard 10 Gigabit Ethernet support as well as it being available for same-day delivery on a Sunday through Amazon Prime (I felt motivated, and didn’t see anything under $500 in stock locally). With four DIMM slots it can handle up to 64GB of relatively-affordable non-ECC RAM, or 128GB of slightly-less-affordable ECC RAM. There’s also an m.2 slot on-board for additional storage, with support for NVMe devices.
While the board is fanless, it apparently gets warm when you’re running it with 4 dimms and the cover on the case. I was trying to install Corral and the system kept shutting down. For this reason, I had to run out to Fry’s and pick up the fans. These fans don’t light up or have variable speed switching, but for $8 with the Fry’s promo code (regular price $12.99) they move the air. The case has a top vent and a magnetic filter cover, and could porbably support a slim fan, but I decided to go with the side design. I’m only using two of the fans and that kept the temperature in check.
Memory choice was based on a lack of DDR4 RAM around the house, and what was available at Central Computers when I began the project. I later ordered a second kit from Amazon to max out the board. Note that for FreeNAS/Corral, you will want to seriously consider ECC RAM, and I may switch it out for a smaller amount of ECC later if I do stick with Corral.
I have a small collection of drive backplane enclosures around the house, but since this board only has 6 SATA ports, I chose not to go with the 8-port backplane. I may put that in an HP Microserver with a SAS/SATA card instead. This IcyDock takes two standard 4-pin Molex power plugs and six individual SATA cables. No cable converters or aftermarket cards needed.
For storage, the SSDs were a bargain lot I got a year or two ago. The USB flash drives were a random deal at Fry’s, and at $5 they’re cheap enough to mirror and replace as needed. I keep a box of spare flash and SD cards around just in case, but most of what I have is overkill for a compact OS like vSphere or FreeNAS. And with the flash drives inside the case, there’s nothing at risk for snapping off or breaking a port out of the case if the lab cat gets overly ambitious.
Perhaps obviously, with this complement of components, the server will not be a full-scale general-purpose NAS. I could affordably stock the system with 6x 1TB spinning disk, or get some 2TB SSDs in there, but for my needs it would not be cost effective.
Where do we go from here?
I will be configuring the storage (4x256GB SSD, 2x 1TB HDD) and putting the unit on my “rack” desk in the home office (where my Synology arrays and Meraki switches live) soon. This will then become a testbed (at least for a few weeks) for a couple of 10GbE switches I have in house to test.
The ASUS XG-U2008, Meraki MS42P, Netgear M5300-28G3, and Cisco Small Business SG500XG-8F8T are queued up for testing in terms of sound level, power use, trunking, and network throughput. I’ll be putting them together with some Intel and ASUS 10GbE PCIe cards to see how they work out. Stay tuned for more in the next month or two.
How is your homelab 10GbE effort going? Have you found some good deals, or mislabeled/rebranded cards/switches, that keep your budget down and your throughput up? Share in the comments if you have.
An afterword about really cheap gear on Amazon
Note: I’ve noticed some $499 “Used” Synology NAS units on Amazon.
You’ll notice in this case, for a DS3612xs that Synology themselves offer refurbished for $1,500, a brand new seller with lots of accent marks in the text of their item status would have you believe they can sell you one for $499 with free shipping. I also saw some of this activity around the antsle server I discussed in March, where an unestablished seller with lots of accents in their comments claimed to have gear for a third of the going price.
Well, believe it or not, it’s a well-known scam. The “contact us prior” bit, that is. Synology refurbs are good stuff. It appears that these “sellers” will try to get you to send money outside of Amazon (i.e. Western Union, where you have little or no recourse), and of course they won’t ship you anything once they’ve got your money.
So if it seems too good to be true, it probably is. Avoid any Amazon sellers who ask you to contact them before making a purchase. Obviously, if you have product questions, or after-the-sale inquiries, contact them through Amazon’s mechanisms. But a reputable seller with actual product to sell will not ask you to go around Amazon.
Disclosure: All system components were purchased as a member of the general public with my own funds. Some items, including the Netgear switch mentioned, were purchased at absurd discount at used gear stores as mentioned above. As such, not all deals can be reproduced on demand by the general public (think garage sales), but there was no special consideration requested or received from any of the manufacturers or vendors.
I do work for Cisco in my day job. However, unless I call out day job influence, my day job has no influence on my content around non-day job topics. The Cisco Small Business switch was provided by the CSB team several years ago (before I worked for Cisco). The Meraki switch was purchased on eBay last year, and I’m using an Employee Purchase Program license for it (migrated from the MS22 I got internally).
Pingback: Quick Take: Antsle coming out with Xeon-D models with 10GbE in December | rsts11 – Robert Novak on system administration
Pingback: Looking ahead into 2018 with rsts11 | rsts11 – Robert Novak on system administration
Pingback: Upgrading the HPE Microserver Gen 8 and putting it into service | rsts11 – Robert Novak on system administration