Overkill in the rsts11 lab workshop – a homelab update for 2017

After being chosen as a VMware vExpert for 2017 this month, I was inspired to get working on refreshing my vSphere “homelab” environment despite a busy travel month in late February/early March. This won’t be a deep technical dive into lab building; rather, I just wanted to share some ideas and adventures from my lab gear accumulation over the past year.

As a disclosure, while I do work for Cisco, my vExpert status and homelab building are at most peripherally-connected (the homelab at home connects to a Meraki switch whose license I get an employee discount on, for example). And even though I’m occasionally surprised when I use older higher end Dell or HP gear, it’s not a conflict of interest or an out-of-bounds effort. It’s just what I get a great deal on at local used hardware shops from time to time.

The legacy lab at Andromedary HQ

Also read: New Hardware thoughts for home labs (Winter 2013)


Stock Photo of a Dell C6100 chassis

During my last months at the Mickey Mouse Operation, I picked up a Dell C6100 chassis (dual twin-style Xeon blade-ish servers) with two XS23-TY3 servers inside. I put a Brocade BR-1020 dual-port 10GBE CNA in each, and cabled them to a Cisco Small Business SG500XG-8F8T 10 Gigabit switch. A standalone VMware instance on my HP Microserver N40L served the vCenter instance and some local storage. For shared storage, the Synology DS1513+ served for about two years before being moved back to my home office for maintenance.

The Dell boxes have been up for almost three years–not bad considering they share a 750VA “office” UPS with the Microserver and the 10Gig Switch and usually a monitor and occasionally an air cleaner. The Microserver was misbehaving, stuck on boot for who knows how long, but with a power cycle it came back up.

I will be upgrading these boxes to vSphere 6.5.0 in the next month, and replacing the NAS for shared storage throughout the workshop network.

The 2017 Lab Gear Upgrades

For 2017, two new instances are being deployed, and will probably run nested ESXi or a purpose-built single-server instance (i.e. an upcoming big data sandbox project). The two hardware instances each have a fair number of DIMM slots and more than one socket, and the initial purchase for each came in under US$200 before upgrades/population.

You may not be able to find these exact boxes on demand, but there are usually similar-scale machines available at Weird Stuff in Sunnyvale for well under $500. Mind you, maxing them out will require very skilled hunting or at least a four figure budget.


CPU/RAM cage in the HP Z800

First, the home box is a HP Z800 workstation. Originally a single processor E5530 workstation with 6GB RAM, I’ve upgraded it to dual E5645 processors (6-core 2.4GHz with 12MB SmartCache) and 192GB DDR3 ECC Registered RAM, replaced the 750GB spinning disk with a 500GB SSD, and added two 4TB SAS drives as secondary storage. I’ve put an Intel X520 single-port 10GbE card in, to connect to a SFP+ port on the Meraki MS42P switch at home, and there are two Gigabit Ethernet ports on the board.


CPU/RAM cage in Intel R2208 chassis

And second, the new shop box is an Intel R2208LT2 server system. This is a 2RU four-socket E5-4600 v1/v2 server with 48 DIMM slots supporting up to 1.5TB of RAM, 8 2.5″ hotswap drive bays, and dual 10GbE on-board in the form of an X540 10GBase-T dual port controller.  I bought the box with no CPUs or RAM, and have installed four E5-4640 (v1) processors and 32GB of RAM so far. There’s more to come, since 1GB/core seems a bit Spartan for this kind of server.

There’s a dual 10GbE SFP+ I/O module on its way, and this board can take two such modules (or dual 10GBase-T or quad Gigabit Ethernet or single/dual Infiniband FDR interfaces).

The Z800 is an impressively quiet system–the fans on my Dell XPS 15 laptops run louder than the Z800 under modest use. But by comparison, the Intel R2208LT2 sounds like a Sun Enterprise 450 server when it starts up… 11 high speed fans warming up for POST can be pretty noisy.

So where do we go from here?

Travel and speaking engagements are starting to pick up a bit, but I’ve been putting some weekend time in between trips to get things going. Deploying vSphere 6.x on the legacy lab as well as the new machines, and setting up the SAN and DR/BC gear, will be spring priorities, and we’ll probably be getting rid of some of the older gear (upgrading the standalone vCenter box from N40L to N54L for example, or perhaps moving it to one of the older NUCs to save space and power).

I also have some more tiny form factor machines to plug in and write up–my theory is that there should be no reason you can’t carry a vSphere system anywhere you go, with a budget not too far above a regular-processor-endowed laptop. And if you have the time and energy, you can do a monster system for less than a high-end ultrabook.


Disclosure: Links to non-current products are eBay Partner Network links; links to current products are Amazon affiliate links. In either case, if you purchase through links on this post, we may receive a small commission to pour back into the lab.


Lowered Expectations – How Low Can Your Laptop Go?

[An Interop Aside: I visited with a couple of vendors at Interop who are sending some gear for me to explore. I’m holding off on their coverage until that happens, although another summary post may be forthcoming.]

I’m a big laptop fan. Afficionado, not cooler, mind you. It’s a problem, especially since my recent rebuild acquisitions and components are blocking the fireplace at the moment.

There’s been a disturbing trend over the last couple of years, whereby laptop manufacturers decide to move more toward the netbook specifications for memory (and often storage), rather than to the state of the art for the current generation of laptop processors. I was commiserating with my friend John Obeto about this recently.

For a couple of months now, you’ve been able to order a Dell Precision 7000-series laptop with 64GB of RAM. That’s twice as much as many desktops can handle today. And even if you don’t have room for four DIMM slots in your laptop design, DDR4 16GB SODIMMs are very affordable and readily available even at retail. So there’s really no reason for a 13″ or larger laptop to have an 8GB limit.

But it is the way of the world, for most lightweight laptops these days. Even Dell’s remarkable XPS 13 9343 maxed at 8GB – the 9350 model this year has a 16GB option but it’s online order only (and in the $2000 range as I recall). Continue reading

A word on PoHo coverage for 2016

Greetings, readers.

As you may have noticed, rsts11 has had limited content over the past year. Day Job(tm) has taken precedence, and while we’ve been creating content, it’s been for the Day Job.

On the one hand, we’ve managed to completely avoid conflicts of interest between rsts11 and the Day Job, but there’s a substantial backlog to come out over the upcoming months, as well as some new industry observations and technology updates.

There are also a couple of eBooks in the works that should be available by early summer. We’ll have one on big data and one on meeting hygiene, and possibly some more.

Feel free to follow me personally on @gallifreyan and the blog update feed at @rsts11 on Twitter, and subscribe to get the latest updates here. If there are topics I’ve mentioned through social media or at events that you’d like to read more about, feel free to make suggestions in the comments here.

What planet are we on? (The Third) — the RSTS11 Interop preview

Greetings from Fabulous Las Vegas, Nevada. For the third year, with apologies to Men Without Hats, I’m back in the Mandalay Bay Convention Center for Interop. 

This week, I’m actually a man without hats as well. My Big Data Safari hat is in my home office, and my virtual Cisco ears are back at home as well, next to the VPN router that was powered down before I headed for the airport. (Alas, after moving from Disney to Cisco, I lost the theme park discounts and the epic mascot reference.)

What are you up to at Interop this year, Robert?

So why am I back at Interop, when a dozen conference calls a day could have been in the cards for me this week? 

My readers, my fans, and my groupie all know that I’ve been a fan of the Psycho Overkill Home Office (POHO) for quite a while, going back to when I had a 19-server, 5-architecture environment with a 3-vendor network in my spare bedroom. Today it’s about 12 servers, all x64 (Shuttle, Intel, Cisco, Supermicro, Dell, and maybe another secret brand or two), and technically a 5-vendor network, but the idea is similar enough.

And having built a couple of startups up from the under-the-desk model to a scalable, sustainable production-grade infrastructure, the overkill in my home office and labs has led to efficient and effective environments in my workplaces. 

This week I’m taking a break from my usual big data evangelism and the identity aspects of working for a huge multinational juggernaut. It’s a bit of a relief, to be honest; earlier this month I attended my first event in 10 months as a non-booth-babe, and now I’m getting to focus on my more traditional interests. 

What’s on the agenda this week?

I’m looking forward to return visits to the folks at Sandisk, Opengear, and Cradlepoint. Cradlepoint was the first interview I did two years ago at Interop 2013, and I’ve been a customer on my own for many years; Opengear was a presenter at Tech Field Day Extra at Cisco Live 2013; and I last talked with Sandisk at Storage Field Day 5 about a year ago, as well as having been a Fusion-io customer at a previous job. 

I have a couple of other meeting requests out, so we may hear from a couple of other POHO/SOHO/ROBO/lab staples, and I’ll at least be dropping by their booths in the Interop Expo to see what’s new. 

While I’m only recording this week for notetaking convenience, I am starting to ponder what to do about the podcast I’ve been thinking about for a couple of years. So maybe I can pull in some interesting people from time to time… last night’s conversation over Burger Bar shakes with Chris Wahl and Howard Marks probably would have been fodder for several podcasts alone (and I don’t think any of us even had any alcohol!).

And seeing as a number of my friends are presenting this year, including Chris and Howard, I’ll be trying to make my way to their sessions (although there’s a LOT of overlap, and triple-booking isn’t uncommon… there’s a lot more than the Expo floor to experience at Interop, as always).

So where do we go from here?

If you’re at Interop, who are you looking forward to seeing/hearing/heckling/buying drinks for? (And if you’d like to meet up, catch me on Twitter at @gallifreyan.) If not, check out the exhibitor list at interop.com/lasvegas and let me know who you are curious about on that list. 

New hardware thoughts for home labs (Winter 2013)

It’s been almost two years since I wrote my first home lab post, on the occasion of rolling a Shuttle SH67H3 VMware server. Since then, I’ve rambled on Twitter about a lot of other options, and figured I would bring some of them to your more-easily-searched-for attention.

I will update this post in the near future – most recent update 2013-12-12 – so you can look (probably at the bottom) for new details and references.

Disclosure: I’m not paid or coerced to promote the items in this post. Anything I own below was bought with my own money. Most of it probably will not blend. Any references to vendors or manufacturers are based on my experience and not any consideration from the company.

Many of the links are to Amazon.com, and if you buy through them, I get a small commission credit to spend on more coffee gear or some of the same things. I appreciate your support and suggestions.

My lab cluster today

I recently bought two batches of rackmount servers at absurd prices. We’re talking less-than-the-memory-was-worth prices. For now, I have an NEC Express 5800/120Rh-1 (dual E5405/16GB) and a HP DL365 G1 (dual-core Opteron 2214HE/16GB) running 5.5. vCenter Server is running on my NUC i3 box out of convenience. When I get some more PC2-5300F RAM, I’ll switch out that Opteron for another Xeon to get a bit more consistency.

The downside to this environment is that it’s noisy and a bit power-hungry. At rest, the two servers use about 400W. So until I upgrade the UPS, I’m a bit stuck on that level of server.

But the upside is that the two servers as configured cost less than I spend on coffee in a month at home. And my lab is in a location that isn’t as sensitive to power load or noise as my home office might be.

Using a Dell Poweredge C6100 for dense rackmount computing

There are a lot of 1u and 2u rackmount servers out there on Craigslist, eBay, Weird Stuff, and such venues. I’ve picked up various HP boxes for chump change and scrounged for memory, so it is an option. You can probably get a dual socket 8-core server (DL160, DL360, DL365, DL380, DL385) with some memory and drive trays for under $100 until you run out of power outlets. If your tolerance for power draw and noise allow, that’s definitely a cost-effective way to go.


There are also a lot of Dell C6100 “blade” servers (pictured above) out there as well. These are 2u enclosures with up to 4 two-socket nodes. Each blade can take 12 DIMMs (up to 192GB), two quad or hex core Xeon processors, and 3 LFF 3.5 drives or 6 SFF 2.5 drives (SATA, SAS, SSD). And from what I’ve read, you can run four dual-L5420 blades at about 300W.

I’m seeing these priced at around $750 for a two-L5520-node config, or a four-L5420-node config, with minimal RAM. You can find a four-L5520-node config for around $1k, or you can add extra nodes later. ServeTheHome has a thread on community update findings, including fan improvements and internal USB.

I don’t know what the noise level is out of the box, but hopefully one of my readers can chime in. Or I may pick one up next month and come back with an update.

Ye Olde HP Proliant Microserver… And Ye Newe Microserver

I have a Proliant Microserver N40L in my environment. It, and its siblings N36L and N54L, are classic home lab servers, with secret BIOS tweaks and undocumented memory upgrades and a $200-300 price tag. Much like the NUC, they are, but perhaps a bit less processing power and a lot more expandability.

Microserver Gen 8

Well, HP released their Microserver Gen 8 this summer, with two dual-core Pentium processor options. One option has a G1610T 2.3GHz processor, and the other has a G2020T 2.5Ghz processor; there’s even a stackable 8-port switch to match. You still get four non-hot-plug SATA bays; the new ones offer a glitzier front door and a laptop-size optical drive bay. You also get dual gigabit Ethernet and a dedicated iLo port.

The price has gone up with the specs; you’re looking at $450-500 for the base 2GB/250GB system, plus your upgrades, so probably $700 with 16GB of RAM.

Be sure not to purchase the Windows Server bundles (unless you’re into that sort of thing). The Microserver Gen 8 shows up in bundles between $700-1200 with various Windows licenses included, and if you’re throwing your own OS on afterward, there’s no reason to shell out the extra money.

NUC NUC… not again…

Intel has added new Next Unit of Computing (NUC) models to their line, with 4th generation i3/i5 processors. There’s an i3-4010U model and an i5-4250U model available. Perhaps obviously, they’re no longer fanless or silent, but probably quieter than the options above.

Wilson Canyon NUC with USB

You still need to add your power cable, some laptop memory (8GB or 16GB depending), an mSATA module if you want internal storage, and a flash drive to boot from. So you’re probably looking at about $600 for a complete system, give or take. But if space is of an essence, and your workloads can handle 16GB dual core modules, this is a great option.

As an aside, Intel has 4th generation NUCs with support for an internal 2.5″ drive. These don’t seem to be as commonly available, but it’s something to watch for if you need more internal storage.

A surprising contender – Dell’s Inspiron 660 desktop

I was having an exchange on Twitter with someone looking for options with Gen 3 PCI Express for virtualization lab use. He ended up getting an Inspiron 660 desktop, which has more convenient expansion options than pretty much everything above.

The i5-3340 model with 8GB of RAM comes in under $600 on Amazon (you can buy it directly from Dell but might get quicker delivery from Amazon). You should be able to load it up with 16GB of RAM, and you can get 4x and 6x SATA (and 4x SAS) drive bay inserts to get dense 2.5″ drive deployments. Probably won’t need that DVD burner on a hypervisor platform, will you?

What else can I read about home lab options?

I’m glad you asked. One thing that pushed me to write this post was Chris Wahl’s update on his home lab. He’s moving to Haswell, and building out a well-optimized lab. He’s an avid advocate of remote management, so definitely take a look at his board selection if you need remote control of your server.

Simon Seagrave at TechHead has a lengthy write-up on the Microserver Gen 8 that’s worth a look if you’re leaning that way.

2013-12-12: Erik Bussink has built a compact lab with the Shuttle XH61V that finds a happy medium between my Shuttle and NUC builds.

2013-12-12: A friend on Facebook reported in with Benjamin Bryan’s blog about installing a Xeon E3 in the HP Microserver Gen 8. This may be the best reason to go with the low-end G1610T model.

2014-01-14: Greg Schulz (@storageio on Twitter) has a new post today on some of his recent discoveries and acquisitions. Check out Dell Inspiron 660 i660, Virtual Server Diamond in the rough? for a surprising choice for virtualization.

If you’ve written a blog post about sub-$1k home lab servers, feel free to let me know and I’ll try to get you added to this list. I’m happy to exchange links and spread the joy of home lab adventures.