Overkill in the rsts11 lab workshop – a homelab update for 2017

After being chosen as a VMware vExpert for 2017 this month, I was inspired to get working on refreshing my vSphere “homelab” environment despite a busy travel month in late February/early March. This won’t be a deep technical dive into lab building; rather, I just wanted to share some ideas and adventures from my lab gear accumulation over the past year.

As a disclosure, while I do work for Cisco, my vExpert status and homelab building are at most peripherally-connected (the homelab at home connects to a Meraki switch whose license I get an employee discount on, for example). And even though I’m occasionally surprised when I use older higher end Dell or HP gear, it’s not a conflict of interest or an out-of-bounds effort. It’s just what I get a great deal on at local used hardware shops from time to time.

The legacy lab at Andromedary HQ

Also read: New Hardware thoughts for home labs (Winter 2013)


Stock Photo of a Dell C6100 chassis

During my last months at the Mickey Mouse Operation, I picked up a Dell C6100 chassis (dual twin-style Xeon blade-ish servers) with two XS23-TY3 servers inside. I put a Brocade BR-1020 dual-port 10GBE CNA in each, and cabled them to a Cisco Small Business SG500XG-8F8T 10 Gigabit switch. A standalone VMware instance on my HP Microserver N40L served the vCenter instance and some local storage. For shared storage, the Synology DS1513+ served for about two years before being moved back to my home office for maintenance.

The Dell boxes have been up for almost three years–not bad considering they share a 750VA “office” UPS with the Microserver and the 10Gig Switch and usually a monitor and occasionally an air cleaner. The Microserver was misbehaving, stuck on boot for who knows how long, but with a power cycle it came back up.

I will be upgrading these boxes to vSphere 6.5.0 in the next month, and replacing the NAS for shared storage throughout the workshop network.

The 2017 Lab Gear Upgrades

For 2017, two new instances are being deployed, and will probably run nested ESXi or a purpose-built single-server instance (i.e. an upcoming big data sandbox project). The two hardware instances each have a fair number of DIMM slots and more than one socket, and the initial purchase for each came in under US$200 before upgrades/population.

You may not be able to find these exact boxes on demand, but there are usually similar-scale machines available at Weird Stuff in Sunnyvale for well under $500. Mind you, maxing them out will require very skilled hunting or at least a four figure budget.


CPU/RAM cage in the HP Z800

First, the home box is a HP Z800 workstation. Originally a single processor E5530 workstation with 6GB RAM, I’ve upgraded it to dual E5645 processors (6-core 2.4GHz with 12MB SmartCache) and 192GB DDR3 ECC Registered RAM, replaced the 750GB spinning disk with a 500GB SSD, and added two 4TB SAS drives as secondary storage. I’ve put an Intel X520 single-port 10GbE card in, to connect to a SFP+ port on the Meraki MS42P switch at home, and there are two Gigabit Ethernet ports on the board.


CPU/RAM cage in Intel R2208 chassis

And second, the new shop box is an Intel R2208LT2 server system. This is a 2RU four-socket E5-4600 v1/v2 server with 48 DIMM slots supporting up to 1.5TB of RAM, 8 2.5″ hotswap drive bays, and dual 10GbE on-board in the form of an X540 10GBase-T dual port controller.  I bought the box with no CPUs or RAM, and have installed four E5-4640 (v1) processors and 32GB of RAM so far. There’s more to come, since 1GB/core seems a bit Spartan for this kind of server.

There’s a dual 10GbE SFP+ I/O module on its way, and this board can take two such modules (or dual 10GBase-T or quad Gigabit Ethernet or single/dual Infiniband FDR interfaces).

The Z800 is an impressively quiet system–the fans on my Dell XPS 15 laptops run louder than the Z800 under modest use. But by comparison, the Intel R2208LT2 sounds like a Sun Enterprise 450 server when it starts up… 11 high speed fans warming up for POST can be pretty noisy.

So where do we go from here?

Travel and speaking engagements are starting to pick up a bit, but I’ve been putting some weekend time in between trips to get things going. Deploying vSphere 6.x on the legacy lab as well as the new machines, and setting up the SAN and DR/BC gear, will be spring priorities, and we’ll probably be getting rid of some of the older gear (upgrading the standalone vCenter box from N40L to N54L for example, or perhaps moving it to one of the older NUCs to save space and power).

I also have some more tiny form factor machines to plug in and write up–my theory is that there should be no reason you can’t carry a vSphere system anywhere you go, with a budget not too far above a regular-processor-endowed laptop. And if you have the time and energy, you can do a monster system for less than a high-end ultrabook.


Disclosure: Links to non-current products are eBay Partner Network links; links to current products are Amazon affiliate links. In either case, if you purchase through links on this post, we may receive a small commission to pour back into the lab.


Rough cut: HP Moonshot and CEO Meg Whitman at Nth Symposium 2013

I gotta say the withdrawal symptoms from daily Disneyland visits are getting milder, but I’m home from a week in Anaheim for HP Storage Tech Day  and Nth Generation’s 13th Symposium. If you didn’t see it, my preview was posted last month here on rsts11.

I’ll have some more detailed thoughts, including at least one topic that I hadn’t really expected to provoke so much thought, in the next few days. But I wanted to touch on two of the highlights from the Symposium while they’re fresh in my mind.

Disclaimer: Travel to HP Storage Tech Day/Nth Generation Symposium was paid for by HP; however, no monetary compensation is expected nor received for the content that is written in this blog.

Quick Overview of Nth Symposium

Nth Symposium is an annual partner and customer summit held by Nth Generation, the leading HP channel partner in southern California. They’ve done this thirteen times now, bringing customer technologists and executives together with HP and partner representatives for a very productive event. It’s free to qualified IT professionals, so I’d suggest checking it out next year if you are in the area.

Two of the three Nth Symposium keynotes were by execs I’ve worked for before. I was farther down the org chart from (now HP CEO) Meg Whitman when I was at the shopping.com division of eBay in 2006, but she gave the executive welcome at my new hire orientation. I reported to a VP at 3PAR who reported directly to (now HP Storage VP/GM) David Scott back in 2001. I knew both would be very impressive speakers for a keynote.

HP CEO Meg Whitman

HP CEO Meg Whitman (not channeling Clint Eastwood, don't worry)

HP CEO Meg Whitman (not channeling Clint Eastwood, don’t worry)

In a definite score for Nth Generation, they convinced Meg Whitman, president and CEO of HP, to give the headline keynote at this year’s symposium.

Whitman’s ability to know on a detailed level, communicate, and see the path forward for a hugely disparate business that probably seems like it’s going that-a-way at full speed in every direction is impressive.

The high level overview of the company’s direction, and the “New Style of IT,”  was to be expected, but her willingness and ability to field unstaged questions from the audience and respond to them in an honest and aware way was what really impressed me.

“Don’t be shy, remember, I ran for public office.”
–Meg Whitman

The three questions I remember involved cross-border ordering and SKU simplification (so that you can easily order the same model for delivery to multiple countries), support cohesiveness and contactability (and responsibility), and the morass that is hp.com.

Fellow blogger John Obeto was set up for a question when Meg called out Nigeria as one of the countries that would not see SKU simplification this year. But she acknowledged that the complexity was counterproductive, and that the company is already working to solve the problems for multinational customers.

Another attendee mentioned the challenges of finding the right contact for support, especially (as I recall) when multiple product lines are involved, or when your contacts at HP leave the company. Having had my HP account manager leave after my first order a couple of jobs ago, and having had her replacements actively and effectively lose my followup business in the months that followed, I know what a pain this can be.

Meg acknowledged the problem as a significant one, suggested using a partner or VAR as an aggregator for contacts within HP (since VARs would have more access to experts and resources within the HP organization), and concluded by offering her personal email address and committing to help until other paths are finalized.

But back to John, who came up to the microphone to decry the exclusion of Nigeria for the 2013 SKU project, and to mention something that probably everyone who has tried to used the HP site for anything but B2C e-commerce already knows… that hp.com is pretty difficult to navigate. Meg once again acknowledged the problem–see a pattern here?–and said that they were working on the business-to-business (B2B) and business-to-consumer (B2C) sites separately. One is already under a substantial reorganization, and the other will follow as soon as practical.

In general, I got the sense of Meg Whitman as a CEO being not entirely unlike the (parody) President Jimmy Carter’s fireside chat from Saturday Night Live in the late 70s. I wouldn’t ask her about acid experiences, but it seemed if you asked her about something even several levels down in the chain that was affecting customers, she’d know what was going on and be able to respond to it (or be willing to take the question on and find an answer).

The Moonshot heard round the worldnth moonshot on stage

On the topic of hp.com, Paul Santeler put further time into the discussion of Moonshot in the talk that followed Meg’s keynote, but as I recall Meg also made the bullet point on Moonshot that hp.com now runs on Moonshot rather than a huge farm of servers.

To be specific, they’re using about 720 watts of power to run the whole site. Think about that… as she suggested, you probably use more power on lighting in your home than they do to run a large enterprise web site with support, e-commerce, marketing, and all sorts of other content. (Unless you’ve gone green–I think I’m at a bit under 700W between all the lighting in my home thanks to CFL bulbs, but steady-state power rating for these power supplies is 653W so they win.)

Moonshot is a sub-5U chassis that contains up to 45 server “cartridges” running the Intel Atom S1260 at 2GHz. The cartridge is a bit larger than a Kindle Fire and sports an 8GB ECC dimm, dual gigabit Ethernet (through a central switching module pair), and a single 2.5″ laptop-style hard drive that can be 500GB or 1TB of 7200rpm spinning disk a 200GB MLC SSD.

The 45G switching modules live in the center of the chassis, and the two 6SFP uplink modules give 6 1GBE/10GBE uplinks each via SFP+ connectors. Standard configuration gives you one switch module and one uplink module; the redundancy option is a custom configuration. A 40GBE module is coming soon. The systems are managed via iLO Chassis Management, and multiple systems can be daisy-chained.

If you’d seen the Seamicro systems circa 2009-2010, the Moonshot will seem like at least an evolutionary development from that concept. The first times I spoke with Seamicro about their 10U chassis, I asked about a smaller system, around 4U, with fewer than the stock 64 systems. Moonshot gives nearly the capacity of that 10U system, 40% more system RAM, dedicated per-system storage,  a third the footprint, and a lower power draw.

There are other cartridges coming, including an 8-core 32GB cassette (good for thin virtualization) and a DSP-targeted cassette (voice processing and so forth, running on ARM), so it shouldn’t be a one-trick pony platform. It won’t replace all rackmount and conventional blade servers, but hyperscale is likely to fill a few niches and simplify management and scalability.

So where do we go from here?

I’ve been a fan of 3PAR’s “Utility Storage” platform since I joined the company in 2001. (They’re now buzzwording around Polymorphic Storage which is also cool.)

One thing I asked about often during my time on Technology Drive in 2001-2002 was a smaller starting point for the InServ platform. With the E and F series, they made some steps in that direction, and I bought an E200 for high performance storage at Trulia a few years ago. But with their new 7200 model, they go even farther into the realm of possibility with a starting list price around $25k.

I’ll be bringing you some details on their platform and enhancements in the next week. I’ll also be looking at the comparison between utility computing platforms from HP and Cisco, a topic that was featured in one of the second tier keynotes.

Stay tuned, and wish me luck on the recovery from convention plague if you don’t mind.