This entry is part of my POHO (Psycho Overkill Home Office) series. 

I have a habit of overdoing my networking. My home core switch is a Juniper EX-series (courtesy of a Bay Area Juniper User Group meeting raffle), and for a while I had a 10-Gigabit Ethernet (10GbE) Extreme Networks switch (that cost less than a good laptop) ready to go in. Do I really need it? Probably not. I sold it last year but am now thinking about 10GbE again.

I’m here today to share some of my tips for finding affordable enterprise-class networking for your personal, home, lab, or photo shoot purposes. I also welcome your thoughts and experiences in the comments below. Let me know if you’ve found other ways to improve your dollars-to-metal KPIs at home or in a lab.

Caveat: I do not advocate these methods for anything your company may depend on, or anything production grade in general. However, if you’re closer to hobby network than to Fortune 500 core network, this may help you build beyond your budget.

Foreword: About 10 Gigabit Transceiver Formats

There are three formats or “sockets” for transceivers for 10GBE ports. Each of them can present short (SR), long (LR), or extended range (ER/XR/ZR) fiber interfaces, or “captive” cables in the form of CX4, twinax, or RJ45-type copper.

XENPAK are the oldest, largest, and least expensive (generally) of the form factors, and can be found integrated into older host interface cards as well as switches like the Extreme Summit 400. XFP are almost as old, smaller and lighter, and probably a bit more pricey than XENPAK. The current generation is SFP+, which has the same size as the gigabit SFP (almost the same size as a RJ45 connector) but supports 1 and 10 gigabit. (The second transceiver pictured above is X2, which I haven’t run into yet.)

Depending on distance between ports and the host adapter you choose, you may find one of these more desirable than the others. Fiber may be more flexible in terms of length and availability, and you can go from SC to LC if your transceivers don’t match, but CX4 or twinax will be more resilient to tension.

Be warned that some networking vendors break the standards and check for their own brand of transceivers, not allowing generic or other brand devices even if they are physically identical down to the manufacturer.  Vendor forums or a quick Google search will help you track down these issues and plan for, or work around, them.

1. Uplink ports are usually just ports. Stacking ports, not so much.

The cheapest and (maybe) easiest way to get 10GbE ports is to buy a 1GbE switch that has a few 10GbE uplink ports. My Extreme was this kind of solution–48 ports of 10/100/1000Base-T[X] and a two-port module on the back for 10GbE via XENPAK modules. There are a lot of switches out there on the used market that offer 2-4 “uplink” ports that can be used to connect a host.

Be warned that in many cases, the uplink modules will cost more than the base switch, if they’re not already included. I’ve been watching for the 10GBE uplink modules for a couple of my cheap 48 port switches, and they’re 4-5x what I paid for those switches. Dig into your datasheets and search for the module part numbers, and see what they’re going to cost you before investing in a base switch. (And see #2 below on this topic too.)

And don’t forget about optics. You can mix SC and LC endpoints with fairly affordable fiber cables, but if your uplink modules don’t have fixed connectors, you will need to get some sort of transceivers (optical, CX4, captive cables, TX) that can connect with your host adapters.

You might be able to use stacking ports as regular network ports, but that will warrant more research (and maybe finding a friend at the network vendor). I wouldn’t count on this option unless you already know otherwise.

2. Look for rebranded (or debranded) gear.

I got my first pair of 10GbE host adapters for about $23 each shipped from an eBay seller. Fully functional, PCI-X (backwards compatible with PCI if you don’t need full 10 gigabit speeds), with optical XENPAK modules built in.

Why so cheap? The seller had posted them with an HP part number, which 99% of the time returned compatibility only with HP/UX. Turns out they are very compatible with Linux, and while they don’t appear to be supported under VMware, you could put something like that in a homebrew SAN and run it into the core network at 10GbE.

Not everything is available rebranded, but you’ll find some network companies selling their products under IBM or Sun or Dell labels or the like. Switches, firewalls, and expansion modules have been known to show up under multiple vendors’ part numbers, and sometimes one vendor’s part can be half the price of the original manufacturer’s part, for identical metal and silicon.

This goes back quite a ways, at least to when Dell resold the Netgear FS switch line, Cisco resold QNAP storage arrays and HP Proliant servers, and IBM resold Brocade Silkworm SAN switches. Just like searching for typos, searching for alternate part numbers may help you get a good deal.

3. If you have a bit of budget, ask around about new and refurbished gear.

The big PC makers have outlet and financial services stores on their websites that sometimes have off-lease models for sale. There may also be first-market options if you have a bit more of a budget than I do.

Recently on Twitter Gabe Chapman and were discussing low-density 10GbE options; Garrett from Brocade chimed in to point us toward a Brocade ICX6450-24 switch as an option. A quick web search showed that for about $3000 street price, you can get a brand new switch with 4 licensed 10GbE ports and a warranty. You could get it for about $2300 with only 2 10GBE ports, and then buy the extras later (or run them at gigabit speeds).

If you’re stocking a little closer to the revenue end of the network, or if you can’t quickly replace a failed model, you might be better off choosing new/retail over a $400 switch as-is on eBay with the same port count.

Disclaimer: I have no connection with Brocade; this is just an example from a Twitter conversation that included a helpful Brocade guy and a quick Google Shopping search. Although I’d be willing to review an ICX6450-24 if the option arose. 🙂 

4. Watch out for port/feature licensing!

Some vendors, especially those in the Fibre Channel world, offer port licensing as a way to reduce the initial outlay for a switch.  A lot of smaller FC switches worked this way, and when I had to sneak a SAN into the budget at a company where the finance folks insisted all servers cost $1000 because that’s what a PC costs at Fry’s, it saved my storage plan from extinction.

In the case of the ICX6450-24 above, the base switch has four SFP+ ports, two of which are enabled for 10GbE out of the box, and two of which are limited to regular gigabit speed. To get the other two ports up to 10GbE, you buy a license kit for about $800 (street) and enable the ports. That’s not too bad for a brand new enterprise switch, but if you buy a ten year old switch that has unlicensed ports, you may have trouble getting the license codes (even for a price).

You’ll probably want to talk to someone knowledgeable about the platform you’re considering, to evaluate the risk. Some vendors tie features to a serial number (Juniper for example), so as long as your device is licensed and the serial number is intact, you may be able to reinstall the licenses automatically. Others require a key code, so unless you can get into the switch and retrieve that, a factory reset could wipe out some of your ports. And in either case, if the equipment you’re buying doesn’t have the features you want, it may be expensive or impossible to obtain them.

5. Consider port-channel or other aggregation methods.

I’ll admit this is sort of a cop-out, but 8 ports of Gigabit Ethernet will likely be cheaper than a single port of 10GbE. You can get 4-port PCI-E 1GBE cards for $75 or so (as low as $25 if you don’t need ESXi 5/5.5 support), and a 48-port GigE switch that supports LACP or the like for under $200. So that’s under $150 per 8gbit link including cables. Check your OS or virtualization platform HCL to make sure the cheaper cards are compatible, of course, but it’s worth checking out this option if it works for your needs.

So where do we go from here?

Those are my tips so far… I’d welcome your comments below on how (or if) they’ve worked for you, or if you have any tips from your own experience to share with other readers. Maybe you’ve found a vendor whose 10GbE switches are more affordable for the home lab, or just had a good experience with a home-lab-friendly reseller? Please chime in.