As you saw in my 3D Printing series, after years of pondering a 3D printer, I was finally inspired to buy one when a pile of clusters came up on eBay from the defunct rabb.it video streaming service.
In this series, I’ll take you through turning a rabbit door into some useful computing resources. You can do something similar even after the clusters are sold out; a lot of people have probably bought the clusters and ended up not using them, or you can adjust the plans here to other models.
Part 2 is out: Rabbit Launch: Loading up the NUC cluster with a usable operating system (but read this one first).
The first thing I will put out there is that these are not latest-and-greatest state-of-the-art computers. If you’re looking for a production environment or DDR4 high density memory, keep looking. But if you want an inexpensive modular cluster that’s only about 5 years out of date, there’s hope for you in here.
Update – As of December 2021, I’ve updated some of the links and prices, and updated the description to show that the door clusters have been sold out for some time.
The original cluster
eBay seller “tryc2” has several hundred of these “door clusters” from rabb.it, a now-defunct video streaming service that closed up shop in mid-2019. I call it a “door cluster” as the 42 inch by 17 inch metal plate resembles a door, and gives you an idea of the ease of manipulating and fitting the environment into your home/homelab as it is delivered.
The cluster bundle would have set you back US$300, plus tax where applicable. The seller seems to have had a thousand or so and they were sold out by late 2020. [Updated 2021]
The cluster includes 10 Intel NUC quad-core boards (mine were NUC5PPYB quad-core Pentium; my friend Stephen Foskett got some that were newer NUC6CAYB Celeron boards which took more RAM). These boards feature one DDR3L SODIMM slot (max of 8GB), one SATA port with a non-standard power connector (more on this later), Gigabit Ethernet, HDMI out with a headless adapter (to fool the computer into activating the GPU despite no monitor being connected), four USB ports, and a tiny m.2 slot originally intended for wireless adapters.
In the center of the “door” are five NVIDIA Jetson TK1 boards. These were NVIDIA’s first low-end foray into GPU development, sold to let individuals try out machine learning and GPU computing. There are much newer units, including the Jetson Nano (whose 2GB version is coming this month), if you really want modern AI and GPU testing gear, but these are reasonably capable machines that will run Ubuntu 14 or 16 quite readily. You get 2GB of RAM and a 32GB onboard eMMC module, plus a SATA port and an SD slot as well as gigabit Ethernet.
The infrastructure for each cluster includes a quality Meanwell power supply, a distribution board assembly I haven’t unpacked yet, two automotive-style fuse blocks with power cords going to the 15 computers, and a 16 port Netgear unmanaged Gigabit Ethernet switch. With some modifications, you can run this entire cluster off one power cord and one network cord.
So there is a catch to a $300 15-node cluster. The Jetson nodes are component complete, meaning they have RAM and storage. However, the NUCs are barebones, and you’ll need some form of storage and some RAM.
For the Jetson nodes, you’ll need an older Ubuntu machine and the NVIDIA Jetpack software loader. For the installation host, Ubuntu 14.04 is supported, 16.x should work, and later versions are at your own risk. You’ll also need an Ethernet connection to a network shared with your Ubuntu machine, as well as a MicroUSB connection between your Ubuntu host and the Jetson, to load the official software bundle.
For the NUCs, you’re looking at needing to add a SODIMM and some form of storage to each. I bought a bunch of 8GB SODIMMs on eBay ($28.50 each) to max out the boards. For storage, I tried USB flash drives and 16GB SD cards and had OS issues with both, so I bought the MicroSATACables NUC internal harness for each board, along with Toshiba Q Pro 128GB SATA III SSDs (these are sold out, but there’s a Samsung SM841N currently available in bulk for the same price, about $25 each).
- 1x Cluster bundle: $300 shipped
- 10x 8GB SODIMM: $285 plus shipping
- 10x SATA harness: $87.50 plus shipping
- 10x 128GB SSD: $250 plus shipping
- Total cost: $922.50 plus shipping (or about $990)
If you do get a cluster bundle with the two-memory-slot NUC boards, you have two options beyond the above. The easy and documented option is to look for 4GB SODIMMs instead of 8GB; you may save a buck or two, or if you’re like me, you may have a box of 4GB SODIMMs from various upgrades and not have to buy anything. The other option is to update your BIOS on the NUC and try out 2x 8GB. For some uses, 16GB will be worth the cost (vSphere or other virtualization clusters). I’d suggest going with a known quantity to update the BIOS to the latest version, and then trying 2x 8GB.
One other thing you may need is a pack of spare fuses. I know they do their job, as I blew a few of them while plugging and unplugging the boards. But you may wish to have a few extras around. They’re the standard 3 amp “mini blade” fuse that can be found at auto parts stores (although my local shops tended to have one card of them, if that, on the shelf). You can also buy a 10 pack for $6.25 (Bussmann brand) , a 25 pack for about the same price (Baomain brand) or a 100 pack (Kodobo brand) for about $9.
Choose your own adventure
There are two paths to take once you have your gear collected and connected.
- How do we lay out the gear?
- What do we do with it?
I’ll look at my journey on both paths in upcoming episodes of this series. Spoiler: I’ve 3d-printed stacking plates for both the NUCs and the Jetsons, and am still working on how to mount the remaining pieces so I can e-waste the door piece. And as I write part 1, I still haven’t figured out what to do with the clusters.
Where do we go from here?
If you’ve bought one of these clusters (or more than one), feel free to chime in on the comment section and let me know what you’ve done with it. And stay tuned to this post (or @rsts11 on Twitter or Facebook) for updates on the next installments.
I did not get that power distribution board on any of my three boards. Do you have a better shot of it?
I’ll put a couple of photos up on the Facebook post. It looks like a ganged buck converter board.
Looks like it may be a remote power control device. A coworker got it on one of his 4-5 doors.
Pingback: Revisiting Fry’s Electronics a year later | rsts11 – Robert Novak on system administration
Pingback: Rabbit Launch: Loading up the NUC cluster with a usable operating system | rsts11 – Robert Novak on system administration
Pingback: Pi in the sky: Seven tips for finding the single board computer of your dreams | rsts11 – Robert Novak on system administration