Category Archives: Networks

Replacing Meraki with TP-Link Omada for the new year

Posted on by
2

[This post was originally teased on Medium – check it out and follow me there too.]

[Update: As of April 2023 I’m an employee of Cisco again, with access to the employee discounts, and I’ve started rolling back to a Meraki plant. I’ll write a post by the end of the year detailing the reasons and choices, once I’m done.]

I’m a big fan of Meraki, but now that I haven’t been an employee of Cisco for over two years*, I no longer have the free license renewals or the employee purchase discounts on new products and licenses. So October 28, 2022, was the end of my Meraki era. (Technically a month later, but I needed a plan by October 28 just in case.)

The home network, mostly decabled, that got me through the last 4-5 years.

I needed a replacement solution that wouldn’t set me back over a thousand dollars a year, and my original plan was to use a Sophos SG310 either with the Sophos home firewall version or PFsense or the like. I even got the dual 10gig module for it, so that I could support larger internal networks and work with higher speed connectivity when the WAN links go above 1Gbps. I racked it up with a gigabit PoE switch with 10gig links, and now a patch panel and power switching module.

The not-really-interim network plan. The Pyle power strip and iwillink keystone patch panel stayed in the “final” network rack.

But I didn’t make the time to figure it out and build an equivalent solution in time.

How do you solve a problem like Omada?

Sometime in early to mid 2022 I discovered that TP-Link had a cloud-manageable solution called Omada.

It’s similar in nature to Meraki’s cloud management, but far less polished. But on the flip side, licensing 12 Omada devices would cost less than $120/year, vs about $1500/year (or $3k for 3 years) with Meraki. So I figured I’d give it a try.

The core element of the Omada ecosystem is the router. Currently they have two models, the ER605 at about $60-70, and the ER7206 at about $150. I went with the ER605, one version 1 without USB failover (for home, where I have two wireline ISPs), and one version 2 model with USB failover (for my shop where I have one wireline ISP and plan to set up cellular failover).

You’ll note I said cloud-manageable above. That’s a distinction for Omada compared to Meraki, in that you can manage the Omada devices individually per unit (router, switch, access point), or through a controller model.

The controller has three deployment models:

  • On-site hardware (OC200 at $100, for up to 100 devices, or OC300 at $160, for up to 500 devices)
  • On-site or virtualized software controller, free, self-managed
  • Cloud-based controller, $9.95 per device per year (30 day free trial for up to 10 devices I believe)

I installed the software controller on a VM on my Synology array, but decided to go web-based so I could manage it from anywhere without managing access into my home network.

Working out the VPN kinks

The complication to my network is that I have VPN connectivity between home and the shop across town. I also had a VPN into a lab network in the garage. Meraki did this seamlessly with what you could call a cloud witness or gateway – didn’t have to open any holes or even put my CPE into bridge mode. With Omada, I did have to tweak things, and it didn’t go well at first.

I was in bridge mode on Comcast CPE on both ends of the VPN, and did the “manual” setup of the VPN, but never established a connection. I tried a lot of things myself, even asked on the Omada subreddit (to no direct avail).

I came up with Plan B including the purchase of a Meraki MX65. I was ready drop $300-500 to license the MX65 at home, MX64 at the shop, and the MR56 access point at home to keep things going, with other brands of switches to replace the 4-5 Meraki switches I had in use.

As a hail-mary effort, I posted on one of the Omada subreddits. The indirect help I got from Reddit had me re-read other documentation on TP-Link’s site, wherein I found the trick to the VPN connectivity – IKEv1, not v2. Once I made that change, the link came up, and the “VPN Status” in Insights gave me the connectivity.

The trick to the manual VPN connectivity was IKEv1, not v2

The last trick, which Meraki handled transparently when you specified exported subnets, was routing between the two. I had to go to Settings -> Transmission -> Routing and add a static route with next hop to the other side of the tunnel. Suddenly it worked, and I was able to connect back and forth.

Looking at the old infrastructure

My old Meraki network had 12 devices, including three security appliances, four switches, a cellular gateway, and four access points. The home network used the MX84 as the core, with a MS42p as core switch, a MS220-24 as the “workbench” switch on the other side of the room, and a MS220-8P downstairs feeding the television, TiVo, printers, MR42 access point, and my honey’s workstation, connected via wireless link with a DLink media access point in client mode. I also had a MS510TXPP from Netgear, primarily to provide 2.5GbE PoE for the Meraki MR56 access point.

There was a SG550XG-8F8T in my core “rack” (a 4U wall-mountable rack sitting on top of the MS42p switch) but it was not in use at the time – I didn’t have any 10GBase-T gear, and the MS42p had four 10GbE SFP+ slots for my needs.

The garage lab had a SG500XG-8F8T behind the Z1 teleworker appliance. TP-Link powerline feeds that network from the home office.

The remote shop had a MX64, MS220-8P, and MR18, as well as the MG21E with a Google Fi sim card.

So there was a lot to replace, and complicate in the process.

Looking at the new infrastructure

The new core router is the TP-Link ER605, feeding the MS510TXPP switch for mgig and 10gig distribution (including WiFi), with another downlink to a TL-SG2008P switch ($90 at time of purchase) which offers 4 PoE+ ports and integrated monitoring with Omada.

The ER605 has front-facing ports, so I have those cables going into the patch panel to connect Internet uplinks and the PoE switch. On the SG2008P, ports are on the back and LEDs are on the front, so I have all 8 ports going to the patch panel and they feed things from there.

The MS510TXPP has downlinks to the powerline network, a SG500-48X switch across the room connected by 10 Gigabit DAC, and a few other things in the office.

I have the wireless needs fulfilled by a Netgear Nighthawk router in AP mode, and a TP-Link Omada EAP650 access point that needs some tuning. I expect to replace the Nighthawk with the EAP650 at some point, and I have a Motorola Q11 mesh network kit coming soon which could replace much of the wifi in the house.

The downstairs network is still fed by the DLink wireless bridge (as a client of the Nighthawk), but now it has a random Linksys 8 port switch serving the first floor needs.

The garage lab still has the SG500XG, bridged via powerline, and very limited hardware running due to California electric prices.

In the shop, I have the ER605v2, feeding a random 8-port TP-Link unmanaged switch for now. I’m thinking about getting an Omada switch there, and I recently installed a UeeVii WiFi6 access point (acquired through Amazon Vine, review and photos here) which is more than enough to cover the 500 square feet I need there.

Why’d it take so long to post?

I had found an Etsy seller who made 3d printed rackmount accessories, and I ordered a cablemodem mount, router mount, and a 5-port keystone patch panel. I ordered December 15, shipping label was issued December 21, and I expected it right after Christmas. Alas, after a month and two shipping labels being generated, I had no gear and no useful response from the seller, so I got a refund and went with rack plan B.

I took a 14″ 1U rack shelf like this one (but fewer slots and about half the price) and used zip ties to attach the router and 8-port switch to it. Not a great fit, but inside the CRS08 carpeted rack it’s not so visible.

Where do we go from here?

Right now the networks are stable, except for no wifi in the garage and occasional wifi flakiness in the house. So my next steps will be fixing the home wifi, and probably moving another AP to the garage (possibly even setting up a wireless bridge to replace the powerline connection).

I am looking at some more switching, possibly upgrading the Omada switch to replace the Netgear at home, and then take the existing 8 port Omada to the shop to provide more manageability (and PoE+) over there.

The front runners for the new switch right now are the SX3008F (8 port SFP+ at $230; 16 port SX3016F is $500), SG3428X (24 port gigabit, 4 port SFP+), and the SG3210XHP-M2 (8 port 2.5GbE copper PoE + 2 SFP+ slots at $400, pretty much the same as the Netgear except with no 5GbE ports).

There are a couple of other options, like the $500 SSG3452X which is equivalent to the MS42p, but I’ll have to consider power budget and hardware budget, and what I can get sold from the retired stash this month to further fund the expansion.

I also need to work out client VPN to connect in to both sites. I had client VPN on my travel laptop to the shop for a couple of years, but haven’t tried it with the new platform yet.

TP-LInk supposedly has a combination router/controller/limited switch coming out this year, the ER7212 which also offers 110W PoE across eight gigabit ports. It’s apparently available in Europe for 279 Euros. Hopefully it (and other new products) will be released in the US at CES Las Vegas this week.

I was going to bemoan the lack of 10G ports, but then I saw the ER8411 VPN router with two SFP+ ports (one WAN, one WAN/LAN). Still doesn’t seem to support my 2.5Gbit cable WAN, but it’s at least listed on Amazon albeit out of stock as of this writing.

Tying cloud provider orchestration, security, performance, and cost management together with Prosimo AXI at Cloud Field Day 12

Posted on by
1

Disclosures at the bottom as usual.

At Cloud Field Day 12 in San Jose, Prosimo joined us to explain how they are “more crazy than you guys think we are.” We got a good introduction to the problems they are working on solving, some of the problems they’re not focused on, and yes, we did learn they might be pretty crazy. But it just might work.

Prosimo was founded in early 2019, the second startup for many of the team (hence their CEO Ramesh Prabagaran’s crazy quoted confession above). The connection I didn’t make at the time is that his first startup was Viptela, who were acquired by Cisco in 2017. So after developing an SD-WAN solution, Prabagaran and his team moved up (or down) a layer to address another element of WAN in the clouds. Maybe we could call it CAN, or Cloud Area Networking. Or maybe not.

To get to the good stuff… Prosimo focuses on multicloud transit (the third lozenge pictured above), an element between the users and edge networks and the applications a customer has in any of the three major clouds or their own datacenters/private clouds.

Continue reading →

When POHO isn’t psycho enough – a home network update in progress

Posted on by
1

If you’ve been around for a while, you will know that POHO, or Psycho Overkill Home Office, is an ongoing theme of this blog. I’ve described it more than twice as “two comma technology on a one comma budget.” It stands to reason that my home network is in the “psycho overkill” range, with three sites connected by VPNs and internal 10 gigabit networking (40 gigabit on its way).

Disclosure: Much of the gear in this post is Cisco Meraki, and much of that was obtained using employee purchase program benefits as a Cisco employee. As a system engineer I was eligible for free renewals on my licenses for the Meraki gear, but the original licenses and most of the hardware purchases were out of my own pocket. Any other gear mentioned was purchased out of my own pocket through mainstream methods (i.e. eBay) unless otherwise noted. Cisco has not reviewed, influenced, or endorsed this post or this blog, and they most likely won’t.

A photo before everything was recabled. There are a lot more ports in use now.

What’s the POHO like today?

In the past two years I’ve been running a somewhat crippled network, despite having pretty good employee purchase benefits at work. Still, with gigabit fiber and 500 megabit cable, I’m at about 2.5x the capacity of my core router.

I’m running a Meraki MX84 as the core of my home network, with AT&T / Sonic fiber as primary, and Comcast as secondary. It downlinks to an MS42p 48-port switch with four ports of 10 Gigabit Ethernet. On the upstream side, it connects via Meraki’s auto-vpn to an MX64 in my shop across town, and to a Z1 Teleworker unit in my garage that keeps some lab gear protected from the world (and simplifies IP addressing).

I have a couple of MS switches around the networks, as well as a Cisco Small Business SG500XG-8F8T, a Netgear MS510TXPP (for mgig POE) and a couple of other brands in use from time to time. Wireless is handled by MR56 and MR34 in the house, MR18 in the garage, and MR16 in the shop.

Unfortunately, the MX84 is limited to 500mbps of stateful firewall or 320mbps of advanced security throughput. I’m getting pretty close to that, but the other half of the uplink is idle unless I switch over to the other side of the MX.

Continue reading →

I didn’t think I’d be able to say this so soon… (He’s baaack at Tech Field Day!)

Posted on by
1

As many of my readers know by now, my time at Cisco came to an end last month. When I decided to leave Disney and come to Cisco 6 1/2 years ago, there were two main things I knew I would miss about being in the “real world” — Disney cast member discounts, and being a Tech Field Day delegate.

Well, there’s no change on the Disney discount front, but this week I’ll be back as a TFD delegate for Tech Field Day 22 the latter half of this week.

Riding in the limo at SFD5 in 2014 – four of the five people pictured will be at TFD22 this week with me

How did you get to this point?

In May 2014, I posted a two part post on storage vendors (“These 3 hot new trends” part 1 and part 2) from Storage Field Day 5, my last full event as a delegate. A month later, I moved to San Francisco for most of a week thanks to TFD sponsors, to participate in my second Cisco Live event and to interview for a position with Cisco.

I was offered the job the day I got home from the event, and a little under a month later I got badged at Building 9 and began the 6+ year adventure in mega-vendor sales engineering. But as a vendor, I wasn’t terribly welcome among the Tech Field Day delegations, although I was still invited to the parties, and managed to qualify for the roundtable at SNIA’s Storage Developer Conference in 2017. I did continue my participation with Interop over the years, leaving my Cisco ears (instead of my Disney ears) at home, and even attending a Cisco briefing during one of the events, in the former Playboy Club at the Palms in Las Vegas.

What is Tech Field Day? Do I need a ham radio?

If you’re new to Tech Field Day, the idea is pretty much the same as it’s been for over ten years, even if the participation venue has moved from conference rooms to Zoom. Stephen Foskett, founder of Tech Field Day and Gestalt IT, brings together independent analysts, practitioners, geeks, and javelin catchers to meet with companies producing something in the tech sphere.

From the huge established names (like Dell, HPE, Cisco) to companies just coming out of stealth and talking to the public for the first time, you get to see companies facing unstaged questions in realtime, discussing the product or service, the decisions behind them, and how people who might actually use the product or service see it rather than how the company’s marketing and PR team want it to be seen.

And unlike most press conferences and analyst events, anyone on the planet (pretty much) can tune in, watch and learn, and pose their own questions through social media to be answered. There’s no registration required, no event fees, and if you missed a company you can go back and watch within a couple of days.

Pro-tip: If, like me, you’re on the tech job market, Tech Field Day’s archives can be a great resource for learning about companies you might be interested in working for. Just go to the main page and search for a company name. Not everyone is in there, but you can get a good feel for the companies that are, from what they do and how they’ve evolved over the years to how well they understand their product and the market they’re competing in.

So what’s different for you this time?

Tech Field Day 9 in Austin, Texas (June 2013)

After five full delegate events in person, and seven roundtable/TFD Extra events (details), I’ll be back as a different kind of delegate, for obvious reasons. TFD22 looks to be the largest event yet, with twenty-five delegates. No, really, 25 delegates. The nine presenting companies will be split up into early and late shfits to accommodate delegates from around the world, and since none of us are traveling to an in-person location, we can focus on presentations in our own time zones… and some of us will be hopping onto the other shift’s events as well.

The early shift, for my European and Eastern colleagues, will feature Commvault, Veeam, VMware, Quantum, and Red Hat. Their sessions run from 5-10am Pacific, and while I’d love to see them live, I’m not sure 5am is a time I believe in just yet.

You’ll find me in the late shift (11am-3pm Pacific), meeting with MemVerge, Riverbed (who I last visited here in Sunnyvale for SFD2), Illumio, and oddly enough, Cisco. I only see three names among the other 24 who I’ve shared TFD events with, but about half of them are in my online circles and I’m looking forward to meeting the others.

If you’d like to watch along with us, check out the TFD page for livestreams on several platforms starting Wednesday morning, December 9th. You can click on this garishly-large TFD logo to get there if you like. And if you miss the sessions you wanted to watch, they’ll be posted on the same link within a couple of days for you to watch at no cost.

Feel free to follow along on Twitter and ask your questions – tag with the hashtag #TFD22 and the delegates will try to relay your questions to the presenters.

Straying into Ubiquiti territory for a home network experiment, part 1

Posted on by
1

As many of you know, I run my home, lab, and store networks primarily on Meraki gear. Employee discounts and internal system engineer promos make it a reasonably priced platform for me, but I can understand why non-Cisco employees might not build out a substantial home network on their own dime with Meraki.

Having cut directly over from the Linksys WRT1900AC as a router to a mix of MX security appliances, MS switches, and MR access points, I didn’t really take the time to evaluate other options. However, with many friends getting into Ubiquiti, I figured it was worth trying that platform out, especially when some of the devices went on sale at a local computer store.

In this post I’ll talk about the initial deployment and the gear I’ve purchased. I do have a few items from Ubiquiti that I won’t be using for this environment (like the EdgeRouters and a couple of relatively ancient 24v POE access points).

Spoiler: I’m still a big Meraki fan, and if I were deploying in a business environment where I didn’t want to tweak much or where I wanted enterprise-grade features, I’d still lean toward that platform. However, for a home network, home office, or early stage  startup, the Ubiquiti option is definitely worth a look.

Initial Bill of Materials

ubnt-cloudkey-aa-1.jpg

UC-CK Cloud Key, with two AA batteries for scale

Note that Amazon offers some combos with multiple elements, like this $349 combo with Cloud Key, Switch, and Security Gateway. You may be able to get quicker shipping and/or save a buck or two that way, but look around at the combos to see what makes the most sense. If you decide to buy multiples, there may be discounted packs of devices (like this 5-pack of AP-AC-PRO which saves you about $15 per device).

You’ll also find the items on Newegg, including Newegg on eBay, Central Computers (if you’re in the SF Bay Area), and direct from Ubiquiti. If you use the Amazon or eBay links above, we get a few bucks that will go back into gear to review here and on rsts11travel.

Why did I choose this particular gear?

ubnt cloudkey

UniFi Cloud Key

Like Meraki, Ubiquiti uses the concept of a “cloud controller.” Unlike Meraki, you can place the controller on your own private cloud, or purchase a “Cloud Key” to run on your own network for management. There is still a “public” website to view and manage the network, but you can access the local controller via ssh, https, or a mobile app.

Since I don’t currently have a full-time system running that would host the controller, I chose to buy the older Cloud Key. They have newer versions, with more powerful controller hardware, battery  backup, and more features, but since this is meant to be a basic deployment on a budget (and I wanted to pick up the cloud key locally), I went with the first gen device. This device is about the size of four AA  batteries; can be powered by PoE or a USB cable; and of course still requires a LAN connection even if powered by USB.

ubnt accesspoint

UniFi AC Pro

For wireless access, there are over a dozen different AP models, compared and contrasted on the Ubiquiti knowledgebase. The three devices in the “wave 1” family (UniFi AC) include the Lite, the LR (long range), and the Pro. My decision on the Pro was based primarily on “ooh, it’s on sale” but I’m pretty comfortable with the features including extended 5GHz radio rate of 1300 Mbps, and the dual Ethernet ports for redundancy.

ubnt switch

UniFi Switch 8 60W

The switch is meant to let me offload both the AP and the Cloud Key from their current home on my Meraki MS42P switch, so that I can put them behind the security gateway for more thorough testing. The AP uses 9 watts and the Cloud Key uses 5 watts, so the 60 watt PoE switch should be enough for the near term.  There is a 150 watt version (US-8-150W, for about $190) with two additional SFP modules, if you do need more power. And interestingly, the switch is the only piece in the bill of materials that has a metal shell as opposed to plastic.

ubnt security gateway

Unifi Security Gateway 3-port

Finally, with the USG security gateway, I get additional visibility into the Internet connection itself and my use thereof. Without the USG in the data path, I can see per-device information within my network, and status of the APs and switches, but I don’t have the visibility at a network level.

Starting the deployment

I bought the access point first, and went back a day or two later for the cloud key once I decided not to run the controller on my own hardware. So the CK went up first, plugged in via the tiny Ethernet cable to a port on my Meraki PoE switch.

When I logged in, of course, it was behind a few versions on the firmware. I had issues with firmware updates and “adopting” the device into my Ubiquiti cloud portal. The adoption failed claiming the device was unreachable, and the firmware upgrade didn’t seem to start, much less complete.

So I ended up doing some minor workarounds using some steps from a community post here for the firmware update. I wish I could remember the fix for the adoption, although I suspect I’ll figure it out again on a future device and can report back then.

Once the Cloud Key was recognized, updated, and working properly, I adopted the Access Point and updated it. I configured a wireless network and went downstairs from the home office to connect my iPad to the new network and test it out.

Not surprisingly, the network was as fast and efficient as it was through the MR34 at the same distance. I did learn from the Ubiquiti interface that there were at least 50 networks detected by the AP-AC-PRO, which was slightly surprising. Despite that, I’m seeing about 20% utilization on 2.4GHz and 3% utilization on 5GHz and noticeable but not overwhelming “interference” registering primarily on 2.4GHz.

I also realized that the extra MR34 downstairs, connected through an MS220-8P switch that was uplinked through Powerline networking, was definitely throttling my connectivity when I associated with it. Unplugging the AP forced my iPad to connect to the upstairs MR34, and I didn’t have any issues even at the distance. So for now, the Powerline network is driving two tiny Verium miners and my two printers, as well as an Intel NUC in the living room.

What comes next?

After reorganizing a bit of the home office, I’ll be turning up the USG security gateway and the 8-port switch very soon. At that point I’m likely to put all four pieces behind my secondary Internet connection (to enable the home network SLA to be maintained), and run some traffic through it.

I’m also giving serious thought to powering the USG through a PoE splitter like the Wifi Texas one ($18 on Amazon) so that all four devices can be powered from a single wall outlet (for the switch).

Check in soon for the second part of this journey, and feel free to share any suggestions, comments, references, designs, etc in the comments below.