FirmwareGate and FCoEgate two months later

I was surprised last week at Interop to hear people still talking about both FCoEgate and HP FirmwareGate. It seems that in the absence of any clarity or resolution, both still bother many in the industry.

For those of you who missed the early February drama (and my relevant blog post):


FCoEgate: An analyst group called The Evaluator Group released a “seriously flawed” competitive comparison between an HP/Brocade/FC environment and a Cisco/FCoE environment. Several technical inquiries were answered with confusing evidence that the testers didn’t really know what they were doing.

Several people I talked to at Interop mentioned that this was a perfectly understandable mistake for a newbie analyst, but experienced analysts should have known better. Brocade should have known better as well, but I believe they still stand by the story.

The take-home from this effort is that if you don’t know how to configure a product or technology, and you don’t know how it works, it may not perform optimally in comparison to the one you’re being paid to show off.

This one doesn’t affect me as much personally, but I’ll note that there doesn’t seem to have been a clear resolution of the flaws in this report. Brocade has no reason to pay Evaluator Group to redo a valid comparison, and technologists worth their salt would see through it anyway (as many have). So we have to count on that latter part.


FirmwareGate: HP’s server division announced that, for the good of their “Customers For Life,” they would stop making server firmware available unless it was “safety and security” updates. How can you tell if it’s “safety and security?” Try to download it.

HP claimed repeatedly that this brings them in line with “industry best practices,” thus defining their “industry” as consisting exclusively of HP and Oracle. I don’t know any working technologists who would go along with that definition.

HP promised clarification on this, and defended their policy change by declaring industry standard x86/x64 servers as equivalent to commercial operating system releases and Cisco routers.

They even had a conversation with my friend John Obeto, wherein they convinced him that nothing had changed. Ah, if only this were true. (It isn’t.)

But I had fleeting faith that maybe they’d fixed the problem. So I went to get the firmware update for a nearly 2-year-old Microserver N40L, which had a critical firmware bug keeping it from installing a couple of current OSes. Turns out it’s not a “safety and security” fix, and my system apparently came with a one year warranty.

So if I wanted to run a current Windows OS, I either have to spend more on the support contract than I did on the server (if I can find the support contract anymore), or go with an aftermarket third party reverse-engineered firmware (which, unlike HP’s offerings actually enhances functionality and adds value).

Or I can go with the option that I suspect I and many other hobbyists, home lab users, influencers, and recommenders will — simply purchase servers by companies that respect their customers.

What should HP be doing instead?

The “industry best practices” HP should be subscribing to include open access to industry standard server firmware that fixes bugs they delivered, not just vaguely declared “safety and security” upgrades, much as every other industry standard server vendor except Oracle does. That includes Dell, Cisco, Supermicro, Fujitsu, NEC, Lenovo/IBM, and probably a number of other smaller players.

As my friend Howard Marks noted, some of us would be satisfied with a software-only or firmware-only support contract. On-site hardware maintenance isn’t necessary or even affordable for many of us. Many of us who buy used servers would be better off buying an extra server for parts, and most of us buying used servers know how to replace a part or swap out a server. Some of us even better than the vendor’s field engineers.

HP has been silent on this matter for over a month now, as far as I can tell. The “Master Technologists” from HP who won’t distinguish an MDS router from an x86 server have gone silent. And I’m sure many of the “customers for life” that the 30-year HP veteran graciously invites to keep buying support contracts will start looking around if there’s not a critical feature in HP servers that they need.

So where do we go from here?

I can no longer advocate HP servers for people with budgets containing fewer than 2 commas, and even for those I’d suggest thinking about what’s next. There are analogous or better options out there from Dell, Cisco, Supermicro, Fujitsu, NEC, Lenovo, and for the smaller lab form factors, Intel, Gigabyte, Shuttle, and others. (It’s also worth noting that most of those also provide fully functional remote management without an extra license cost as well.)

If you do want to go with HP, or if you can’t replace your current homelab investment, there are ways to find firmware out there (as there has been in the past for Sun^wOracle Solaris). It took me about 15 minutes to find the newly-locked-down Microserver firmware, for example. It didn’t even require a torrent. I can’t advocate that path, as there may be legal, ethical, and safety concerns, but it might be better than going without, at least until you can replace your servers.

And I’ve replaced most of my HP servers in the lab with Dell servers. One more to go. If anyone wants to buy a couple of orphaned DL servers in Silicon Valley (maybe for parts), contact me.

If anyone else has seen any clarity or correction in the state of FCoEgate or FirmwareGate in the last month or so, let me know in the comments. I’d love to be wrong.

Interop Las Vegas 2014 – Highlights, Lowlights, Footlights

I’m back from a week in Las Vegas for the annual Interop convention. Had some great conversations with vendors and technologists, got punched by Mark Twain, and graduated to the next level of mLife. I’ll have more to talk about in the next weeks, but I wanted to share some summary thoughts for those of you who are curious.

Disclosure: I attended Interop on a media/blogger pass, which was provided at no cost to myself, under the auspices of Tech Field Day. However, travel and incidentals were out of my own pocket, and aside from some trinkets that were offered to everyone in the expo, I have received no valuable consideration from any of the companies mentioned in this post. If this changes, I will advise in future articles.

My other coverage: “A Context For Cloud” covering my interpretations of Adrian Cockcroft’s Cloud Connect Summit opening keynote.

Good Stuff, Maynard

I’ve written before about how obnoxious the badge scanners can be at trade shows. This year’s Interop brought (I believe) a new feature that almost makes the scanners worthwhile — a trip report summarizing my event contacts and sessions. As I was leaving Las Vegas, I got an email that pointed me to a listing of my Interop sessions, surveys for the ones I hadn’t filled out, and the exhibitors who scanned my badge (or at least most of them… the list seems short, but some of the vendors were polite about not scanning–especially the ones I already communicate with weekly). I hope that other conventions like Cisco Live and VMworld add this functionality as well.

2014-04-01 11.37.07The Media Lounge was well-appointed, fiercely guarded by the incredible UBM PR team, populated with coffee, very edible breakfast and lunch at appropriate times, electricity and network connectivity, and except for the UNLV marching band incident, relatively quiet. It even featured a Bay Networks-branded Netgear dual-speed hub.

The best swag of the event (for me) is probably a tie between the Backupify Travel Hoodie Pillow and the 15% Off coupon for the new Linksys WRT1900AC.

The best physical technology I saw at the event would be the 16GB DDR3 SODIMM from Memphis Electronic, the Linksys WRT1900AC (luckily the big one isn’t the one that’s shipping next week), and the new Shuttle DS81 (Haswell compact system with dual 4k) video.

The best soft topics I saw included Circle Technology’s Circle Host/Circle Viewer private network screen share technology and Synology’s DSM 5.0 Central Management System.

And unrelated to Interop, I got notification on April 1 (seriously) that I was selected again as a VMware vExpert for 2014-2015. I continue to feel humbled and honored by this designation, and I hope to continue to provide useful contributions to the POHO community around virtualization technology.

Not So Good Stuff, Maynard

I’ll admit the first shock I got was the “Airline Chicken” in the media room at lunch. Several of the other folks in the room and I were concerned about an association between food and airplanes, having eaten on airplanes before. However, Meredith Corley from UBM Tech PR helped us get over that concern by looking up the worrisome product. It wasn’t so worrisome after all.

We did see a couple of “unclear on the concept” moments during the Expo, especially around “sponsorship” of refreshments. I believe Verisign “sponsored” the welcome reception on Tuesday, which made it possible for us to have cans of soda for only $4.25 each. Spiceworks apparently “sponsored” the coffee stand at the entrance, again with the $5 beverages. In the future, I’d suggest finding a different term, or perhaps making it clear how to take advantage of the sponsorship. Even the hotel only wanted $3.25 for a 20oz bottle of soda, and that’s not even sponsored.

And unrelated to Interop itself, I will restate that I hate hotel pillows.

One suggestion I will throw out there… if it would be possible to have lockers for media/bloggers (if not for everyone), so that we can leave laptops securely stowed during Expo and evening events without going all the way back to the hotel, that would be a welcome enhancement. I do think next year I will probably stay in the Mandalay Bay hotel, to optimize mLife points and minimize commute.

All things considered, not much to complain about.

Things to watch for

I had good conversations with a couple of vendors during the event.

Check out the links in this graf for each company’s Tech Field Day presentations.

I also had a charming conversation about security and network compliance and Doctor Who with Andy Williams, Nicola Whiting, and Ian Whiting of Titania. They have a compelling product line for auditing network device configurations, and weren’t too shocked that Rowan Atkinson was my Doctor.

So where do we go from here?

Well, for me, I’m catching up on a week’s email and then headed back to Las Vegas on Friday to work on some work stuff (seriously). No rest for the wicked, and PTO never seems to reduce the backlog of work.

Coming up in about 2 weeks is Storage Field Day 5 featuring Diablo Technologies, EMC, PernixData, Sandisk, Solidfire, Veeam, and X-Io. I’ve threatened to blog more from this event, although I’ll also be providing realtime feedback and twitter analysis as usual.

If you think I left out a highlight of Interop, or if you’d like to share your feedback, the comments are open. Hope to hear from you down there.

A Context For Cloud From Within And Without

Cloud Connect Summit is co-located with Interop this week in Las Vegas, Nevada. This is part of a series of highlights from my experience here. Disclaimers where applicable will follow the commentary. Check for presentation materials if available.

Update: Adrian Cockcroft’s slides are available at Powered by Battery.

I usually don’t give a lot of focus to keynotes, because I have conference-strophobia or something like that. A room with thousands of people in it is rather uncomfortable for me. And so are buzzwords.

However, Cloud Connect opened with one speaker I know and have spoken with before, another whose business I am familiar with, and a third guy who I didn’t know, but had to assume either did something wrong in a past conference, or is on par with the first two speakers.

Adrian Cockcroft probably needs no introduction.

Mark Thiele is a well-known figure in the datacenter and colocation world. He is currently executive VP and evangelist of datacenter technology for Switch, known for their SUPERNAP datacenters here in Las Vegas and elsewhere.

And the poor guy who got stuck between them… Chris Wolf is CTO Americas of VMware.

Okay, maybe Adrian deserves an introduction

It’s no surprise that Adrian Cockcroft focused on implementing and migrating to cloud. If you’ve seen him speak in the past 4 years it’s probably been about what Netflix was going to do, was doing, or has already done in their migration to an entirely-off-premises cloud-based solution (AWS). He’s now in the venture capital world with Battery Ventures, guiding other companies to do things similar to what Netflix did.

I first met Adrian at a BayLISA meeting in 2009.  I’d been a fan from his Sun legacy; as author of *the* Sun Performance and Tuning book in the 90s, you would be hard pressed to find a Solaris admin who hadn’t read the book, along with Brian Wong’s Configuration and Capacity Planning book. In 2009, he talked about dynamically spinning up and down AWS instances for testing and scaling–it was an uncommon idea at the time, but nowadays few would imagine an environment that didn’t work that way (other than storage-heavy/archival environments). I had a long ad-hoc chat with him at the last free Devops Days event in Sunnyvale, where he predicted the SSD offerings for AWS a couple of months before they happened.

As most of my readers already know, Netflix has had to build their own tools to handle, manage, and test their cloud infrastructure. With a goal to have no dependencies on any given host, service, availability zone, or (someday) provider, you have to think about things differently, and vendor-specific tools and generic open source products don’t always fit. The result is generally known as NetflixOSS, and is available on Github and the usual places.

When Adrian asked who in the room was using Netflix’s OSS offerings, somewhere between a third and half of the attendees raised their hands. Fairly impressive for a movement that just four years ago brought responses of “there’s no way that could work, you’ll be back in datacenters in months.”

One key point he made was that if you’re deploying into a cloud environment, you want to be a small fish in a big pond, not a shark in a small pond. Netflix had to cope with the issues of being that shark for some time; if you are the largest user of a product you will likely have a higher number of issues that aren’t “oh we fixed that last year” but more “oh, that shouldn’t have happened.” Smaller fish tend to get the benefits of collective experience without having to be guinea pigs as much.

I’ve felt the pain of this in a couple of environments in the past, and I’m not even all that much of a bleeding edge implementer. It’s just that when you do something bigger than most people, the odds of adventure are in your favor.

The Good, The Bad, and The Ugly

The talk was called “The Good, The Bad, and The Ugly,” taking into consideration the big cloud announcements from Amazon’s AWS and Google Cloud Platform. There is plenty of coverage of these announcements elsewhere (I’ll link as I find other coverage of Monday’s comparison), but in short, there are improvements, glaring omissions, and a substantial lack of interoperability/exchange standards.

One item from the GB&U talk that I will call out is Microsoft Azure, which has graduated from “Other” to its own slide.

Azure’s greatest strength and greatest weakness is that it focuses almost entirely on the Windows platforms. Most companies, however, are apparently not moving *to* Windows, but away from it, if they are making a substantial migration at all. Linux is the lay of the land in large scale virtual hosting, and to be a universal provider, an IaaS/PaaS platform has to handle the majority platform as well as the #2 platform.

The unicorn in the cloud room is likely to be interchangeability between cloud providers. There are solutions for resilience within Amazon or within Google platforms, but it’s not so easy to run workloads across providers without some major bandaids and crutches. So far.

Time for Q&A: SLAs and where Cloud still doesn’t fit

Two questions were presented in this section of the opening keynote.

The first question was around service level agreements (SLAs). A tradition in hosted services, server platforms, network providers, etc… you don’t see SLAs offered in cloud platforms very often. You might think there were guarantees, based on the ruckus raised by single-availability-zone site owners during AWS outages over the past 2-3 years, but the key to making AWS (or other platforms) work is pretty much what Netflix has spent the last few years doing–making the service work around any outage it can.

This isn’t easy, or it would’ve been done years ago and we wouldn’t be talking about it. And my interpretation of Adrian’s response is that we shouldn’t expect to see them anytime soon. He noted that the underlying hardware is no less reliable than the servers you buy for your physical datacenter. And if you’re doing it right, you can lose servers, networks, entire time zones… and other than some degradation and loss of redundancy, your customers won’t notice.

The second question was heralded by Bernard Golden of enStratius Networks thusly, I believe:

I’ve taken to asking companies and tech advocates where their solutions don’t fit… because there is no universal business adapter (virtual or otherwise), and it’s important to have a sense of context and proportion when considering anything technological. If someone says their product fits everywhere, they don’t know their product or their environment (or either). 

Adrian called out two cases where you may not be able to move to a public cloud: Capacity/scale, and compliance-sensitive environments.

Capacity and scale goes back to the shark in a small pond conundrum. Companies on the scale of Google and Facebook don’t have the option to outsource a lot of their services, as there aren’t any providers able to handle that volume. But even a smaller company might find it impractical to move their data and processing environment outside their datacenter, depending on the amount and persistence of storage, along with other factors. If you’ve ever tried to move several petabytes even between datacenters, you’ll know the pain that arises in this situation (either time, technological complexity, cost, or even all three).

Compliance issues are a bit easier to deal with–only slightly, mind you. As Adrian mentioned, they’re having to train auditors and regulators to understand cloud contexts, and as that process continues, people will find it easier to meet regulatory requirements (whether PCI, HIPAA, 404, or others) using current-decade technological constructs.

So where do we go from here?

My take: Cloud may be ubiquitous, but it’s not perfect (anyone who tells you otherwise is trying to sell you something you don’t need). As regulatory settings catch up to technology, and as cloud service providers realize there’s room for more than one in the market, we’ll hopefully see more interoperability, consistent features across providers, and a world where performance and service are the differentiating factors.

Also, there is still technological life outside the cloud. And once again, anyone who tells you otherwise is trying to sell you a left-handed laser spanner. For the foreseeable future, even the cloud runs on hardware, and some workloads and data pipelines still warrant an on-premises solution. You can (and should) still apply the magic wands of automation and instrumentation to physical environments.


I am attending Interop on a media/blogger pass, thanks to the support of UBM and Tech Field Day. Other than the complimentary media pass, I am attending at my own expense and under my own auspices. No consideration has been provided by any speakers, sponsors, or vendors in return for coverage. .


Interop® is the leading independent technology conference and expo series designed to inform and inspire the world’s IT community. Part of UBM Tech’s family of global brands, Interop® drives the adoption of technology, providing knowledge and insight to help IT and corporate decision-makers achieve business success. Through in-depth educational programs, workshops, real-world demonstrations and live technology implementations in its unique InteropNet program, Interopprovides the forum for the most powerful innovations and solutions the industry has to offer. Interop Las Vegas is the flagship event held each spring, with Interop New York held each fall, with annual international events in India, London and Tokyo, all produced by UBM Tech and partners. For more information about these events visit

Three traps to avoid on the Internets

I have gotten into the habit of tweeting every few weeks about some of the most common basic misconceptions on a couple of buzzwords.

1. The plural of ‘anecdote’ is not ‘data.’

The first one still stands. And if you’re wondering, it’s also not ‘big data’ either. Or ‘data science.’

No matter how politically advantageous it may be to extrapolate a small number of observations as “proof” of a hypothesis, it just doesn’t work that way.

As Neil deGrasse Tyson mentions on the new Cosmos, humans are wired to find patterns where there are none. Sometimes, we decide the pattern we want to “prove” and only pay attention to the data that backs up that pattern.

But if you’re getting ready to use that one study, or those two or three incidents, to express ‘proof’ of your hypothesis… think about whether there are other hypotheses that might be just as valid. Or better yet, go get more data. Very little in this world is cut and dried, other than flowers. And jerky. So don’t be jerky.

2. If you have to tell people you’re disruptive, maybe you’re not.

This one was also a painfully obvious observation for me. If storage vendors say “we put a flash drive in our array, look at us, we’re disruptive” in 2014 (or even 2013), you’re not going to take them seriously.

The term has some value in moderation, but maybe find a less dramatic synonym for “innovative” or “unexpected” next time  you’re writing a blurb.

When you tell me your company, product, CEO, or incredibly attractive animated mascot is “disruptive” I think of a 2 or 3 year old in the middle of the room howling and throwing things around. And I’m the neighbor/friend standing there feeling bad for that kid’s parents who aren’t handling it very well. Is that the image you’ve spent beaucoup bucks trying to cultivate?

3. Give the absurdly long and pointless lists a break

cosmo-listsOkay, I’ll admit this was a stretch. I wrote the headline and had to make the data fit. See what I did there?

I know all the Social Media Gurus say that you have to have long lists of things to get people’s attention. And they each have one anecdote of when that actually worked. But how many of them can really make use of 16 new sex positions, or 173 ways to make your toes really sparkle, or 5,150 unbeatable ways to increase traffic to your website?

Keep the lists to a minimum, and remember that you’re probably far more interested in the list than anyone who might see a retweet of it in the next decade. But on the upside, your list is (probably) clearly anecdotal, so you’re dodging trap #1 above.

So where do we go from here?

Well, first, some of you might not know where that tag comes from. It’s a an old classic I heard on the Dr Demento show decades ago called “Spock On.” The video is below, or you can skip to the pertinent bit here.

(To debunk another classic faux pas… very few of the songs usually attributed to Dr Demento were “sung” or performed by him. He just made many of them famous, including Weird Al Yankovic and Elmo and Patsy. The one you might have actually heard the good Doctor on was his revision of “Shaving Cream”)

And now that your day is brightened a little bit… don’t make your readers step in a big pile of shaving cream (unless you are a disruptive SCaaS provider).

What terms are bugging you these days? I would add “game changer” (which is great if you’re running the boardgame room at a convention) and guru/ninja/rockstar, but I’m hoping for a cheerful and upbeat Friday so I’ll leave my gripes there. Chime in on the comments if you like.

DON’T PANIC: The new era of Gallifrey One?


Since a number of my blog followers are Doctor Who fans who might be interested, I’m doing a rare cross-post from #gallyhelp.

Originally posted on The Secret Guide To #gally1 (formerly #gally):

tl;dr: Gallifrey One 2015 tickets are all sold out. Ticket transfers open in October, at face value. There is no shortage of hotel rooms. Gallifrey One will not be expanding. And what’s with the kidneys?


Today at 10am Pacific time, Gallifrey One sold 3200 tickets to the 2015 convention in seventy-five minutes.

Think about that, I’ll give you a moment.

DISCLAIMER: While you’re pondering, I’ll remind you that this is an unofficial site not affiliated with Gallifrey One, and I’m just a guy who’s been to six Gallifrey One conventions and likes to try to help folks who are attending or want to attend. 

That’s 7/10 of a ticket every second on average. Let’s round up. One ticket a second, for a show…

View original 1,040 more words