Category Archives: Soft Topics

Join me on Thwack next week!

Posted on by
Reply

I was going to make a joke about an uncomfortable unicorn and tooting my own horn… but I’ll leave that as an exercise to the reader.

Seriously though, starting next week I’ll be a Thwack Ambassador on the Thwack Community hosted by SolarWinds.

Thwack is a community for people who manage, monitor, or deploy stuff. There are consultants, administrators, a few SolarWinds gurus, and generally people interested in having an easier life in systems/network/virtualization administration.

I’ll be sharing, and soliciting, some thoughts on topics around alerting and oncall, for the next four weeks. I also hope, if I can, to use, in a manner of speaking, way more commas than one, in his right mind, that is, might use. </shatner>

SolarWinds, as you may know, produces software to monitor and manage networks, servers, virtualization, logs, and more. They have a number of free products that you can use, from an SSH client and a very popular tftp/sftp server to IP address management and a subnet calculator (great for those of you with ten fingers rather than two). But this isn’t about their products.

#include <disclaimer.h>

I am paid a bit for my stint as a Thwack Ambassador. I am given a very broad theme to work from, but they don’t get to specify what I write about. So as with my own blog, you’ll be reading what I find interesting and/or true.

Come see me on Thwack next week, share your thoughts (there’s bound to be a contest for people who interact with me and my co-ambassador for March, Jeremy Stretch), and hopefully you’ll find (and share) some new ideas and solutions.

Update: My first post is up: Let’s just eliminate alerting altogether, okay? Check it out for a chance to win an iPod Nano!

Have you hugged your server today?

Posted on by
Reply

(Warning: Please don’t take this as an admonition to engage in unwanted intimate contact with waiters or other hospitality personnel.)

But seriously, have you messed with hardware recently?

Just a few years ago, the thought of a senior sysadmin who didn’t know the current de facto standard platform like he knew his kid’s first name would’ve been unthinkable. But as more people move toward cloud, virtualization, siloing, and general service provider clientship, it’s not unthinkable anymore.

I feel like one of a dying breed in my own environment, a sysadmin with live hardware skills. I primarily support departments that have high compute/ram/IO requirements, so until we’re ready to put Nutanix’s Hadoop workload platform to the test, we’re still doing analytics on bare metal. (Virtualize Hadoop? That’s like running Oracle on NFS. Oh, wait.)

I’m okay with this, as I’ve been doing the dirty work of hardware, from cardboard disposal to troubleshooting blinky lights for most of my two decades in technical operations.

But the challenge comes in when a coworker, who’s more into the “devops” and “software defined career” side of technical operations, gets tasked with figuring out why one of my hardware servers won’t boot, or why Kickstart/Jumpstart/FAI doesn’t find a hard drive. You can’t run your VMware CLI to figure this out, especially if VMware has never touched the machine, and Nagios won’t give you any hints if there’s no OS.

Be our host, be our host

If you’re in the final job of your career, at a company that uses no physical hardware, then this probably doesn’t apply to you.

But for the other 99% of you, if you haven’t dealt with hardware recently, it might be a good time to do so. Find an inexpensive but server-grade system and start messing with it. Get some familiarity with the current interconnects, PCI-e connectors (and why x16 isn’t always x16), and perhaps most poignant to my experience, how to work your way around a RAID card.

I recently picked up an older workstation, an HP XW9400 dual Opteron workstation, that’s perfect for this sort of effort. It is a heavy duty piece of hardware, with a 1000w power supply, but it has PCI-X, PCI, and PCI-e slots enough to use almost any gear I want to test out. It also has an onboard 8-port LSI 1068 SAS controller, which is the same family as I work with on Cisco and Dell hardware platforms.

I can also throw a PCI-X Fibre Channel card in, or a 10 Gigabit Ethernet card, or whatever I may want to mess with next. The one missing piece that every sysadmin should have some awareness of is lights-out management, so if you haven’t done iLo or iDRAC or CIMC or the like, you may want to consider a slightly outdated (read: affordable) HP/Dell/Cisco server with the risk of having fewer slots for expansion. The flavors are different but the behavior and expectations are close enough.

Buddy can you spare a server?

If your company isn’t throwing hardware away, consider eBay or your local Craigslist to source a base server, upgrades if needed, and whatever else you want. I built my first hadoop cluster on a pile of $100 HP DL180 servers off eBay, and continued buying gear from the seller who always had a supply of spare parts. Most of them won’t complain if you buy from their web store vs eBay, or the other way around–whatever your budget and preferences allow.

You can find a few of the XW9400 workstations for $200ish buy-it-now with a reasonable base config, if you don’t want to wait for an auction to finish. Don’t worry about the OS license, and make sure you check the shipping cost before bidding/buying.

But where do we go from here?

So you have this nice 1000w server and you’ve figured out the ins and outs of RAID configuration. What now?

Well, you were looking for a nice new space heater, right?

Seriously though, you probably have a good start for your home lab now. For me, this machine will probably be good for testing PCI-X and PCI-e cards going forward, but if I get past that stage, it’d make a great storage server. Or for that matter, a reasonable virtualization box, with a bit of upgrading. I pulled up the quickspecs on HP’s website for this model, and they sold hex-core Opterons for these workstations, and they support up to 64gb of RAM. Mind you, the 64GB of RAM would cost me about $2k on eBay, so that won’t happen this month… but 32GB of RAM will set me back less  than $200. I can probably stash half a dozen disks inside, or use a $40 SAS adapter to connect an external array.

If you got a rackmount box, you may not want to run it at home for the sake of domestic tranquility. But if you have a quiet closet, rack space at work, or the budget for a small colo service, give it some thought. Check your upgrade paths, or find a good use to fit your current config. Or worst case, if you kept the box, post it on eBay.

I have a stack of servers in my lab waiting for triage, so there may be a hardware troubleshooting post coming in March.

References:

  • Check out my TFD colleague Chris Wahl’s posts on his home lab category.
  • HP QuickSpecs. Good place to see options for HP workstations and servers, before digging into eBay

Got other links you think my readers would relate to this topic? Any thoughts of your own to share with other readers? Send me a comment below.

Maybe Yahoo’s Mayer Made The Right Decision After All

Posted on by
Reply

Marissa Mayer, CEO of Yahoo, and Jackie Reses, their “EVP of People and Development,” made waves last week by apparently banning working from home (and to some extent, flexible schedules).

As a worker who can do 90% or more of my work from somewhere other than my own company’s offices, I take some offense at this, and it makes Yahoo another company I’d not want to work for.

But I can see why Yahoo’s management would do this, and it may well be the right choice for them.

The losers in this decision are many of the workers with families, long commutes, and/or existing agreements with their direct managers to accommodate working from home. Also losing out will be their coworkers, and anyone counting on their efficiency and effectiveness. Also the recruiters trying to bring in Silicon Valley talent with this albatross around their necks, the people on understaffed teams counting on new talent coming in, and quite possibly Yahoo’s shareholders.

I’ve been in a couple of environments where working from home (briefly) became a rift. In both cases, there was a problem with a very small number of employees who couldn’t work effectively from home, and rather than actively managing those few people, management attempted to overmanage a large group.

office-space-face-to-faceIn one environment, the politics at the director level and above were toxic, and were chasing out good talent in droves. I think 2-3 of a dozen coworkers were still there at the end of six months, including none of the people who interviewed me. The director who stepped in felt that people wanted to leave (he was correct), and that the best way to keep that from happening was to quietly cancel work-from-home without communicating this change (he was incorrect). There were a lot of people who “had to take [their] cat to the vet” for a month or two, even people with no cat, and I hear the director finally figured out that he was doing it wrong. I was gone by then as well.

In another case, about two or three employees were abusing work-from-home privileges, sending their “I’m working from home” email at 2pm (if at all) and not answering their phones. So the person in management responsible for these people’s managers declared that anyone wanting (or needing) to work from home more than once a month had to take sick time to do it, even if they were working the whole day. This was thwarted by HR, to the company’s benefit–silly state laws would’ve gotten in the way anyway–but it communicated management’s respect for employees very clearly.

Don’t get me wrong here. Working from home (unless contractually obligated) is a privilege, and if it’s abused, it can and should be revoked on a per-employee basis. It should also be a matter between the abusing employee and his/her manager, not the C-levels and the press.

But if you want to acquire and retain the kind of talent that Silicon Valley thrives on, you need better leadership, not just suboptimal management and one-size-fits-all backtracking mandates. I think people wanted to believe Marissa Mayer brought that sort of leadership to Yahoo, but a lot are thinking better of that belief now.

Some references:

Update: Some interesting articles on the matter from Business Insider:

What do you think? Is this another step in the recovery direction for Yahoo? Does it change how you view them as a prospective (or current) employer?

Making your way in the world today…

Posted on by
Reply

One of the most unusual things about my current job, other than not having to explain what my company does (at least in general and 4/5 of the specifics), is getting to travel for business.

Having come to the point where the waiters and front desk staff at one hotel in Las Vegas recognize me, I thought I’d share some suggestions and insights for lower grade business travel.

Some of my professional friends, like Stephen Foskett, travel a lot more than I do. Stephen has some useful recommendations on frequent flyer programs, worth a look.

But here are some thoughts from my measly dozen flights in the last year and change. Oh, and some Amazon links (not all of them), in case you want to help with my upcoming personal travel accessories.

Loyalty Programs

Sign up for them. You will probably see benefits on your first stay.

I learned this about 12 years ago when I joined Wyndham’s hotel rewards program. My first stay, thanks to that membership (obtained a few days before I made my reservation) got me a deep discount and every 5th night free on a two week stay in a Key West resort with an indirect ocean view. So about $100/night for a room that I later saw going for $400+ a night. You won’t always get so lucky though.

Check with your company’s travel group, or administrative assistant, and see if you have corporate discount agreements. Sometimes it’s that your admin’s wife works for the local branch of a car rental company, or maybe your company puts enough butts in car seats to get a discount.

Failing that, figure out who you like for air, car, and hotel, and sign up for their plans. If you travel as uncreatively as I do, you can get by with one of each, but as Stephen says, try to get balance/coverage.

Watch for points expiration. Some airlines (Delta, Southwest) don’t expire their points. Others may have 12, 18, or 24 month expiration. Generate some activity once a year and you’re probably going to be okay. Sometimes you can move points between air and hotel, or get points for airport shuttles and car rentals, so make the most of it.

If you change your airline preferences, don’t throw your points away. Most points programs will let you spend the points on certain retail purchases (flowers, magazines, gift certificates) which is better than throwing them out. For example, my fiancee’s V-day flowers used up my about-to-expire United miles last week, so we both win.

If you’re going to Vegas or another resort destination, you can sign up for the resort chain’s rewards program and get bonuses for dining, attractions, and hotel stays. They’re geared toward rewarding high rollers in the casinos, but you can get discounts and benefits on less expensive pursuits as well.

It’s also worth looking into a rewards credit card, if you want to supplement your personal travel with expensed travel charges. Check the fine print and consult a tax professional just to be safe though–bonus points/miles/rewards may be taxable in some jurisdictions and situations.

Packing to Go

Everyone has their little quirks to packing for travel. I’m  usually still jamming stuff into a suitcase an hour before I leave. But do as I say, not as I do.

Get change for a twenty and stuff it in your wallet or purse. You’ll see why later.

Keep a spare set of chargers/cables in your travel bag. Consider a portable battery as well–I have a iSound 16,000 mAh 5-port usb charger because my phone and ipad take a lot of juice. But you can get much smaller ones for a single device, for those times when you forget your charger at the hotel or can’t find an open outlet at the airport.

Plug in whenever you can. NaNoWriMo taught me this one, when I lost the third battery in my D830 about 20 minutes before the deadline. Battery indicators are infamously capricious, and you shouldn’t count on Office’s auto-save to save your butt. Keep your backup battery charged too. And remember that the larger batteries must be carried on.

I carry my own coffee gear wherever I go. Some hotels have good single serve so I just bring extra pods and raw sugar. Others have disastrous coffee options (I’m looking at you, Seattle financial district) so I bring a Clever Coffee Dripper or Aeropress and some fresh ground coffee.

And one of the best choices I’ve made recently was to get a lay-flat checkpoint-friendly laptop bag. No more pulling the laptop out and taking up a third plastic bin. Targus has a few that are pretty good, especially if you have a big laptop (mine’s 16.4″ diag and thick too), but it’s worth checking local thrift stores if you’re on a budget. The bag I bought was about $12.95 at Goodwill, new with tags, and somewhat resembles the Mobile Elite in size/form factor.

Once You Get There

I recommend checking for the nearest grocery, drug store, and electronics shop to your hotel or work destination. You’ll probably forget something, or find a need for something unexpected while you’re there. And it might save you a couple thousand on room service bills as well.

Something to keep in mind if you need something predictably and affordably but realize it too late to shop before you leave… Amazon will ship to a third party address (i.e. a hotel), and with Prime shipping, you can easily decide on Tuesday afternoon that you need, say, that iSound battery, order it, and have it waiting for you at the hotel on Thursday. You could probably do this with clothes and snacks as well. Check in advance to see the vendor’s policy on non-credit-card-billing-address shipping addresses. Newegg can be a challenge on this, although you can get another address added as an alternate shipping address by calling your credit card company a few days in advance.

And here’s where that change for a twenty comes in.

Housekeeping at a hotel is a thankless job, for the most part. You may have a rough day now and then, but if you’re traveling for business, you probably haven’t had to clean up bio spills as part of your job, for at least a few months. Leave a buck or two a night for the cleaning folks and you’ll probably make their day a little bit brighter. Do it each night, rather than the last night, since crews may change from day to day. If everyone left $2 a night for the housekeeping staff, in most hotels they’d probably make as much as most of the guests. And if you have to vomit, find a trash bin or the toilet, not the curtains or the carpet.

Coming Home

The last time I went to Disney World, I realized I was going to be getting tee shirts and souvenirs with my cast member discount, and didn’t really want to check two bags coming back. So I set myself up to ship the souvenirs home separately. Get a flat rate box or two from the post office, print out flat rate postage labels at usps.com, and put them at the bottom of your suitcase. When you’re packing up to come home, fill up the boxes, stick the labels on, and ask the front desk to give your packages to the mailman. If you don’t end up using the labels, you can file an e-refund and shred them when you get home. Cheaper than over-weight baggage fees as well.

Now it’s your turn

What suggestions do you have for streamlining business travel? What makes your stay away from  home more comfortable and convenient? What have I forgotten to mention? Chime in the comments.

Whoa, I did something interesting…

Posted on by

One of the greatest barriers to my blogging is that I have to find something I believe people will want to read about. I’ve set myself up to write about a couple of things, but decided they wouldn’t be as interesting after I poured them into a text box on the WordPress site. So you’ll see a few gaps in my timeline here, which is odd for someone who’s been accused of being “one of the bloggers.”

I’m working on some details on my recent data migration project, which I think will help a few people learn something through my efforts. Watch for the first installment this week.

I’m also planning to do some Fibre Channel SAN stuff in ways you don’t normally see on your VAR’s cut sheet.

And I’m coming up on the one year anniversary of the Andromedary Instinct store, hopefully with some store events to build on my eBay sales of the past year.

A lot of you keep coming back to look at my home lab post. I have actually rebuilt the NAS (went from FreeNAS 8.x to NexentaStor CE and then went to clearing off the living room coffee table for the holidays), and have not been doing as much with the ESXI 5.0 side of things either. But that should take off soon as well.

So stay tuned, I’ll try to get up to the point of a blog post a week, or at least one a month. I appreciate the comments and suggestions, and welcome your thoughts going forward.

 

[2013-06-28: Comments disabled since this page seems to be drawing the automated scourge of the earth. You can contact me through other pages or about.me/rnovak if you like.]