Beware the Island of Automation

If you know me, you know how much I like to share stories from our work with clients—names omitted, of course! I could spend all day giving you advice. Do this. Don’t do that. Definitely, don’t do that. But, I believe it’s the stories that allow you to relate this advice to your own experiences and challenges. Given how often I see today’s story unfolding, I’ll bet it’s a familiar one to some of you.

What do you mean the backup doesn’t work?
Company X has multiple locations, each with similar IT infrastructures and an IT manager who oversees everything at the location. I call these “islands of automation.” As you’ll see, this is not the kind of island you want to get stuck on.

One fateful morning, the server crashed in one of these locations. Not a good way to start the day under the best of circumstances, but things went downhill quickly when management discovered their systems hadn’t been backed up in over three months. The IT manager had tried to do backups but ran into some issues. Like many of today’s IT managers who are responsible for “everything IT,” his schedule was hectic, and he never managed to get back to fixing the issue with the backup systems.

Luckily for this organization, the story has a happy ending. The somewhat desperate IT manager called one of his counterparts in another office who had experienced the same issue and was able to talk him through fixing the server. Everything was back up and running, with no data lost. In fact, the only losses were the few hours of downtime and a little bit of the IT manager’s professional reputation.

Thankfully, this incident didn’t affect the organization’s mission-critical systems, but it could have. Imagine if this had been the data center housing your mission-critical data or systems. Could you handle your systems being unavailable for hours? How much data could you afford to lose before it impacted your business? How long would it take you to recover?

The obvious moral of the story is: Test your backups. Believe it or not, more than half of IT managers say they never test their backups, and those are just the ones who admit it when surveyed. The percentage could be much higher, especially in smaller companies where IT managers are often pulled in a dozen directions at once.

However, there is another moral to this story that could have an even more significant impact on the success of your business.

Organize your business like a Managed Service Provider
Malcolm Gladwell claims it takes 10,000 hours of practice to master any area. If your IT managers work a full 40 hours a week—no time off for lunch, no trips to the vending machine, no vacations—it will take them a full five years to master an area. And, that’s if they’re only wearing one hat. The way Company X was organized, with each location as its own separate island of automation, no one had the bandwidth to master anything. Even the technician who solved the problem was only able to do so because he had already experienced the pain himself.

As a Managed Service Provider (MSP), we’re able to develop the kind of deep expertise our clients need because we dedicate people to specific areas: security, database administration, security systems, VOIP, etc. For especially complex areas, like security, we have several people working as a team to ensure we stay abreast of the threats and most current risk mitigation tools and strategies. Whether our multi-site IT clients outsource some of their day-to-day management tasks to us or not, we often recommend they organize their IT staff the same way.

We’re not suggesting that you need to replicate the staffing across locations. In fact, just the opposite. With today’s connectivity solutions, there is very little that cannot be done remotely. And, putting this concept into action is relatively straightforward:

1/ Assess your IT staff’s current skill sets and aptitudes.
2/ Identify your most important tasks.
3/ Assign individuals to those tasks.

Minor tasks can be assigned as secondary priorities, or better yet, if they are time-consuming, consider outsourcing them to get them off your plate entirely. Also, as you look at realigning your human resources to your business needs, you may spot areas where you need to hire additional staff or leverage an MSP to make sure you have everything covered. For example, if you don’t have anyone on staff who has kept abreast of the latest cyber-security threats, consider bringing in some outside assistance to make sure you’ve got this vital area covered.

In addition to helping you sleep better at night, you’ll be making your IT department a better place to work and better utilizing your existing talent. In my experience, people who go into IT like to be the expert. Plus, with a bit of cross-training, i.e. a backup expert, they can finally use those vacation hours your HR team has been bugging them about.