The enemy within

Internal threats to uptime have to be dealt with as assiduously as external ones.

A couple of months ago, I wrote about prioritizing security issues based on the real financial risk they potentially represent. This month, I’d like to return to that issue again.

With all the media hype surrounding hacker attacks and so-called “cyberterrorism,” it would be easy to think that the greatest threats to network uptime come from outside the enterprise. However, as most of us know, that’s not the case. According to all available data, the security incidents continue to be caused primarily by internal users. Even more surprising to anyone who would live and die by mainstream reporting is that the overwhelming majority of those incidents are accidental, rather than malicious.

Anyone who has worked in a technology organization for any length of time knows this. People are always deleting critical files, screwing up routing tables, or taking essential resources off-line. These actions cumulatively result in more lost productivity and compromised customer service than any denial-of-service attack. Just look at the uptime struggles of companies like eBay or service providers like AT&T. They have more to do with poorly installed patches and badly configured servers than they do with viruses or hacker exploits.

Of course, none of this is news to Communications News’ readers. And yet we continue to focus on firewalls, virus protection and encryption tools as if they offered us salvation from the scourge of downtime. They can’t and won’t. However, they are certainly more straightforward and simpler to implement than the solutions for our internal problems. And therein lies the rub.

PROCESS, NOT PRODUCTS

The solution, it turns out, for reducing the frequency of internally caused problems has more to do with processes and policies than it does with installation of any specific tool. In particular, technical organizations have to become more disciplined in their change-management procedures. When the consequence of a poorly executed change was loss of a LAN segment for 45 minutes, technical staffs didn’t have to be so rigorous about such things. Now that such errors can directly impact customers and supply chains, on the other hand, discipline has become fairly critical.

One of the biggest obstacles to implementing formal change-management practices is that, well, they require change. Network jockeys have never had as formalized worklives as, say, mainframe administrators—whose established procedures are not totally unlike a modern flight crew’s. Yet, as any expert in aviation safety will tell you, the preflight checklist—as elementary as it may seem—was perhaps the single most effective safety measure ever instituted in the history of flight. Procedures work; flying by the seat of your pants doesn’t.

Another challenge is complexity. Corporate IT environments now consist of more moving parts than ever, each of which is running more lines of code than ever. If you give network managers and systems administrators more complex stuff to manage, then they naturally tend to get more and more wrapped up in managing their own personal “turf.” But effective change management requires that individual turf owners communicate better with each other. Therefore, when you try to institute new procedures, you’re really adding to the strain on staffers who are already fairly maxed out as it is.

Finally, there is the rate of change. New patches and bug fixes appear relentlessly on vendor and third-party tracking sites. As changes have to be executed more often, it becomes increasingly difficult for administrators to be disciplined about how they implement those changes. Most technicians, for example, never actually test the security patches that they install. They simply assume that the problem is taken care of once the new code is running. That’s simply bad business.

EDUCATE YOUR EMPLOYEES

This isn’t just an issue that affects technical staffs. End users need to be educated about safe computing practices, as well. After all, if new employees don’t know enough not to open an e-mailed file attachment that ends in .vbs, then your ability to protect your company against virus attacks is going to be fairly limited. And if they don’t protect the data on their laptops while they’re on the road, then you can really be out of luck.

While it’s certainly important to protect the enterprise from malicious external security threats, network managers should be careful not to get distracted from the real problems that continue to cause costly downtime day in and day out. Technical staffs need to implement the collaborative applications and processes necessary to track changes, assess their impact, and execute immediate rollbacks when unexpected results occur. By addressing these critical internal inadequacies, IT organizations can effectively safeguard the companies they serve from breakdowns in e-business processes that can result in lost revenues and ruined relationships.

By the way, the same principle holds true for most of us in our personal lives. Business advertising likes to paint a picture of the business warrior who boldly takes on a hostile world of bloodthirsty competitors, irrational customers and incompetent teammates. Most of us, if we’re honest with ourselves, know that we are really our own worst enemies. Our inconsistent motivation, our bad attitudes, our dimmed understanding of people and situations, and our lack of real vision is usually what prevents us from realizing our goals—not the little obstacles that the outside world puts in our way.

Walt Kelly was right when he said through his cartoon creation, Pogo, that the enemy is us. The trick is to recognize that fact and deal with it appropriately.

To comment on this month’s column, contact me at ll@exit109.com.