Defining e-business
performance

The nature of online applications requires a rethinking of monitoring tools and metrics.

Those of us who have been in the industry for a while can remember the impact that the shift from mainframe-and-terminal architectures to client/server technology had on everything associated with networking. The effect on network management was particularly dramatic.

In the mainframe world, all the intelligence resided on the central processor. If your SNA network connections were up and running, you could be certain that end users were getting the application services they needed to do their jobs.

In the client/server world, that certainty disappeared. Suddenly, services to end users were dependent on multiple components—servers, routers, hubs and desktops—that all had to be functioning properly as individual devices and with each other.

Now, the ascendance of e-business is driving a new set of equally dramatic changes in how we manage and monitor the delivery of application services to end users.

An obvious change in the e-business world is that end users now include customers, not just internal employees. This obviously raises the stakes of the game. Failing to deliver adequate service means more than just a few lost percentage points of worker productivity. You can lose revenue.

Another important change is that now much of the service-delivery infrastructure is beyond the IT department’s control. The failure of an ISP, a Web portal provider, or even a customer’s own systems administration staff to keep its environment up and running properly can undermine delivery of an e-business service. But there’s a third issue that’s starting to crop up—content integrity.

Let’s say one of the functions you provide on your site is a natural-language search for certain types of events or businesses in any specified geographic area. How do you ensure that such a service is functioning correctly on your site?

One approach is to simply instrument all the infrastructure components that support delivery of the service: the servers, switches, network connections. It’s certainly necessary to monitor these components, but—as experience has demonstrated—it’s not sufficient. You can have a monitoring screen that’s clear of system or network alarms and still have customers calling to complain that something on your site isn’t working.

That’s why many technical e-business management teams also make use of services such as Keynote or Mercury Interactive to check whether specific URLs are accessible from various points across the Internet. These services can also track how long it takes pages to load, so site managers can detect incipient performance problems before they become critical.

But even services that track page delivery from the end user’s point-of-view can be inadequate. That’s because users of e-business services are looking for content, not technical performance. So, you need a monitoring system that can somehow check to see if the right content is being delivered.

Monitoring content integrity is no easy task. It’s not quantifiable in the same way that network latency is. It requires the intelligent interpretation of data on a delivered page. That’s not something most of us have had to think about doing before.

It’s also tough for management software vendors to help us with this challenge. After all, the nature of content varies so much from company to company. In the aforementioned example, the content is a city-specific listing. For a shopping portal, it may be a selection of catalog entries from other sites. A site like Eyeweb.com has to present the customer with a photo of his or her own face that has a selected eyeglass frame superimposed over it. If it’s the wrong face, or the wrong frame, or the superimposition doesn’t look right, then there’s a content failure.

Diligent e-business managers have solved this problem in a variety of clever ways. For example, one CTO who has to deliver the type of city-specific search mentioned above has created a make-believe location solely for testing purposes. Every few minutes, remote “robot” users query the site for data about the imaginary city. The result that the site returns is converted into a single number. If that number matches the expected result, all is well. If not, an alarm goes off and the technical team goes to work finding the problem.

Most sites, however, still depend on humans to check content, for the time being. Ultimately, such an approach simply won’t scale. You can also rely on your customers and other external users to tell you when there’s a problem. But you’d better be pro-active in soliciting feedback. Otherwise, they’ll just leave your site in disgust—rather than let you know you have a problem you need to check into.

The main point for anyone involved in site management to understand is that healthy Web servers and plenty of Net bandwidth are not sufficient to ensure e-business success. You also have to make sure that the data that end users are receiving is the data they want.