/var/blog
Industry and Technology Insights and reflections by Mike Bushong
/var/blog
Automation: More than addressing the mundane
04.10.17

For most people, automation is a set of scripts designed to make their lives just a little bit easier when handling the common day-to-day tasks that always seem to take longer than we wish. But if the end goal of automation is to take away the mundane, we are collectively aiming way, way, way, way, way too low. 
 
Ok, so where should be aiming instead?
 
Removing keystrokes
Automation in the networking industry is dominated by shell and Perl scripts. The primary tasks are a relatively small set of workflows that are repeated often and can be applied with very little tuning between different instances of a task. For the most part, the objective of these tasks is to reduce the number of keystrokes required to perform the task so that it can be done more quickly and with less likelihood of human error.
 
Making things faster and less prone to fat fingering are both noble goals. By themselves, they are worthy of pursuit. But if these are your sole objectives in achieving a more automated environment, there are easier ways.
 
If you are a business leader and all you care about is making your people more efficient at the keyboard, just send your entire operations team to an intense typing class. They will, in fact, take less time to type, and they will make fewer mistakes. 
 
More than keystrokes
Of course that is a ridiculous solution. But it speaks to what ought to be the natural targets for automation. Yes, automation should make things faster, but it isn’t the time it takes to key in configuration or operational commands that dominate the elapsed time between request and fulfillment. Let me give you an example.
 
A couple of years ago, I was talking to a large enterprise in Europe. They had been aggressively pursuing an automation agenda because an internal audit had yielded that it takes several months to stand up a new server. For this retail company, this created a competitive disadvantage as they were pushing to expand their online sales.
 
When they examined the workflow to get a new server online, they discovered that it wasn’t that the task was overly difficult. Instead, most of the delays were elapsed time as the change tickets were volleyed back and forth between the teams required to do the work. Additionally, they had to wait for a standard review committee meeting that only occurred every 6 weeks before they could push the changes out. The combination of coordination between teams and an ITIL review process took the time from maybe a couple of hours to more than 2 months on average. 
 
If we want to make the most out of automation, it’s the largest contributors to elapsed time that we need to target. Reducing the time it takes to type the commands is like shaving a few seconds off a cross-country flight. Should you do it? Yes. Is it enough? Unequivocally no. 
 
Where does most time elapse?
If we want to target the largest contributors to elapsed time, we need to understand where time accumulates. In the most general sense, elapsed time will accumulate at the boundaries of two things.
 
In the previous example, those boundaries can be between organizations. When people from different teams with different priorities are asked to collaborate, there is an inherent tax that must be paid whenever the action changes from one party to another.
 
In other cases, the boundaries can be between systems. Whenever two tools, for example, have to be used to do something, there is a manual step to take information from one and use it to provide context to the other. This typically requires a human to read the output, determine what to do next, and then execute the next step in the workflow elsewhere.
 
Why all the talk of artificial intelligence?
In true industry fashion, before we have nailed the previous thing, we are already moving on to the next big thing: artificial intelligence and machine learning. Forgetting for a moment the impracticality of layering even more complexity on top of something that has been insufficiently adopted, it’s instructive to understand what role AI and machine learning can play.
 
In most of workflows, the role of the human is to examine the data and determine next steps. The way we do this, at least in networking, is to pattern match. Networking for the vast majority of operators is less theory than practice. It’s actually pretty similar to the way medicine is practiced by most doctors. When you see a symptom that you have seen before, you prescribe a remedy. If the symptom has several remedies, you look for other adjacent symptoms. There is no magic here—it’s just pattern matching.
 
The problem is that humans are limited to matching only patterns they have seen before. So a network operator’s ability to correctly diagnose and eventually remedy a situation is dependent on her having seen something similar before. This is why every big issue is a big issue. If it hasn’t been seen before, working through to root cause is exceptionally difficult (especially given the fantastic ways in which things fail). 
 
One advantage of machine learning is that the models can be trained and, more importantly, shared. Effectively, we can create a pool of shared experiences, which allows the pattern matching to be done far more efficiently and to a depth no individual could ever hope for. 
 
The bottom line
Automation is a noble pursuit. But if we think that the best we can do is return a few minutes of time that would otherwise be spent keying in commands, we are setting the bar so low that we can just shuffle our feet over it. Automation ought to be about collaboration—between tools, people, and even organizations. 
 
And because automation will naturally apply at the boundaries, it means anyone focusing on workflows confined to a single device or set of devices needs to look around. There are larger windmills at which to tilt.

Top Kudoed Authors
User Kudos Count
4