Network effects: Why automation is limited without integration
Apr 27, 2017
Most of the industry dialogue around automation focuses on tools. There are countless examples of people touting this API or that framework. There are templates and recipes and playbooks. We have libraries and GitHub repositories.
But until we talk more directly about the role of integration in making useful automation, we will have unnecessarily limited the potential impacts of automation.
For those who have not been following the automation discussion, let me briefly hit on two points that are critical to the rest of this post.
First, automation is about workflows. Workflows are the set of tasks that are typically chained together in support of some broader objective. If you do not know what the workflow is, you will never be able to automate it.
Second, the vast, vast majority of workflows extend beyond the network. They typically begin on the application or server side, and then include the network, but conclude with some validation step back outside the network.
Increasingly, IT is more about the whole of the infrastructure than any individual silo. Indeed, most of the new world is predicated on the notion that flat, scale-out applications can be orchestrated across an infrastructure consisting of compute, storage, networking, and the applications themselves. These applications demand an adaptive infrastructure capable of handling change. If that change requires the coordination of resources beyond just the network, then network automation alone will not be sufficient.
Put simply, integration is required whenever two or more resources coordinate to complete a task. If that task is to be automated, then the workflow automation must span the integrated actors. If the workflow is not automated, integration is not required.
Not one, but many
Integration between the network and the surrounding infrastructure is more than selecting a single tool and performing some deep integration.
Rather the IT infrastructure landscape increasingly features a multitude of tools, each of which has its own fan base. The rate of change in the tooling infrastructure actually exceeds the rate of change in the underlying equipment itself.
It is tempting to select a tool based on its install base. A large install base means a large path to market. But the dominant tools have huge install bases because of their longevity. In most cases, they were created in a time that has already past, and designed for an infrastructure that was not highly automated.
For the remainder of tools, it is impossible to predict with certainty which will emerge as winners. A far more likely outcome is a fractured market with pockets of people who align to specific tools based on market niche or peer influence.
Automation + integration
The real strategic value for operators lies not with either network automation or integration, but rather in the combination of the two. In fact, at a high level, it is probably objectively correct to say that either by itself is unlikely to have a material impact on the underlying infrastructure.
The networking community has pushed automation as a general theme for years. But when polling customers over the last several years in this specific area, most customers self-assess at just above CLI level in terms of overall automation.
The question to ask here is why?
Why network automation alone is not enough
When people talk about network automation, the workflows they immediately imagine are provisioning workflows. And when people think about provisioning, there simply are not ubiquitously applied workflows that apply uniformly across a lot of different deployment types. So when people get in groups to talk about what to automate, they invariably fall back to a small number of universal provisioning items (VLANs primarily, and then things like ACLs).
This means that, while very interesting, network automation focused on provisioning workflows represents a relatively small opportunity for improvement.
And the number of useful provisioning workflows decreases when those workflows are bound to the network alone. That is to say that if only workflows that originate on the network side are considered, the total is sufficiently small to wonder if network automation is impactful at all to the broader industry.
Monitoring and troubleshooting, not provisioning
The reality is that the bulk of the workflows to be automated (at least initially) likely are not provisioning but rather monitoring and troubleshooting.
By volume, operational tasks outnumber provisioning tasks in a typical network operator’s life. Moreover, those tasks tend to be executed under duress (when something is broken and requires rapid diagnosis and remediation).
It is worth noting here that monitoring and troubleshooting workflows are not completely unlike provisioning workflows in that they rarely originate or terminate in the network. There is almost always some external information that is required. It could be a packet sniffer, a flow analyzer, or even an application-monitoring tool. Whatever the tool, the point is that the workflow requires context switching between the network and things outside or on the network.
When the network is taken in isolation, automation is interesting but not necessarily difference making—at least not to the point of defining a market-moving opportunity for anyone. When you include other tools and infrastructure, automation becomes very powerful.
Imagine an infrastructure that includes various tools, each of which is a source of data. Based on information from one data source, action in another part of the infrastructure can be taken.
If, for example, application performance declines (latency, for instance), then the user might want to take a snapshot of operational state in the network (interface stats, buffers, and so on). In this case, a monitoring event detected by an application performance monitoring app triggers a troubleshooting workflow that is executed on the devices in the network.
Network effects generally refer to the effect that a user of something has on the value of that something. A good example in the high tech space is social media. Platforms like Twitter and Facebook have greater value to each individual user as more users join, which in turn attracts more users, which makes the platforms still more valuable.
The same dynamic is true for network automation and integration. The ability to automate workflows has more value as there are more high-value workflows to target. The more automation there is, the higher the value of integration.
This means that the network automation space will be best served by a large number of lightweight integrations that extend the potential umbrella of automation to more workflows.
The bottom line
As the industry begins to push a much more aggressive automation position, there is going to be a very interesting question for consumers: where should I start?
While, historically, the answer to that has been with out-of-box provisioning tasks, the likely go-forward answer is going to depend a lot more on the tooling infrastructure that is in place. Automation ought to start closest to the user, and that means that these tools—be it Ansible, SolarWinds, ThousandEyes, Splunk, Nagios, Wireshark, or something else—should be the starting point for meaningful automation discussions.
That, of course, requires that the automation efforts treat integration as just as important as automation.