Welcome!

Machine Learning Authors: Mark Ross-Smith, Jyoti Bansal, Kevin Jackson, Lucia Schöpfer, William Schmarzo

Blog Feed Post

Automation through workflow state

The benefits of automation are well understood: more agile service provisioning, faster time to insight when there are issues, and a reduction in human error as manual interaction is reduced. Much of the premise behind long-term SDN architectural advantages is steeped in the hope that SDN will help enable and ultimately promote automation. But while centralizing control has significant operational advantages, by itself, it doesn’t actually address the most important requirement for automation.

If automation is going to be more than just reducing keystrokes, there will have to be a rise of workflow state.

Referential space

Successfully managing a network is an exercise in constant iteration through network state. Whenever something needs to be done, the architect or operator examines her current frame of reference to figure out the starting point. That frame of reference usually starts with some implicit understanding of how the network is designed. From there, she takes some action. Maybe she pings an endpoint, checks the state of a BGP neighbor, or examines some interface statistics. Whatever the first step, the point is that she knows when she starts that there is work after the first step.

The information gleaned from the first step yields additional understanding. Her frame of reference changes as she now knows more than before. With her new position in referential space, she takes the next step. And the next, and the next after that. Each step yields a different piece of information, and the process of iterating through a constantly changing referential space ultimately yields some outcome or resolution.

Byproducts of iterative workflows

There are two major byproducts of this iterative approach to workflow. The first is that the starting point is rarely based on an absolute understanding of fact. Rather it is an interpretation that the individual operator or architect creates based on a number of somewhat soft conditions – knowledge, experience, intuition, whatever. This means that for each task, the workflow is somewhat unique, depending on the operator and the environment.

The impact here is important. If workflows are unique based on the operator and the conditions (i.e., the referential space or frame of reference), then the outcomes driven by those workflows are difficult to repeat. Part of why networking is so hard is that so much of it borders on arcane dark art. Science demands repeatability, but the very nature of workflow management in networking makes that challenging.

The second byproduct of networking’s iterative nature is that workflows frequently depend on a set of chained tasks, each of which has a dependency on the preceding task. To make things worse, that dependency is actually rarely known at the start of a a workflow. It’s not that tasks cannot be predictably chained – first, you look at the physical layer, and then you move up stack perhaps. But each subsequent task is executed based on not just the previous task but also the output of the previous task. This creates a complex set of if/then statements in most workflows.

Part of the challenge in automation is providing the logic to navigate the conditional nature of networking workflows.

“Network engineers need to think like programmers”

With the rise of movements like DevOps, “network engineers thinking like programmers” has become a popular phrase. This is a very important change in how we handle network architecture and operations. But there are subtleties here that get lost in the cliche.

First, when people toss the phrase around, they often mean that network engineers need to pick up a scripting language (Python, Ruby, even Perl). Thinking like a software developer has very little to do with programming languages. Languages are a way of expressing intent, but it’s entirely possible to know Python and think nothing like a developer.

Second, when people refer to programming in the context of DevOps, they generally mean that network operators need to think about configuration less as a collection of commands and more like code. Once you make that shift, then you can think about things like source code management, automated testing, and rapid deployment.

But networking needs to do more than just treat configuration as code.  DevOps has more to do with deploying and validating changes. It doesn’t fundamentally change how workflows are executed, and it barely touches more operational tasks like troubleshooting network conditions.

Before anyone picks a religious battle over DevOps here, my point is not that DevOps is bad. It’s just that DevOps by itself is not sufficient. And there are things that ought to be done that are separate from DevOps.

Tiny feedback loops

So if thinking like a programmer isn’t about learning a programming language and it’s more than treating configuration as code, what is it?

Software development is really about creating something out of lots of tiny feedback loops. When you write functions, you don’t just execute some task. You generally execute that task and then return a value. The value provides some immediate feedback about the outcome. In some cases, the function returns the value of a computation; in other cases, it simply returns an indication that the function succeeded or failed.

These values are obviously then used by other functions, which allows us to string together small building blocks into complex chains. The important part? These chains can then be repeatably executed in a deterministic way.

Networking workflows shouldn’t be that different. Each individual activity yields some value (sometimes a specific value as when looking at some counter, other times a success or failure as with a ping). The problem is that while networking commands frequently return information, it is up to the operator themselves to parse this information, analyze what it means, and then take the next action.

Workflow state

What we need if we really want to make automation happen in ways that extend beyond just scripting keystrokes is a means of creating deterministic networking workflows. For this to happen, we need people who construct workflows to think more like developers. Each activity within a workflow needs to be a tiny feedback loop with explicit workflow state that is programmatically passed between workflow elements.

We actually instinctively do this at times. XML, NETCONF, and the like have been used to encapsulate networking inputs and outputs for awhile with the intent of making things parseable and thus more automatable.

But we stopped short. We made the outputs more automation-friendly without ever really creating workflows. So while we can programmatically act on values, it only works if someone has automated a particular workflow. As an industry, we haven’t gotten to actually addressing the workflow problem.

Maybe it’s the highly conditional nature of networking combined with the uniqueness of individual networks. Or maybe it’s that outside of a few automation savants, our industry doesn’t generally think about workflows the way a software developer would.

The bottom line

Networking workflows rely way too heavily on an iterative pass through referential space. The reason change is so scary and troubleshooting so hard is that very little in networking is actually deterministic. But if we really want to improve the overall user experience en route to making workflows both repeatable and reliable, we do need to start thinking a bit more like developers. It all starts with a more explicit understanding of the workflows we rely on, and the expression of feedback via some form of workflow state.

And for everyone betting on abstractions, just know that abstracting a poorly-defined workflow results in an equally poor abstraction. We need to be starting elsewhere.

[Today’s fun fact: Only male fireflies can fly. Take that, females!]

The post Automation through workflow state appeared first on Plexxi.

Read the original blog entry...

More Stories By Michael Bushong

The best marketing efforts leverage deep technology understanding with a highly-approachable means of communicating. Plexxi's Vice President of Marketing Michael Bushong has acquired these skills having spent 12 years at Juniper Networks where he led product management, product strategy and product marketing organizations for Juniper's flagship operating system, Junos. Michael spent the last several years at Juniper leading their SDN efforts across both service provider and enterprise markets. Prior to Juniper, Michael spent time at database supplier Sybase, and ASIC design tool companies Synopsis and Magma Design Automation. Michael's undergraduate work at the University of California Berkeley in advanced fluid mechanics and heat transfer lend new meaning to the marketing phrase "This isn't rocket science."

@CloudExpo Stories
"There's a growing demand from users for things to be faster. When you think about all the transactions or interactions users will have with your product and everything that is between those transactions and interactions - what drives us at Catchpoint Systems is the idea to measure that and to analyze it," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York Ci...
The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal ...
@DevOpsSummit taking place June 6-8, 2017 at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @DevOpsSummit at Cloud Expo New York Call for Papers is now open.
SYS-CON Events announced today that Dataloop.IO, an innovator in cloud IT-monitoring whose products help organizations save time and money, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Dataloop.IO is an emerging software company on the cutting edge of major IT-infrastructure trends including cloud computing and microservices. The company, founded in the UK but now based in San Fran...
20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
In the next five to ten years, millions, if not billions of things will become smarter. This smartness goes beyond connected things in our homes like the fridge, thermostat and fancy lighting, and into heavily regulated industries including aerospace, pharmaceutical/medical devices and energy. “Smartness” will embed itself within individual products that are part of our daily lives. We will engage with smart products - learning from them, informing them, and communicating with them. Smart produc...
Internet of @ThingsExpo, taking place June 6-8, 2017 at the Javits Center in New York City, New York, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @ThingsExpo New York Call for Papers is now open.
SYS-CON Events announced today that Catchpoint, a leading digital experience intelligence company, has been named “Silver Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Catchpoint Systems is a leading Digital Performance Analytics company that provides unparalleled insight into your customer-critical services to help you consistently deliver an amazing customer experience. Designed for digital business, C...
“DevOps is really about the business. The business is under pressure today, competitively in the marketplace to respond to the expectations of the customer. The business is driving IT and the problem is that IT isn't responding fast enough," explained Mark Levy, Senior Product Marketing Manager at Serena Software, in this SYS-CON.tv interview at DevOps Summit, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart...
SYS-CON Events announced today that Catchpoint Systems, Inc., a provider of innovative web and infrastructure monitoring solutions, has been named “Silver Sponsor” of SYS-CON's DevOps Summit at 18th Cloud Expo New York, which will take place June 7-9, 2016, at the Javits Center in New York City, NY. Catchpoint is a leading Digital Performance Analytics company that provides unparalleled insight into customer-critical services to help consistently deliver an amazing customer experience. Designed ...
WebRTC sits at the intersection between VoIP and the Web. As such, it poses some interesting challenges for those developing services on top of it, but also for those who need to test and monitor these services. In his session at WebRTC Summit, Tsahi Levent-Levi, co-founder of testRTC, reviewed the various challenges posed by WebRTC when it comes to testing and monitoring and on ways to overcome them.
In their general session at 16th Cloud Expo, Michael Piccininni, Global Account Manager - Cloud SP at EMC Corporation, and Mike Dietze, Regional Director at Windstream Hosted Solutions, reviewed next generation cloud services, including the Windstream-EMC Tier Storage solutions, and discussed how to increase efficiencies, improve service delivery and enhance corporate cloud solution development. Michael Piccininni is Global Account Manager – Cloud SP at EMC Corporation. He has been engaged in t...
Cloud Expo, Inc. has announced today that Andi Mann returns to 'DevOps at Cloud Expo 2017' as Conference Chair The @DevOpsSummit at Cloud Expo will take place on June 6-8, 2017, at the Javits Center in New York City, NY. "DevOps is set to be one of the most profound disruptions to hit IT in decades," said Andi Mann. "It is a natural extension of cloud computing, and I have seen both firsthand and in independent research the fantastic results DevOps delivers. So I am excited to help the great t...
Every successful software product evolves from an idea to an enterprise system. Notably, the same way is passed by the product owner's company. In his session at 20th Cloud Expo, Oleg Lola, CEO of MobiDev, will provide a generalized overview of the evolution of a software product, the product owner, the needs that arise at various stages of this process, and the value brought by a software development partner to the product owner as a response to these needs.
In 2014, Amazon announced a new form of compute called Lambda. We didn't know it at the time, but this represented a fundamental shift in what we expect from cloud computing. Now, all of the major cloud computing vendors want to take part in this disruptive technology. In his session at 20th Cloud Expo, John Jelinek IV, a web developer at Linux Academy, will discuss why major players like AWS, Microsoft Azure, IBM Bluemix, and Google Cloud Platform are all trying to sidestep VMs and containers...
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex softw...
SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive ad...
Smart Cities are here to stay, but for their promise to be delivered, the data they produce must not be put in new siloes. In his session at @ThingsExpo, Mathias Herberts, Co-founder and CTO of Cityzen Data, discussed the best practices that will ensure a successful smart city journey.
In his session at 19th Cloud Expo, Claude Remillard, Principal Program Manager in Developer Division at Microsoft, contrasted how his team used config as code and immutable patterns for continuous delivery of microservices and apps to the cloud. He showed how the immutable patterns helps developers do away with most of the complexity of config as code-enabling scenarios such as rollback, zero downtime upgrades with far greater simplicity. He also demoed building immutable pipelines in the cloud ...