Machine Learning Authors: Zakia Bouachraoui, Liz McMillan, Yeshim Deniz, Pat Romanski, Elizabeth White

Related Topics: Java IoT, Industrial IoT, Microservices Expo, Microsoft Cloud, Machine Learning , Agile Computing

Java IoT: Article

Analyzing Shades of Grey

...It’s no longer about red, yellow and green

Over the last year there have been an influx of new graphical user interfaces (UIs) as monitoring and dashboard vendors scramble to keep up with large enterprises increasingly focused on understanding the business impact of IT, and Netuitive is no exception.

Today’s enterprises are challenged with monitoring and interpreting increasing volumes of infrastructure performance, application data, and business metrics.  Like Big Data, advanced predictive analytics have moved to the forefront for their ability to identify anomalies that could lead to business and revenue impacting performance problems.  As such, the UIs visualization capabilities have become an increasingly important means to this end, and ultimately to proactive management of critical application performance.

But to understand where the UI is now, and where it must go, one must look back at how APM use cases have evolved and changed.

In the “old” world, the Big Four and APM tool vendors developed dashboards that monitored individual data sources visualizing results in green, yellow, and red buttons, bar charts, and pie graphs.  Basically, all these did was show when a server or storage platform was either over-utilized or broken, and that you needed to physically add capacity and/or replace it.  They were no sophisticated algorithms and it was up to the user to interpret any impact on IT performance now and in the future.

Over time, data sources continued to increase in volume and variety, virtualization happened, and complexity increased to the point of making it  humanly impossible to monitor and interpret IT and application performance … let alone understand business impact.

To address this, IT operations, particularly those large enterprises with mission critical applications, began using IT analytics to monitor real-time data and metrics to identify anomalies that were leading indicators of future performance problems.

Fast forward to 2013.  The volume of performance and application data and metrics are orders of magnitude higher and IT operations analytics are now central to any advanced monitoring strategy. As such, the demands on the monitoring tools have changed significantly.  It is no longer simply about green, yellow and red buttons.  While they are still primary indicators of the “health” of an application or business service, it is now about being able to process and automate the analysis and correlation of multiple data types with analytics that detect and interpret anomalies.  Today’s tools can store and process large quantities of information.  Advanced algorithms assist in highly synthesized, meaningful visualization that you can use it for decision support.

However, the industry overall is still struggling to develop processing that is sophisticated enough to identify the “anomaly in a haystack” that may be the key indicator of future performance degradation and/or service interruption.

The focus is now on developing UIs that assist in the detection of anomalies and allow you to act on them in real time.  And not every anomaly is a problem.  You need to be able to filter them.  And when an anomaly does happen, you need to quickly determine 1) whether or not this anomaly is going to lead to a real problem? 2) and if it is an early indicator, what do you do about it? The new UIs are focused on helping you address that.

In Netuitive’s case, we focus on the IT infrastructure, application data, and key business metrics associated with a particular mission critical application or business service with the goal of understanding business impact.  We integrate our predictive analytics technologies as part of large scale APM monitoring solutions that focus on the key IT performance, application data, and business metrics for a particular business service instead of trying to boil the ocean across the enterprise. We bring all of this analysis and correlation for a particular business service or application together in our new UI that visualizes the correlation of IT performance, application data and key business metrics in a single screen at your fingertips.  It also integrates with local knowledge bases to further extend its ability to distinguish between true anomalies and known trends based on historical behaviors.   It is essentially a bridge from the old approach to the new.

Looking forward the UI will need to continue to evolve. Data will continue to increase, analytics will need better algorithms, and the goal will continue to be about finding anomalies and addressing them before an issue cascaded into a larger performance problem or service interruption.  The UI will need to process large amounts of data and help you determine if an anomaly is a problem or just an anomaly, and what do you do about it…proactively.

Ultimately it is this proactive approach that allows you to prevent impact on the business.  And to achieve this, the new UIs need to help you understand the impact on the business.

More Stories By Marcus Jackson

Marcus is Director of Product Management at Netuitive. He is responsible for the direction of Netuitive's flagship product, including analytics and data visualization. He has over 20 years of experience in software engineering and performance management. Previously, he headed development for Netuitive and the IEEE Computer Society. Marcus holds a bachelor's degree in Computer Science from Harvard University.

CloudEXPO Stories
The challenges of aggregating data from consumer-oriented devices, such as wearable technologies and smart thermostats, are fairly well-understood. However, there are a new set of challenges for IoT devices that generate megabytes or gigabytes of data per second. Certainly, the infrastructure will have to change, as those volumes of data will likely overwhelm the available bandwidth for aggregating the data into a central repository. Ochandarena discusses a whole new way to think about your next-gen applications and how to address the challenges of building applications that harness all data types and sources.
DXWorldEXPO LLC announced today that Big Data Federation to Exhibit at the 22nd International CloudEXPO, colocated with DevOpsSUMMIT and DXWorldEXPO, November 12-13, 2018 in New York City. Big Data Federation, Inc. develops and applies artificial intelligence to predict financial and economic events that matter. The company uncovers patterns and precise drivers of performance and outcomes with the aid of machine-learning algorithms, big data, and fundamental analysis. Their products are deployed by some of the world's largest financial institutions. The company develops and applies innovative machine-learning technologies to big data to predict financial, economic, and world events. The team is a group of passionate technologists, mathematicians, data scientists and programmers in Silicon Valley with over 100 patents to their names. Big Data Federation was incorporated in 2015 and is ...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like "How is my application doing" but no idea how to get a proper answer.
All in Mobile is a place where we continually maximize their impact by fostering understanding, empathy, insights, creativity and joy. They believe that a truly useful and desirable mobile app doesn't need the brightest idea or the most advanced technology. A great product begins with understanding people. It's easy to think that customers will love your app, but can you justify it? They make sure your final app is something that users truly want and need. The only way to do this is by researching target group and involving users in the designing process.
CloudEXPO New York 2018, colocated with DevOpsSUMMIT and DXWorldEXPO New York 2018 will be held November 12-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI and Machine Learning to one location.