Welcome!

Machine Learning Authors: Ed Featherston, Liz McMillan, Yeshim Deniz, Pat Romanski, Carmen Gonzalez

Related Topics: Microservices Expo, Java IoT, Industrial IoT, Microsoft Cloud, Machine Learning , Artificial Intelligence

Microservices Expo: Article

In Search of the Unknown

The importance of business process monitoring alongside of system monitoring,

I've talked at length about the importance of business process monitoring alongside of system monitoring, but in discussions I found that sometimes an overview and simple examples are not enough to convince people about the benefits of this approach. Business owners think they don't need to know anything about the operational performance of their systems as long as they have their numbers, and engineers often don't feel they need to invest time into understanding the business they are supporting in detail, finding examples shown too "common sense."

One question we ask our engineers during an interview at work is to describe the process of how they would go about troubleshooting a hypothetical issue, given only the minimum information. We often hear from our clients things like, "our website is slow" and "something's wrong with registration" with no additional information, and in order to figure out the potential issue we need to review the whole system at a glance. For large applications, with a myriad of moving and interweaving components, this is not an easy task. This is one of the reasons we are looking for best of the best. But if you are monitoring all of those components, in a lot of cases, the task can be simplified.

So let's examine a real problem. A large e-commerce company called and said that they are seeing less money coming in from web transactions. They have a pretty complex system with a lot of different revenue generation points, so this observation shed very little light on the root cause of the problem. Luckily, both systems and business processes were being monitored with Circonus, so the data was available to review.

As any engineer knows - step one of troubleshooting the problem is to confirm the problem, so looking at the revenue trends seemed like a good starting point.

The graph clearly shows that, starting around April 30th, the trend looked abnormal in comparison to the previous few weeks. So it seemed like there was an actual problem, and potentially, the issue could lie in payment processor itself or somewhere in the system, preventing certain users from making a purchase. So let's overlay the traffic trends, collected from Google Analytics, against revenue graph and see if there are any common trends.

Even though the traffic showed a clear drop at the same time as revenue, the ratio remained the same, allowing us to exclude payment processor and other application logic from the equation (for now).

Note: This is the first potential breaking point in the process. It is very tempting to look at the ratios, attribute revenue decrease to traffic decrease, and stop the investigation. 99% of the time, unfortunately, nothing "just" happens, so on we go.

Now for the next step - what would be a logical cause for a drop in overall traffic to the site? Response time is probably the first thought that should come to mind. So let's look at what the HTTP checks collected.

Load times didn't seem to be deviating from the norm, but the HTTP response metric doesn't provide full visibility into the load times for a dynamic application, so let's check the health of the database and CPU usage on the server(s), to validate that the underlying platform is not the bottleneck. There are numerous metrics to monitor database and system health that should be, and in this case, are collected, but when researching the root cause of the elusive problem, diving deep into a specific component can waste time early in the process.

Both of the metrics appear well within norm, so at first glance, it seems like the problem is not a systems issue.

Note: This is the second point of the investigation where the process can break down. A lot of technologists will either report that there is no confirmation to the problem reported; the reported problem is just an anomaly because the system monitors don't exhibit any issues. This is exactly why understanding of the business by the technology team is vital.

With that said, what would be the next logical process to validate? It is not uncommon for an e-commerce site to see a drop in purchases if they either stop promoting or if their marketing campaign is ineffective: traffic to the site slows down, subsequently decreasing the number of transactions. This company, in particular, sends out tens of millions of emails a day which bring in new users, and subsequently, new conversions. So let's take a look at the email deliverability and bounce rates collected from the company's MTAs.

Bingo! The bounce rates sky rocketed at the same time as the drop in traffic and revenue stream occurred. Upon closer investigation, it appeared that one of the major ESPs accidentally blocked the delivery domain, and the emails did not go through to the recipients. The issue was resolved (after some discussions with the ESP) and the trends returned back to the expected level.

Keep in mind, if email deliverability was not the issue - there are multiple other metrics that were on a list to be verified, both system (operational and development alike) and business. The amazing part of all of this is that I was able to view the whole system at a glance in just one graph. Granted, stacking everything on one graph is probably not the most optimal every day approach, but it is very useful in a certain cases when the direct overlay correlation is needed. For everything else - a real-time dashboard that displays all the vital points of the business at any given moment is a must-have for anyone responsible for business and/or system health.

Everyone responsible for the success of a business, regardless of the role, needs an ability to see the status of the whole business at a glance at any given point. System engineers don't need to know all the ins and out of marketing, but they should be aware of the overall organizational goals, and should be able to spot irregularities in the business trends. Similarly, CEOs don't need to know how systems work in the background, but should be able to correlate high email bounce rates (if it's critical to the business) to a decrease in purchases.

The point of all of this is that everything should be monitored, and to suggest some tools and methods that can enable users in all roles--within any organization--to ensure the success of the business. Get ‘em, learn 'em, use 'em! You will thank me later.

More Stories By Leon Fayer

Leon Fayer is Vice President at OmniTI, a provider of web infrastructures and applications for companies that require scalable, high performance, mission critical solutions. He possesses a proven background of both web application development and production deployment for complex systems and in his current role advises clients about critical aspects of project strategies and plans to help ensure project success. Leon can be contacted at [email protected]

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@CloudExpo Stories
Cloud promises the agility required by today’s digital businesses. As organizations adopt cloud based infrastructures and services, their IT resources become increasingly dynamic and hybrid in nature. Managing these require modern IT operations and tools. In his session at 20th Cloud Expo, Raj Sundaram, Senior Principal Product Manager at CA Technologies, will discuss how to modernize your IT operations in order to proactively manage your hybrid cloud and IT environments. He will be sharing be...
As DevOps methodologies expand their reach across the enterprise, organizations face the daunting challenge of adapting related cloud strategies to ensure optimal alignment, from managing complexity to ensuring proper governance. How can culture, automation, legacy apps and even budget be reexamined to enable this ongoing shift within the modern software factory?
Building a cross-cloud operational model can be a daunting task. Per-cloud silos are not the answer, but neither is a fully generic abstraction plane that strips out capabilities unique to a particular provider. In his session at 20th Cloud Expo, Chris Wolf, VP & Chief Technology Officer, Global Field & Industry at VMware, will discuss how successful organizations approach cloud operations and management, with insights into where operations should be centralized and when it’s best to decentraliz...
SYS-CON Events announced today that T-Mobile will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. As America's Un-carrier, T-Mobile US, Inc., is redefining the way consumers and businesses buy wireless services through leading product and service innovation. The Company's advanced nationwide 4G LTE network delivers outstanding wireless experiences to 67.4 million customers who are unwilling to compromise on ...
@GonzalezCarmen has been ranked the Number One Influencer and @ThingsExpo has been named the Number One Brand in the “M2M 2016: Top 100 Influencers and Brands” by Analytic. Onalytica analyzed tweets over the last 6 months mentioning the keywords M2M OR “Machine to Machine.” They then identified the top 100 most influential brands and individuals leading the discussion on Twitter.
In recent years, containers have taken the world by storm. Companies of all sizes and industries have realized the massive benefits of containers, such as unprecedented mobility, higher hardware utilization, and increased flexibility and agility; however, many containers today are non-persistent. Containers without persistence miss out on many benefits, and in many cases simply pass the responsibility of persistence onto other infrastructure, adding additional complexity.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
SYS-CON Events announced today that Juniper Networks (NYSE: JNPR), an industry leader in automated, scalable and secure networks, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Juniper Networks challenges the status quo with products, solutions and services that transform the economics of networking. The company co-innovates with customers and partners to deliver automated, scalable and secure network...
20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, New York, and 21st International Cloud Expo, which will take place in November in Silicon Valley, California.
Five years ago development was seen as a dead-end career, now it’s anything but – with an explosion in mobile and IoT initiatives increasing the demand for skilled engineers. But apart from having a ready supply of great coders, what constitutes true ‘DevOps Royalty’? It’ll be the ability to craft resilient architectures, supportability, security everywhere across the software lifecycle. In his keynote at @DevOpsSummit at 20th Cloud Expo, Jeffrey Scheaffer, GM and SVP, Continuous Delivery Busine...
Back in February of 2017, Andrew Clay Schafer of Pivotal tweeted the following: “seriously tho, the whole software industry is stuck on deployment when we desperately need architecture and telemetry.” Intrigue in a 140 characters. For me, I hear Andrew saying, “we’re jumping to step 5 before we’ve successfully completed steps 1-4.”
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists will examine how DevOps helps to meet th...
SYS-CON Events announced today that Juniper Networks (NYSE: JNPR), an industry leader in automated, scalable and secure networks, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Juniper Networks challenges the status quo with products, solutions and services that transform the economics of networking. The company co-innovates with customers and partners to deliver automated, scalable and secure network...
@DevOpsSummit has been named the ‘Top DevOps Influencer' by iTrend. iTred processes millions of conversations, tweets, interactions, news articles, press releases, blog posts - and extract meaning form them and analyzes mobile and desktop software platforms used to communicate, various metadata (such as geo location), and automation tools. In overall placement, @DevOpsSummit ranked as the number one ‘DevOps Influencer' followed by @CloudExpo at third, and @MicroservicesE at 24th.
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...
The explosion of new web/cloud/IoT-based applications and the data they generate are transforming our world right before our eyes. In this rush to adopt these new technologies, organizations are often ignoring fundamental questions concerning who owns the data and failing to ask for permission to conduct invasive surveillance of their customers. Organizations that are not transparent about how their systems gather data telemetry without offering shared data ownership risk product rejection, regu...
SYS-CON Events announced today that Hitachi, the leading provider the Internet of Things and Digital Transformation, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Hitachi Data Systems, a wholly owned subsidiary of Hitachi, Ltd., offers an integrated portfolio of services and solutions that enable digital transformation through enhanced data management, governance, mobility and analytics. We help globa...
Blockchain is a shared, secure record of exchange that establishes trust, accountability and transparency across supply chain networks. Supported by the Linux Foundation's open source, open-standards based Hyperledger Project, Blockchain has the potential to improve regulatory compliance, reduce cost and time for product recall as well as advance trade. Are you curious about Blockchain and how it can provide you with new opportunities for innovation and growth? In her session at 20th Cloud Exp...
SYS-CON Events announced today that Hitachi Data Systems, a wholly owned subsidiary of Hitachi LTD., will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City. Hitachi Data Systems (HDS) will be featuring the Hitachi Content Platform (HCP) portfolio. This is the industry’s only offering that allows organizations to bring together object storage, file sync and share, cloud storage gateways, and sophisticated search an...