Welcome!

Machine Learning Authors: William Schmarzo, Kevin Jackson, Stackify Blog, Elizabeth White, Pat Romanski

Related Topics: Microservices Expo, Java IoT, Microsoft Cloud, Cognitive Computing , Machine Learning , Agile Computing

Microservices Expo: Article

Let’s Talk About Your Performance

Do you know how long your customers are going to wait for a page load?

Do you know how long your customers are going to wait for a page load? More importantly, how long are you making them wait right now?

Back in the misty eons of time, it used to be easy to measure the performance of your application. You’d grab a stopwatch, load up your web application, and see what happened. If it was slow, you’d look at the mess of PHP, HTML and CSS you crammed into index.php and make sure that you weren’t using bubble sort anywhere. In these modern times, you typically take a few more extra steps:

  • Add Varnish to cache any generated content
  • Split your MySQL InnoDB tables into separate files
  • Add another Memcached server
  • Tune the buffer size on your Cassandra/Mongo/Couch reads
  • Create and/or delete a few indexes in your SQL DB

And of course you still want to look at the performance of the Ruby/Python/PHP code that you wrote, with some insight into how that performance relates to the web framework you’re currently using.

Measure First
We all accumulate questions about the performance and behavior of the systems that we build. But asking a question is just the first of serveral steps that precede taking action and modifying code. You may be right to say that your memcache hit/miss ratio on a particular page seems low, or that you could add an index to a MySQL table to make a certain query faster. However, it could very well be that the majority of page load times are being spent in MySQL executing queries that aren’t being cached, and aren’t even using table you wanted to add an index to. Until you measure your performance you don’t know.

You need to know what’s slow before you can make any informed decisions about which changes you need to make to your system or your code. You need need to measure your web application’s performance, preferably broken down by software layer, and machine, and available to you in real-time. Once you have this data, you can start digging into it.

Then Look at Your Data
Once you have the data, you can start to look at it for interesting trends and patterns. You can look at it in different ways that suit the sort of question that you’re trying to ask. There are many ways to examine performance data, each with its own strengths and weaknesses. Here are some of those ways:

Timeseries Chart
First, let’s dispense with the notion that you can actually control every request. Modern web applications are a complicated endeavor, consisting of many dependent moving parts that each take a variable amount of time. An easy way to simplify all this is to divide all the data into time buckets of, say, 15 minutes, and take the average (mean) across entire requests. Then, you can see how fast your overall web application is:

This plot is called a timeseries. There’s a couple things worth noting about this picture. First of all, it’s not a straight line. Even if you have tons of data and everything is going swimmingly, you’ll see a bit of jumpiness. Secondly, it’s not a very rich visualization. You’ve taken everything you know about all your servers and all your code in the last day and compressed it down to 96 data points. The average word in Moby-Dick is ljklk, but that seems to miss the point.

Stacked-Area Chart
One of the dimensions we’re ignoring in the timeseries chart is the different software components involved in each request. Instead of adding up all the various latencies, let’s separate each layer of an application into its own timeseries. Plotting all those timeseries at once could get messy, so let’s stack them on top of each other and color the area between them, which now represents the latency of each layer.

Since we started from the same data, the top of the line is still our average request latency, but now we can see where each request is spending its time. In this graph, we can actually see that not only is most of the time spent in PHP, but PHP is also the most variable layer.

This cuts out a whole class of potential upgrades and changes. Upgrade the DB? Add an index to a table to speed up some queries? Add RAM to the memcache servers? None of that seems necessary; all the action here is in PHP. Even if we reduced the time spent in every other layer to zero, we would see only a 40% decrease in latency at best.

Heatmaps
The stacked area chart can point us in the right direction, but we’re still losing a lot of information. Timeseries and stacked area charts show us trends. What if we’re looking for patterns in latency in a single time bucket? If we take all those data points and then bucketize them by latency, we get a histogram:

This chart breaks down how often we’ve served up requests at a given latency. If you compare it to the timeseries chart above, it looks fairly different. About 80% of these requests finish in 2 seconds, even though the timeseries seems to hover right around 2 seconds. The culprit here is the long tail — a request that takes 10 or 15 seconds to finish pulls the average up more than a request that finishes in 0.01 seconds pulls it down.

Even more interesting, there are actually two different populations here. There is a cluster at 0.1s, then a larger, broader hump at 0.4s. This information had been lost in the simple timeseries, and even a stacked area chart couldn’t seperate those two. On the other hand, a single histogram could never tell you if you’ve improved. You’d have to compare two histograms, and even then,  you’d have to squint pretty hard to figure out where the shape had changed.

One way to get around this problem is to plot a bunch of histograms over time. We’ll keep time on the X axis, left to right, but at each point, let’s plot a distribution. In this graph, we’ll put latency (the X axis of the histogram) on the Y axis and color each point in proportion with the amount of data we see there.

A picture is worth 1000 words:

This is just the data from memcache, which typically has very short calls. Just like the above histogram, we can see a double-humped distribution at 0.6ms and 1.3ms. On the other hand, we can also see abnormalities such as the traffic spike just after 6:15PM that caused the latency to temporarily increase.

Now what?
We’ve shown you a few different ways of visualizing your performance data. But that’s just the beginning. Any one of these charts can give you tremendous insight into the performance and behavior of your application. These insights can be positive an actionable (i.e., the problem really is in the app layer), or negative but still valuable (i.e., the data seems to indicate that my original hypothesis that we needed an index on this table was wrong). Nevertheless, you can’t always come to these insights by just examining latency. We’ll show you what we mean by that next time.

Related Articles

TraceView: Now With Free Tracing (and more)!

The Taming of the Queue: Measuring the Impact of Request Queueing

Tracing Celery Performance For Web Applications

More Stories By TR Jordan

A veteran of MIT’s Lincoln Labs, TR is a reformed physicist and full-stack hacker – for some limited definition of full stack. After a few years as Software Development Lead with Thermopylae Science and Techology, he left to join Tracelytics as its first engineer. Following Tracelytics merger with AppNeta, TR was tapped to run all of its developer and market evangelism efforts. TR still harbors a not-so-secret love for Matlab-esque graphs and half-baked statistics, as well as elegant and highly-performant code. Read more of his articles at www.appneta.com/blog or visit www.appneta.com.

@CloudExpo Stories
"ZeroStack is a startup in Silicon Valley. We're solving a very interesting problem around bringing public cloud convenience with private cloud control for enterprises and mid-size companies," explained Kamesh Pemmaraju, VP of Product Management at ZeroStack, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"Codigm is based on the cloud and we are here to explore marketing opportunities in America. Our mission is to make an ecosystem of the SW environment that anyone can understand, learn, teach, and develop the SW on the cloud," explained Sung Tae Ryu, CEO of Codigm, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
High-velocity engineering teams are applying not only continuous delivery processes, but also lessons in experimentation from established leaders like Amazon, Netflix, and Facebook. These companies have made experimentation a foundation for their release processes, allowing them to try out major feature releases and redesigns within smaller groups before making them broadly available. In his session at 21st Cloud Expo, Brian Lucas, Senior Staff Engineer at Optimizely, discussed how by using ne...
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
Enterprises are moving to the cloud faster than most of us in security expected. CIOs are going from 0 to 100 in cloud adoption and leaving security teams in the dust. Once cloud is part of an enterprise stack, it’s unclear who has responsibility for the protection of applications, services, and data. When cloud breaches occur, whether active compromise or a publicly accessible database, the blame must fall on both service providers and users. In his session at 21st Cloud Expo, Ben Johnson, C...
"Infoblox does DNS, DHCP and IP address management for not only enterprise networks but cloud networks as well. Customers are looking for a single platform that can extend not only in their private enterprise environment but private cloud, public cloud, tracking all the IP space and everything that is going on in that environment," explained Steve Salo, Principal Systems Engineer at Infoblox, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventio...
Data scientists must access high-performance computing resources across a wide-area network. To achieve cloud-based HPC visualization, researchers must transfer datasets and visualization results efficiently. HPC clusters now compute GPU-accelerated visualization in the cloud cluster. To efficiently display results remotely, a high-performance, low-latency protocol transfers the display from the cluster to a remote desktop. Further, tools to easily mount remote datasets and efficiently transfer...
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the conn...
"We're developing a software that is based on the cloud environment and we are providing those services to corporations and the general public," explained Seungmin Kim, CEO/CTO of SM Systems Inc., in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
The question before companies today is not whether to become intelligent, it’s a question of how and how fast. The key is to adopt and deploy an intelligent application strategy while simultaneously preparing to scale that intelligence. In her session at 21st Cloud Expo, Sangeeta Chakraborty, Chief Customer Officer at Ayasdi, provided a tactical framework to become a truly intelligent enterprise, including how to identify the right applications for AI, how to build a Center of Excellence to oper...
"IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
In his session at 21st Cloud Expo, James Henry, Co-CEO/CTO of Calgary Scientific Inc., introduced you to the challenges, solutions and benefits of training AI systems to solve visual problems with an emphasis on improving AIs with continuous training in the field. He explored applications in several industries and discussed technologies that allow the deployment of advanced visualization solutions to the cloud.
While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, discussed how data centers of the future will be managed, how the p...