|By Andreas Grabner||
|February 10, 2012 08:30 AM EST||
Whether or not you are a fan of U.S. football - it was really hard to avoid this huge sports event on February 5. In addition to the actual game, it's the Super Bowl commercials that - besides being very expensive to air - usually drive a lot of load on the websites of the companies that run their ads. The question is whether the millions of dollars spent really drive consumers to these websites and make them do business with them.
As we won't get an answer from the top brands that advertised about the actual conversion rates we can look at the End User Experience and performance of their websites while these ads were aired. By analyzing the data that we can get through continuous synthetic monitoring combined with deep dive browser diagnostics, we'll be able to see whether their web application was actually able to handle the load and didn't leave too many of these users with a bad user experience.
Use Case: What Was the User Experience at One of the Top Brands on Super Bowl Day
In order to avoid any discussions on whether the numbers we present are good or bad and mean that this company did a good or terrible job, we present all this data without giving an actual name as the purpose of this exercise is to show how to perform this type of analysis.
Synthetic Monitoring of a Web Site
In order to monitor web site performance of a site we set up a Gomez Synthetic Monitor to be executed on a scheduled basis. The monitor not only monitored the performance of the initial home page but also walked through several main use-case scenarios on that page, e.g., searched for a product. These tests were also executed from multiple U.S. locations, using low- and high-bandwidth connections and also using both the Internet Explorer and Firefox browsers. In addition we captured individual performance sessions using dynaTrace Browser Diagnostics.
The following screenshot shows the response time of the home page of this website via our Last Mile platform via a Firefox browser in the days before the Super Bowl as well as during the Super Bowl. The average page load time was around 9 seconds with the exception of the time-frame during the Super Bowl - that's when it jumped up to 30 seconds:
Synthetic Monitoring shows peak during the Super Bowl of up to 30s to load the Homepage
Analyzing Page Load Time from Monitoring Data
There were two factors that drove this spike:
- Higher load on their application due to the commercial aired during the Super Bowl
- Additional content on their page that doubled the page size
Factor 1: Higher Load
That's of course intended and a success for their marketing campaign. What you want to make sure is that you can handle the additional expected load by using CDNs (Content Delivery Networks) to deliver that static content to your end users as well as provide additional resources on your web and application servers to handle the extra load. To be prepared for that it's highly recommended that you do some up-front large scale load-testing. For more information on that you can read my recent blog posts on To Load Test or Not to Load Test: That Is Not the Question.
Factor 2: Additional Content
Additional content downloaded during the Super Bowl resulting in up to 3 times higher page load times
The additional content in this case may have been necessary to fulfill the company's marketing campaign. The question is whether the additional content could have been optimized not to download 40 additional resources with a total size of > 700kb.
Deep Dive Browser Diagnostics
Looking at the dynaTrace Browser Diagnostics Data we can observe several interesting aspects.
Observation 1: Load Impact of additional resources
We can see the actual impact of these additional resources that got downloaded during the time of the game. Two of these resources took a very long time and had a major impact on page load time. The background image was delivered by their own web server and was not put on a CDN, which results in more traffic on their web servers and less than optimal performance for end users who are far away from their data centers. Another interesting aspect was an additional Facebook Like button that took 600ms to download and execute.
dynaTrace Timeline View showing the impact of additional resources on the page load time
Observation 2: Page Transition Impact of CDN User Tracking Logic
When you are responsible for a website that is going to see high user volume for a marketing campaign you want to:
- Make sure the additional content for that marketing campaign is optimized, e.g., make sure this content is on CDNs and follow the Best Practices for Web Performance Optimization
- Make sure you test your application with the campaign-specific features under realistic load
- Analyze the impact of third-party tracking tools and other widgets you put on your page
Connected devices and the Internet of Things are getting significant momentum in 2014. In his session at Internet of @ThingsExpo, Jim Hunter, Chief Scientist & Technology Evangelist at Greenwave Systems, examined three key elements that together will drive mass adoption of the IoT before the end of 2015. The first element is the recent advent of robust open source protocols (like AllJoyn and WebRTC) that facilitate M2M communication. The second is broad availability of flexible, cost-effective ...
Nov. 26, 2014 10:45 PM EST Reads: 966
How do APIs and IoT relate? The answer is not as simple as merely adding an API on top of a dumb device, but rather about understanding the architectural patterns for implementing an IoT fabric. There are typically two or three trends: Exposing the device to a management framework Exposing that management framework to a business centric logic Exposing that business layer and data to end users. This last trend is the IoT stack, which involves a new shift in the separation of what stuff happe...
Nov. 26, 2014 10:30 PM EST Reads: 817
The Internet of Things will put IT to its ultimate test by creating infinite new opportunities to digitize products and services, generate and analyze new data to improve customer satisfaction, and discover new ways to gain a competitive advantage across nearly every industry. In order to help corporate business units to capitalize on the rapidly evolving IoT opportunities, IT must stand up to a new set of challenges. In his session at @ThingsExpo, Jeff Kaplan, Managing Director of THINKstrateg...
Nov. 26, 2014 09:00 PM EST Reads: 934
Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water,...
Nov. 26, 2014 06:00 PM EST Reads: 966
"Matrix is an ambitious open standard and implementation that's set up to break down the fragmentation problems that exist in IP messaging and VoIP communication," explained John Woolf, Technical Evangelist at Matrix, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Nov. 26, 2014 05:45 PM EST Reads: 912
When an enterprise builds a hybrid IaaS cloud connecting its data center to one or more public clouds, security is often a major topic along with the other challenges involved. Security is closely intertwined with the networking choices made for the hybrid cloud. Traditional networking approaches for building a hybrid cloud try to kludge together the enterprise infrastructure with the public cloud. Consequently this approach requires risky, deep "surgery" including changes to firewalls, subnets...
Nov. 26, 2014 04:45 PM EST Reads: 616
DevOps is all about agility. However, you don't want to be on a high-speed bus to nowhere. The right DevOps approach controls velocity with a tight feedback loop that not only consists of operational data but also incorporates business context. With a business context in the decision making, the right business priorities are incorporated, which results in a higher value creation. In his session at DevOps Summit, Todd Rader, Solutions Architect at AppDynamics, discussed key monitoring techniques...
Nov. 26, 2014 04:45 PM EST Reads: 634
Want to enable self-service provisioning of application environments in minutes that mirror production? Can you automatically provide rich data with code-level detail back to the developers when issues occur in production? In his session at DevOps Summit, David Tesar, Microsoft Technical Evangelist on Microsoft Azure and DevOps, will discuss how to accomplish this and more utilizing technologies such as Microsoft Azure, Visual Studio online, and Application Insights in this demo-heavy session.
Nov. 26, 2014 04:45 PM EST Reads: 618
The Internet of Things will greatly expand the opportunities for data collection and new business models driven off of that data. In her session at @ThingsExpo, Esmeralda Swartz, CMO of MetraTech, discussed how for this to be effective you not only need to have infrastructure and operational models capable of utilizing this new phenomenon, but increasingly service providers will need to convince a skeptical public to participate. Get ready to show them the money!
Nov. 26, 2014 04:00 PM EST Reads: 1,004
One of the biggest challenges when developing connected devices is identifying user value and delivering it through successful user experiences. In his session at Internet of @ThingsExpo, Mike Kuniavsky, Principal Scientist, Innovation Services at PARC, described an IoT-specific approach to user experience design that combines approaches from interaction design, industrial design and service design to create experiences that go beyond simple connected gadgets to create lasting, multi-device exp...
Nov. 26, 2014 03:45 PM EST Reads: 972
P2P RTC will impact the landscape of communications, shifting from traditional telephony style communications models to OTT (Over-The-Top) cloud assisted & PaaS (Platform as a Service) communication services. The P2P shift will impact many areas of our lives, from mobile communication, human interactive web services, RTC and telephony infrastructure, user federation, security and privacy implications, business costs, and scalability. In his session at @ThingsExpo, Robin Raymond, Chief Architect...
Nov. 26, 2014 02:00 PM EST Reads: 1,469
High-performing enterprise Software Quality Assurance (SQA) teams validate systems that are ready for use - getting most actively involved as components integrate and form complete systems. These teams catch and report on defects, making sure the customer gets the best software possible. SQA teams have leveraged automation and virtualization to execute more thorough testing in less time - bringing Dev and Ops together, ensuring production readiness. Does the emergence of DevOps mean the end of E...
Nov. 25, 2014 11:30 PM EST Reads: 1,137
Scott Jenson leads a project called The Physical Web within the Chrome team at Google. Project members are working to take the scalability and openness of the web and use it to talk to the exponentially exploding range of smart devices. Nearly every company today working on the IoT comes up with the same basic solution: use my server and you'll be fine. But if we really believe there will be trillions of these devices, that just can't scale. We need a system that is open a scalable and by using ...
Nov. 25, 2014 09:30 PM EST Reads: 1,223
The Internet of Things is tied together with a thin strand that is known as time. Coincidentally, at the core of nearly all data analytics is a timestamp. When working with time series data there are a few core principles that everyone should consider, especially across datasets where time is the common boundary. In his session at Internet of @ThingsExpo, Jim Scott, Director of Enterprise Strategy & Architecture at MapR Technologies, discussed single-value, geo-spatial, and log time series dat...
Nov. 25, 2014 09:30 PM EST Reads: 1,274
"Verizon offers public cloud, virtual private cloud as well as private cloud on-premises - many different alternatives. Verizon's deep knowledge in applications and the fact that we are responsible for applications that make call outs to other systems. Those systems and those resources may not be in Verizon Cloud, we understand at the end of the day it's going to be federated," explained Anne Plese, Senior Consultant, Cloud Product Marketing at Verizon Enterprise, in this SYS-CON.tv interview at...
Nov. 25, 2014 09:00 PM EST Reads: 1,336
"For the past 4 years we have been working mainly to export. For the last 3 or 4 years the main market was Russia. In the past year we have been working to expand our footprint in Europe and the United States," explained Andris Gailitis, CEO of DEAC, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Nov. 25, 2014 08:15 PM EST Reads: 1,119
The Domain Name Service (DNS) is one of the most important components in networking infrastructure, enabling users and services to access applications by translating URLs (names) into IP addresses (numbers). Because every icon and URL and all embedded content on a website requires a DNS lookup loading complex sites necessitates hundreds of DNS queries. In addition, as more internet-enabled ‘Things' get connected, people will rely on DNS to name and find their fridges, toasters and toilets. Acco...
Nov. 25, 2014 07:00 PM EST Reads: 1,313
The term culture has had a polarizing effect among DevOps supporters. Some propose that culture change is critical for success with DevOps, but are remiss to define culture. Some talk about a DevOps culture but then reference activities that could lead to culture change and there are those that talk about culture change as a set of behaviors that need to be adopted by those in IT. There is no question that businesses successful in adopting a DevOps mindset have seen departmental culture change, ...
Nov. 25, 2014 07:00 PM EST Reads: 1,007
"Cloud consumption is something we envision at Solgenia. That is trying to let the cloud spread to the user as a consumption, as utility computing. We want to allow the people to just pay for what they use, not a subscription model," explained Ermanno Bonifazi, CEO & Founder of Solgenia, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Nov. 25, 2014 06:15 PM EST Reads: 981
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With "smart" appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user's habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps,...
Nov. 25, 2014 04:30 PM EST Reads: 1,318