|By Steve Weisfeldt||
|January 21, 2013 08:00 AM EST||
Developing with Performance Testing in Mind
A tester friend of mine recently came to me with a complaint that I think is fairly common in the testing community. He said, "Every time there is a new release of the software for us to test, we have to rework our testing scripts." I've heard this complaint throughout my career, not only in performance testing but in functional testing with automation tools as well.
This state of affairs arises from three fairly straightforward observations:
- Change is inevitable. Everything changes, and in no industry is this more apparent than software development. It makes no sense for testers to ask developers to stop changing the code, but it does make sense to encourage wise changes.
- Developers and testers don't always communicate well. The proverbial wall between developers and testers is still quite formidable. When developers throw a new version over the wall to be tested, too often they've given little thought to how it will be tested.
- All testing tools are not created equal. Some make it easier to identify and handle changes than others. If your testing tool is designed to handle change well, then your entire team is better positioned to embrace change rather than fear it.
Thinking Like a Tester
Most development organizations make a real effort to improve communication between developers and testers, but it's not always enough. Beyond encouraging developers to talk with testers, I ask them to take it a step further and think like testers.
I find that it's a good idea for developers to sit through some of the training that the test engineers complete. In my experience, the developers who do are more careful and avoid making arbitrary changes with little or no justification. They don't, for example, change the name of a field in a form simply because they don't agree with the name the initial developer gave it. When developers are aware of the kinds of changes that make a tester's job harder and what kinds of changes make it easier, then from an organizational standpoint the entire process is more productive.
An Analogy from the Early Days of Functional Testing
Some of the earliest automated functional testing tools for GUIs would simply record the location of the mouse pointer on the screen during tests, and then play back those mouse clicks to execute the test script. If a developer moved the location of a button on the screen, the script would break. Other tools would record the label on the button, so the button could be moved around the UI without breaking the script but changing the button text from "Submit" to "OK" would break the script. More advanced tools used the button's ID to identify it in the script so that the developer could change both the position and the label of the button without making the tester's job more difficult.
One key lesson here is that the choice of testing tool makes a big difference in the productivity of the testing team when the software under test changes, even in relatively trivial ways.
The other key lesson is that developer awareness of testing tools and procedures goes a long way in facilitating a smooth testing operation. I saw this firsthand during a training session I gave years back. While describing how button label changes affected testing, a developer who happened to be sitting in on the training sat upright when he finally understood why his colleagues in testing were so frustrated by many of his changes. He never knew why they objected so much to his changing a button label from "Clear" to "Reset". Going forward, that knowledge didn't stop the developer from making necessary changes. It did, however, make him pause when he made such changes to consider whether they were really necessary.
Performance Testing Tools That Make It Easy to Handle Change
In performance testing, we are not concerned with the location of buttons, but we're not immune to seemingly trivial changes.
For example, when a web form is submitted to the server, the form fields will be a series of name-value pairs. Changing the name of a form field, adding a field, or deleting a field can cause problems during performance testing. With a less capable testing tool, these problems can be hard to identify and diagnose, especially if there is poor communication between developers and testers.
File difference viewers (diff viewers) that enable the tester to compare multiple recordings against each other are particularly helpful in pinpointing the changed fields. When it's time to modify the script, an effective tool will enable you to add, delete, and update fields without programming. Just right-click and choose add, or simply drag-and-drop to update your load testing script.
Form fields are relatively easy to handle for load testers. Parameters that are session specific are more difficult. (These parameters change from session to session but stay the same for the duration of each user session). By default, the hard-coded session values are captured by a load testing tool in each script, and a test engineer needs to parameterize them to make the script usable for load testing. Double-clicking on a hard-coded value to make it a variable is easier than diving into the script code. Here again, tools that help automate the process can reduce test creation time from many hours to a few minutes.
When a new script is needed or maintenance is required on an existing script, tools that are easier to use can make the task orders of magnitude faster.
Overcoming the Fear of Change
I know of development teams that gradually became more and more afraid to change their software because of the difficulties that the changes introduced in testing and elsewhere in the process. Needless to say, this had a negative effect on their ability to deliver new features and fixes. A root of the problem, it turned out, was the testing tool that they were using, which made changes arduous and error-prone. Once they switched to a modern tool, the required script changes were easier to make. Performance testing times shrank from a week to less than a day and development was once again free to make long-needed changes. Agile development shops in particular depend on this ability to rapidly implement changes in testing scripts, and get the tests going in minutes or hours instead of days and weeks.
If your organization is starting to fear change, encourage your developers to think like testers and encourage your testers to use tools that make inevitable change easier to handle.
- The Odd Couple: Marrying Agile and Waterfall
- The Butterfly Effect Within IT
- The Agile PMO
- The Linux Foundation's Core Infrastructure Initiative Announces New Backers, First Projects to Receive Support and Advisory Board Members
- How to Strategically Benefit from AANPM
- Zuora Caps Record Breaking Subscribed 2014 with the 63rd Release of the Award Winning Z-Business Platform
- Web Service Monitoring 101: Identifying Bad Deployments
- How to Monitor Swift/iOS8 Applications for Crashes and Performance Issues
- How to Monitor the Web Performance of Your Competition
- Complete Surface Pro 3 Review - 3 days later
- "Cloud Computing 2.0" -- I'm Serious
- Storage Made Easy Brings Private Enterprise File Share and Sync for Amazon S3 to AWS GovCloud
- The Odd Couple: Marrying Agile and Waterfall
- Flexera Software’s InstallShield 2014 Release Introduces New Support of Cloud and Virtualised Installations, High-DPI Displays and Touch Devices, and Agile Development
- FlexNet Manager Suite Wins CODiE Award for Best Asset Management Solution - 4th CODiE Award for Flexera Software
- Adobe Releases Update to Patch Zero Day Vulnerability
- The Butterfly Effect Within IT
- As Registration Deadline Nears, dmexco 2014 Is More Popular Than Ever
- Software Quality Metrics for Your Continuous Delivery Pipeline | Part 2
- Atmel Extends Industry Leading maXTouch T Series with the Market's Highest Performance Touchscreen Controller
- The Problem with Cloud SLAs
- New Photo Sharing Service izitru Establishes Photoshop-Free Zone
- Emulex and Compuware Team Up to Drastically Improve Data Centre Problem Identification and Resolution
- The Agile PMO
- Building a Drag-and-Drop Shopping Cart with AJAX
- What Is AJAX?
- Google Maps! AJAX-Style Web Development Using ASP.NET
- Where Are RIA Technologies Headed in 2008?
- Dolphin Announces Open API With Over 50 Add-ons Including Dropbox and Wikipedia
- How and Why AJAX, Not Java, Became the Favored Technology for Rich Internet Applications
- Flashback to January 2006: Exclusive SYS-CON.TV Interviews on "OpenAjax Alliance" Announcement
- "Real-World AJAX" One-Day Seminar Arrives in Silicon Valley
- AJAXWorld Conference & Expo to Take Place October 2-4, 2006, at the Santa Clara Convention Center, California
- AJAX Sponsor Webcasts Are Now Available at AJAXWorld Website
- AJAXWorld University Announces AJAX Developer Bootcamp
- i-Technology 2008 Predictions: Where's RIAs, AJAX, SOA and Virtualization Headed in 2008?
Cloud Computing is evolving into a Big Three of Amazon Web Services, Google Cloud, and Microsoft Azure. Cloud 360: Multi-Cloud Bootcamp, being held Nov 4–5, 2014, in conjunction with 15th Cloud Expo in Santa Clara, CA, delivers a real-world demonstration of how to deploy and configure a scalable and available web application on all three platforms. The Cloud 360 Bootcamp, led by Janakiram MSV, an analyst with Gigaom Research, is the first bootcamp that introduces the core concepts of Infrastructure as a Service (IaaS) based on the workings of the Big Three platforms – Amazon EC2, Google Compute Engine, and Azure VMs. Bootcamp attendees will get to see the big picture and also receive the knowledge needed to make the best cloud decisions for their business applications and entire enterprise IT organization.
Jul. 22, 2014 11:00 AM EDT Reads: 906
Scott Jenson leads a project called The Physical Web within the Chrome team at Google. Project members are working to take the scalability and openness of the web and use it to talk to the exponentially exploding range of smart devices. Nearly every company today working on the IoT comes up with the same basic solution: use my server and you'll be fine. But if we really believe there will be trillions of these devices, that just can't scale. We need a system that is open a scalable and by using the URL as a basic building block, we open this up and get the same resilience that the web enjoys.
Jul. 22, 2014 09:00 AM EDT Reads: 1,061
The Internet of Things is a natural complement to the cloud and related technologies such as Big Data, analytics, and mobility. In his session at Internet of @ThingsExpo, Joe Weinman will lay out four generic strategies – digital disciplines – to exploit emerging digital technologies for strategic advantage. Joe Weinman has held executive leadership positions at Bell Labs, AT&T, Hewlett-Packard, and Telx, in areas such as corporate strategy, business development, product management, operations, and R&D.
Jul. 21, 2014 11:17 AM EDT Reads: 1,689
SYS-CON Events announced today that DevOps.com has been named “Media Sponsor” of SYS-CON's “DevOps Summit at Cloud Expo,” which will take place on June 10–12, 2014, at the Javits Center in New York City, New York. DevOps.com is where the world meets DevOps. It is the largest collection of original content relating to DevOps on the web today Featuring up-to-the-minute news, feature stories, blogs, bylined articles and more, DevOps.com is where the thought leaders of the DevOps movement make their ideas known.
Jul. 20, 2014 03:00 PM EDT Reads: 1,505
There are 182 billion emails sent every day, generating a lot of data about how recipients and ISPs respond. Many marketers take a more-is-better approach to stats, preferring to have the ability to slice and dice their email lists based numerous arbitrary stats. However, fundamentally what really matters is whether or not sending an email to a particular recipient will generate value. Data Scientists can design high-level insights such as engagement prediction models and content clusters that allow marketers to cut through the noise and design their campaigns around strong, predictive signals, rather than arbitrary statistics. SendGrid sends up to half a billion emails a day for customers such as Pinterest and GitHub. All this email adds up to more text than produced in the entire twitterverse. We track events like clicks, opens and deliveries to help improve deliverability for our customers – adding up to over 50 billion useful events every month. While SendGrid data covers only abo...
Jul. 20, 2014 02:00 PM EDT Reads: 2,145
SYS-CON Events announced today that the Web Host Industry Review has been named “Media Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Since 2000, The Web Host Industry Review has made a name for itself as the foremost authority of the Web hosting industry providing reliable, insightful and comprehensive news, reviews and resources to the hosting community. TheWHIR Blogs provides a community of expert industry perspectives. The Web Host Industry Review Magazine also offers a business-minded, issue-driven perspective of interest to executives and decision-makers. WHIR TV offers on demand web hosting video interviews and web hosting video features of the key persons and events of the web hosting industry. WHIR Events brings together like-minded hosting industry professionals and decision-makers in local communities. TheWHIR is an iNET Interactive property.
Jul. 20, 2014 09:15 AM EDT Reads: 1,600
SYS-CON Events announced today that O'Reilly Media has been named “Media Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. O'Reilly Media spreads the knowledge of innovators through its books, online services, magazines, and conferences. Since 1978, O'Reilly Media has been a chronicler and catalyst of cutting-edge development, homing in on the technology trends that really matter and spurring their adoption by amplifying "faint signals" from the alpha geeks who are creating the future. An active participant in the technology community, the company has a long history of advocacy, meme-making, and evangelism.
Jul. 19, 2014 10:00 AM EDT Reads: 1,694
SYS-CON Events announced today that Verizon has been named “Gold Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Verizon Enterprise Solutions creates global connections that generate growth, drive business innovation and move society forward. With industry-specific solutions and a full range of global wholesale offerings provided over the company's secure mobility, cloud, strategic networking and advanced communications platforms, Verizon Enterprise Solutions helps open new opportunities around the world for innovation, investment and business transformation. Visit verizonenterprise.com to learn more.
Jul. 18, 2014 11:00 AM EDT Reads: 1,683
SYS-CON Events announced today that TMCnet has been named “Media Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Technology Marketing Corporation (TMC) is the world's leading business to business and integrated marketing media company, servicing niche markets within the communications and technology industries.
Jul. 15, 2014 04:21 PM EDT Reads: 1,154
"In my session I spoke about enterprise cloud analytics and how we can leverage analytics as a service," explained Ajay Budhraja, CTO at the Department of Justice, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
Jul. 15, 2014 10:15 AM EDT Reads: 1,760
“We are starting to see people move beyond the commodity cloud and enterprises need to start focusing on additional value added services in order to really drive their adoption," explained Jason Mondanaro, Director of Product Management at MetraTech, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
Jul. 15, 2014 09:45 AM EDT Reads: 1,733
"We are automated capacity control software, which basically looks at all the supply and demand and running a virtual cloud environment and does a deep analysis of that and says where should things go," explained Andrew Hillier, Co-founder & CTO of CiRBA, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
Jul. 15, 2014 09:45 AM EDT Reads: 1,932
Almost everyone sees the potential of Internet of Things but how can businesses truly unlock that potential. The key will be in the ability to discover business insight in the midst of an ocean of Big Data generated from billions of embedded devices via Systems of Discover. Businesses will also need to ensure that they can sustain that insight by leveraging the cloud for global reach, scale and elasticity. In his session at Internet of @ThingsExpo, Mac Devine, Distinguished Engineer at IBM, will discuss bringing these three elements together via Systems of Discover.
Jul. 15, 2014 08:00 AM EDT Reads: 2,065
The Internet of Things promises to transform businesses (and lives), but navigating the business and technical path to success can be difficult to understand. In his session at 15th Internet of @ThingsExpo, Chad Jones, Vice President, Product Strategy of LogMeIn's Xively IoT Platform, will show you how to approach creating broadly successful connected customer solutions using real world business transformation studies including New England BioLabs and more.
Jul. 14, 2014 09:00 AM EDT Reads: 2,103
All too many discussions about DevOps conclude that the solution is an all-purpose player: developer and operations guru, complete with pager for round-the-clock duty. For most organizations that is not the way forward. In his session at DevOps Summit, Bernard Golden, Vice President of Strategy at ActiveState, will discuss how to achieve the agility and speed of end-to-end automation without requiring an organization stocked with Supermen and Superwomen.
Jul. 14, 2014 08:45 AM EDT Reads: 1,804