“Open source has always provided a number of benefits, including easing adoption costs, propagating a better understanding of the technology, and allowing for faster evolution and commercialization of products and services based on it,” noted Terry Woloszyn, Founder & CEO, Leeward Security Ltd., in this exclusive Q&A with Cloud Expo Conference Chair Jeremy Geelan. “This is clearly evident with the OpenStack and CloudStack,” Woloszyn continued, “and others that have been quickly commercialized as...| By Steve Weisfeldt | Article Rating: |
|
| January 21, 2013 08:00 AM EST | Reads: |
3,243 |
Developing with Performance Testing in Mind
A tester friend of mine recently came to me with a complaint that I think is fairly common in the testing community. He said, "Every time there is a new release of the software for us to test, we have to rework our testing scripts." I've heard this complaint throughout my career, not only in performance testing but in functional testing with automation tools as well.
This state of affairs arises from three fairly straightforward observations:

- Change is inevitable. Everything changes, and in no industry is this more apparent than software development. It makes no sense for testers to ask developers to stop changing the code, but it does make sense to encourage wise changes.
- Developers and testers don't always communicate well. The proverbial wall between developers and testers is still quite formidable. When developers throw a new version over the wall to be tested, too often they've given little thought to how it will be tested.
- All testing tools are not created equal. Some make it easier to identify and handle changes than others. If your testing tool is designed to handle change well, then your entire team is better positioned to embrace change rather than fear it.
Thinking Like a Tester
Most development organizations make a real effort to improve communication between developers and testers, but it's not always enough. Beyond encouraging developers to talk with testers, I ask them to take it a step further and think like testers.
I find that it's a good idea for developers to sit through some of the training that the test engineers complete. In my experience, the developers who do are more careful and avoid making arbitrary changes with little or no justification. They don't, for example, change the name of a field in a form simply because they don't agree with the name the initial developer gave it. When developers are aware of the kinds of changes that make a tester's job harder and what kinds of changes make it easier, then from an organizational standpoint the entire process is more productive.
An Analogy from the Early Days of Functional Testing
Some of the earliest automated functional testing tools for GUIs would simply record the location of the mouse pointer on the screen during tests, and then play back those mouse clicks to execute the test script. If a developer moved the location of a button on the screen, the script would break. Other tools would record the label on the button, so the button could be moved around the UI without breaking the script but changing the button text from "Submit" to "OK" would break the script. More advanced tools used the button's ID to identify it in the script so that the developer could change both the position and the label of the button without making the tester's job more difficult.
One key lesson here is that the choice of testing tool makes a big difference in the productivity of the testing team when the software under test changes, even in relatively trivial ways.
The other key lesson is that developer awareness of testing tools and procedures goes a long way in facilitating a smooth testing operation. I saw this firsthand during a training session I gave years back. While describing how button label changes affected testing, a developer who happened to be sitting in on the training sat upright when he finally understood why his colleagues in testing were so frustrated by many of his changes. He never knew why they objected so much to his changing a button label from "Clear" to "Reset". Going forward, that knowledge didn't stop the developer from making necessary changes. It did, however, make him pause when he made such changes to consider whether they were really necessary.
Performance Testing Tools That Make It Easy to Handle Change
In performance testing, we are not concerned with the location of buttons, but we're not immune to seemingly trivial changes.
For example, when a web form is submitted to the server, the form fields will be a series of name-value pairs. Changing the name of a form field, adding a field, or deleting a field can cause problems during performance testing. With a less capable testing tool, these problems can be hard to identify and diagnose, especially if there is poor communication between developers and testers.
File difference viewers (diff viewers) that enable the tester to compare multiple recordings against each other are particularly helpful in pinpointing the changed fields. When it's time to modify the script, an effective tool will enable you to add, delete, and update fields without programming. Just right-click and choose add, or simply drag-and-drop to update your load testing script.
Form fields are relatively easy to handle for load testers. Parameters that are session specific are more difficult. (These parameters change from session to session but stay the same for the duration of each user session). By default, the hard-coded session values are captured by a load testing tool in each script, and a test engineer needs to parameterize them to make the script usable for load testing. Double-clicking on a hard-coded value to make it a variable is easier than diving into the script code. Here again, tools that help automate the process can reduce test creation time from many hours to a few minutes.
When a new script is needed or maintenance is required on an existing script, tools that are easier to use can make the task orders of magnitude faster.
Overcoming the Fear of Change
I know of development teams that gradually became more and more afraid to change their software because of the difficulties that the changes introduced in testing and elsewhere in the process. Needless to say, this had a negative effect on their ability to deliver new features and fixes. A root of the problem, it turned out, was the testing tool that they were using, which made changes arduous and error-prone. Once they switched to a modern tool, the required script changes were easier to make. Performance testing times shrank from a week to less than a day and development was once again free to make long-needed changes. Agile development shops in particular depend on this ability to rapidly implement changes in testing scripts, and get the tests going in minutes or hours instead of days and weeks.
If your organization is starting to fear change, encourage your developers to think like testers and encourage your testers to use tools that make inevitable change easier to handle.
Published January 21, 2013 Reads 3,243
Copyright © 2013 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Steve Weisfeldt
Steve Weisfeldt is a Senior Performance Engineer at Neotys, a provider of load testing software for Web applications. Previously, he has worked as the President of Engine 1 Consulting, a services firm specializing in all facets of test automation. Prior to his involvement at Engine 1 Consulting, he was a Senior Systems Engineer at Aternity. Prior to that, Steve spent seven years at automated testing vendor Segue Software (acquired by Borland). While spending most of his time at Segue delivering professional services and training, he was also involved in pre-sales and product marketing efforts.
Being in the load and performance testing space since 1999, Steve has been involved in load and performance testing projects of all sizes, in industries that span the retail, financial services, insurance and manufacturing sectors. His expertise lies in enabling organizations to optimize their ability to develop, test and launch high-quality applications efficiently, on-time and on-budget. Steve graduated from the University of Massachusetts-Lowell with a BS in Electrical Engineering and an MS in Computer Engineering.
- Cloud People: A Who's Who of Cloud Computing
- AMD and Adobe Collaborate on Upcoming Version of Adobe Premiere Pro Software to Enable Breakthrough Video Editing Performance Through Open Standards
- New Relic Q1 2013 Blazes Past Growth Targets and Reaches 40,000 Active Customer Accounts
- Predixion Software Announces General Availability of the Latest Version of its Predictive Analytics Platform
- Social Loginwall Failure
- Five Big Data Features in SQL Server
- GoBank Announces Timing of General Availability and National Distribution Relationships at FinovateSpring
- MicroStrategy Announces General Availability of MicroStrategy 9.3.1
- Cloud Expo NY: Cloud & Location-Aware Big Data Is Changing Our World
- How Bon-Ton Stores Align Business Goals with IT Requirements
- WordsEye Announces Upcoming Beta of a First-of-Its-Kind Text-to-Scene Application
- MicroStrategy Announces General Availability of MicroStrategy 9.3.1
- Cloud People: A Who's Who of Cloud Computing
- AMD and Adobe Collaborate on Upcoming Version of Adobe Premiere Pro Software to Enable Breakthrough Video Editing Performance Through Open Standards
- New Relic Q1 2013 Blazes Past Growth Targets and Reaches 40,000 Active Customer Accounts
- Predixion Software Announces General Availability of the Latest Version of its Predictive Analytics Platform
- Red Hat Reinforces Java Commitment
- Social Loginwall Failure
- VCE Revisited, Now and Zen
- Five Big Data Features in SQL Server
- Five Steps Toward Achieving Better Compliance with Identity Analytics
- Big Data Is Not Just About Marketing: Don’t Forget the IT Department’s Needs
- GoBank Announces Timing of General Availability and National Distribution Relationships at FinovateSpring
- MicroStrategy Announces General Availability of MicroStrategy 9.3.1
- Building a Drag-and-Drop Shopping Cart with AJAX
- What Is AJAX?
- Google Maps! AJAX-Style Web Development Using ASP.NET
- Flashback to January 2006: Exclusive SYS-CON.TV Interviews on "OpenAjax Alliance" Announcement
- How and Why AJAX, Not Java, Became the Favored Technology for Rich Internet Applications
- Where Are RIA Technologies Headed in 2008?
- AJAXWorld Conference & Expo to Take Place October 2-4, 2006, at the Santa Clara Convention Center, California
- "Real-World AJAX" One-Day Seminar Arrives in Silicon Valley
- AJAX Sponsor Webcasts Are Now Available at AJAXWorld Website
- AJAXWorld University Announces AJAX Developer Bootcamp
- AJAX Support In JadeLiquid WebRenderer v3.1
- Struts Validations Framework Using AJAX
“Open source has always provided a number of benefits, including easing adoption costs, propagating a better understanding of the technology, and allowing for faster evolution and commercialization of products and services based on it,” noted Terry Woloszyn, Founder & CEO, Leeward Security Ltd., in this exclusive Q&A with Cloud Expo Conference Chair Jeremy Geelan. “This is clearly evident with the OpenStack and CloudStack,” Woloszyn continued, “and others that have been quickly commercialized as...May. 21, 2013 06:30 AM EDT Reads: 803 |
By Elizabeth White Cloud enables SMBs to access new, scalable resources – previously only available to enterprises – in flexible and cost-effective ways. McKinsey’s SMB Cloud Report projects the public cloud market to reach $40-$50 billion by 2015, with SMBs comprising 65% of public cloud spending in 2015. But selling cloud to SMBs raises the questions of who, what and how.
In her session at the 12th International Cloud Expo, Manjula Talreja, VP of Cisco’s Global Cloud Business Development Team, will discuss the...May. 21, 2013 06:15 AM EDT Reads: 1,827 |
By Pat Romanski In the face of rapidly increasing amounts of unstructured data, industry is investing heavily to turn machines into services and connect them to analytics engines that will extract an extraordinary amount of value and unleash a productivity revolution for both businesses and consumers.
In the health care, transportation and energy sectors alone, the combination of machine diagnostics software and analytics will eliminate as much as $150 billion in waste.
In his session at the 12th Internation...May. 21, 2013 05:00 AM EDT Reads: 2,961 |
By Jeremy Geelan The economics of business are radically changing due to the way in which software and services are being delivered thanks to cloud computing. In his session at 12th Cloud Expo | Cloud Expo New York [10-13 June, 2013], Mike Kavis will cover six reasons for the disruption.May. 21, 2013 04:00 AM EDT Reads: 4,609 |
By Jeremy Geelan New, "Super-Sized" 4-Day Cloud Computing Bootcamp is a brief introduction to cloud computing carefully created and devised to help you keep up with evolving trends like Big Data, PaaS, APIs, Mobile, Social and Data Analytics. Solutions built around these topics require a sound cloud computing infrastructure to be successful while assisting customers harvest real benefits from this transformational change that is happening in the IT ecosystem.May. 20, 2013 10:30 AM EDT Reads: 936 |
By Elizabeth White As enterprises deploy private IaaS clouds into production they are reevaluating their future application delivery models. SUSE and WSO2 believe that private PaaS will leverage the automation and scalability of Private IaaS solutions, such as OpenStack-based SUSE Cloud, to deliver the secure, standardized development environments that will make migrating to an agile, serviceoriented delivery model possible.
In their session at the 12th International Cloud Expo, Chris Haddad, VP of Technology Ev...May. 20, 2013 10:07 AM EDT Reads: 683 |
By Liz McMillan “Trust is an ongoing journey and sits at the foundation of any vendor relationship – the companies that don’t consistently earn trust won’t be around long,” noted Henrik Rosendahl, Senior VP of Cloud Solutions at Quantum, in this exclusive Q&A with Cloud Expo Conference Chair Jeremy Geelan. “As they do more with cloud, trust will organically grow – maybe it’s just about meeting SLAs or seeing firsthand that data is there when you need it,” Rosendahl continued.
Cloud Computing Journal: The move ...May. 20, 2013 09:00 AM EDT Reads: 1,901 |
By Jeremy Geelan May. 20, 2013 08:15 AM EDT Reads: 3,012 |
By Jeremy Geelan
Cloud enables SMBs to access new, scalable resources – previously only available to enterprises – in flexible and cost-effective ways. McKinsey’s SMB Cloud Report projects the public cloud market to reach $40-$50 billion by 2015, with SMBs comprising 65% of public cloud spending in 2015. But selling cloud to SMBs raises the questions of who, what and how.
In this session Manjula Talreja, VP of Cisco’s Global Cloud Business Development Team, will discuss the importance of knowing who SMB...May. 20, 2013 08:00 AM EDT Reads: 1,275 |
By Jeremy Geelan Analyzing Hadoop jobs and speeding them up is often a tedious and time consuming effort that requires experts. In his upcoming session at 12th Cloud Expo | Cloud Expo New York [10-13 June, 2013], Michael Kopp will be showing how proven APM techniques can be used to speed up Hadoop jobs at the core, without going through tons of log files, beyond just adding more hardware and within minutes instead of hours or days. May. 20, 2013 08:00 AM EDT Reads: 1,774 |








Cloud enables SMBs to access new, scalable resources – previously only available to enterprises – in flexible and cost-effective ways. McKinsey’s SMB Cloud Report projects the public cloud market to reach $40-$50 billion by 2015, with SMBs comprising 65% of public cloud spending in 2015. But selling cloud to SMBs raises the questions of who, what and how.
In her session at the 12th International Cloud Expo, Manjula Talreja, VP of Cisco’s Global Cloud Business Development Team, will discuss the...
In the face of rapidly increasing amounts of unstructured data, industry is investing heavily to turn machines into services and connect them to analytics engines that will extract an extraordinary amount of value and unleash a productivity revolution for both businesses and consumers.
In the health care, transportation and energy sectors alone, the combination of machine diagnostics software and analytics will eliminate as much as $150 billion in waste.
In his session at the 12th Internation...
The economics of business are radically changing due to the way in which software and services are being delivered thanks to cloud computing. In his session at 12th Cloud Expo | Cloud Expo New York [10-13 June, 2013], Mike Kavis will cover six reasons for the disruption.
New, "Super-Sized" 4-Day Cloud Computing Bootcamp is a brief introduction to cloud computing carefully created and devised to help you keep up with evolving trends like Big Data, PaaS, APIs, Mobile, Social and Data Analytics. Solutions built around these topics require a sound cloud computing infrastructure to be successful while assisting customers harvest real benefits from this transformational change that is happening in the IT ecosystem.
As enterprises deploy private IaaS clouds into production they are reevaluating their future application delivery models. SUSE and WSO2 believe that private PaaS will leverage the automation and scalability of Private IaaS solutions, such as OpenStack-based SUSE Cloud, to deliver the secure, standardized development environments that will make migrating to an agile, serviceoriented delivery model possible.
In their session at the 12th International Cloud Expo, Chris Haddad, VP of Technology Ev...
“Trust is an ongoing journey and sits at the foundation of any vendor relationship – the companies that don’t consistently earn trust won’t be around long,” noted Henrik Rosendahl, Senior VP of Cloud Solutions at Quantum, in this exclusive Q&A with Cloud Expo Conference Chair Jeremy Geelan. “As they do more with cloud, trust will organically grow – maybe it’s just about meeting SLAs or seeing firsthand that data is there when you need it,” Rosendahl continued.
Cloud Computing Journal: The move ...
Cloud enables SMBs to access new, scalable resources – previously only available to enterprises – in flexible and cost-effective ways. McKinsey’s SMB Cloud Report projects the public cloud market to reach $40-$50 billion by 2015, with SMBs comprising 65% of public cloud spending in 2015. But selling cloud to SMBs raises the questions of who, what and how.
In this session Manjula Talreja, VP of Cisco’s Global Cloud Business Development Team, will discuss the importance of knowing who SMB...
Analyzing Hadoop jobs and speeding them up is often a tedious and time consuming effort that requires experts. In his upcoming session at 12th Cloud Expo | Cloud Expo New York [10-13 June, 2013], Michael Kopp will be showing how proven APM techniques can be used to speed up Hadoop jobs at the core, without going through tons of log files, beyond just adding more hardware and within minutes instead of hours or days.
New technologies allow schools, colleges and universities to analyze absolutely everything that happens. From student behavior, testing results, career development of students as well as educational needs based on changing societies. A lot of this data has already been stored and is used for statist...
A recent Gartner study states that the function of the modern CIO is in flux and that his or her future focus must incorporate digital assets (aka cloud-based data and applications) to remain relevant. Towards the goal of riding the sea change a compiler of stacks to a broker of business needs, secu...
In the coming years, big data will change the way organisations and societies are operated and managed. Big data however, is not the only trend that will impact significantly how organisations operate. Another major trend at the moment is gamification. Gamification will change the way organisations ...
We all talk about cloud differently, but is there a way we should be speaking about this tech?
Cloud computing is now a widely reported, if not accepted, IT movement that, depending on who you talk to, has changed or is changing the way businesses utilize infrastructure.
The age of data center automation is upon us. Whether it's cloud or SDN or devops in general, automation as a means to achieve efficiency and, one hopes, free up resources that can be then redirected to focus on innovation.
As is always the case when we begin to move further upwards, abstracting ...
Windows Azure Virtual Networks offers the power to open up several cross-premises use case scenarios, including Active Directory Disaster Recovery, SQL Database Replication, Windows Server 2012 DFS-R File Replication, Accelerated Cloud File Services with BranchCache, Hybrid Web Applications and MORE...
As the infrastructure cloud market (IaaS and PaaS) continues to grow rapidly, we are seeing quite a few customers who are delivering an application – whether it is a mission-critical or SaaS application – and basing their solution on VMware.
VMware Security Cloud Encryption cloud keyboard Cloud Enc...
Have you heard of products like IBM’s InfoSphere Streams, Tibco’s Event Processing product, or Oracle’s CEP product? All good examples of commercially available stream processing technologies which help you process events in real-time.
I’ve been asked what I consider as “Big Data” versus “Small Dat...
My fellow Technical Evangelists and I have authored a content series that steps through building your very own Private Cloud by leveraging Windows Server 2012, our FREE Hyper-V Server 2012, Windows Azure Infrastructure Services ( IaaS ) and System Center 2012 Service Pack 1.
Week-by-week, we walk ...











