Performance Testing
We do the uncool stuff so that you don't have to...
The performance test area is a bit strange. On a new project testing of its performance is usually not regarded as very cool. On an existing project it’s something that needs to be done periodically (especially around future capacity) but usually it falls to some unfortunate soul who has to do it alongside their usual responsibilities.
We like uncool. We have the experience and tools to make it attractive.
How do we do it?
We usually come on-board for the requirements gathering and build and after a successful run or runs, you’ll have the option for on-demand support. These are the phases we usually go through:
Gather requirements
We want to find out if a system meets certain performance requirements. We may argue about what these requirements are. Perhaps as a business we have figured out that if a transaction is not performed within 5 seconds, the user is more likely to abandon it. Or maybe we’re bound by some regulation or contract to maintain a SLA (Service Level Agreement).
Setting the targets for the requirement is usually a bit arbitrary and often it’s a good idea to question where they came from and why they are important. By this process we are able to find out the relative priority of the criteria which will inform your test plan. We always ask why a requirement is in place. If there is no answer chances are it’s not a real requirement.
It may be a good idea to document performance requirements in the form of a user story:
-
As a user I want the product search page to return results within 1 second so that I don’t switch to another service.
-
As a business we want to be able to return any page within 1 second to up to 1000 concurrent users so that potential customers don’t switch to another service.
The advantage of a user story is that it neatly encapsulates the reason for a feature of the system including what, who and why in a simple readable sentence.
Build & Run
​
We work with several industry standard tools as well as proprietary ones. Generally we found that generating the input data for a capacity test is the biggest challenge and we’ve developed proprietary tools to do this. Especially in the financial services settings where the data can be date dependent it’s a big challenge which we have successfully met several times in the past.
Results Reporting
All results are gathered in a central DB from where you can easily gather stats on either granular or aggregate level. The data we store depends on your requirements. Report generation is very flexible so as long as we have the data, any type of report can be generated regularly or ad-hoc.
Repeat
Once all of the above is done it’s ready to be repeated as often as you need. We’re on stand-by to help.