It is always good to add all the performance testing – engineering priorities on the scrum board and prioritization from this point helps. Once these tasks are prioritized the efforts required for each of them can be identified through planning poker exercise. Now as the priorities are set, the performance engineering tasks are planned in respective sprints. Based on this prioritization, performance testing activities are also planned and completed.
It is very essential in Agile and in performance testing to test the app once the bug is fixed. Retesting not only helps in testing weather the bug is fixed but also helps in identifying the degree of issue resolution. If the issue is partly fixed and business target is not met through the fix then what could be done to fix it so that the customers can be satiated.
Performance of the application being it’s one of the most important selling attributes cannot be neglected and so are the performance tests. If there is any bug that is impacting the performance tests to proceed, then that must be resolved at priority. Resolution of performance issues require many teams to participate. For all of them to proceed in their tasks and work to improvise the application further, it is critical that their hurdles are removed.
Many times after tests a lot of performance issues are identified and those are prioritized after functional bugs. This attitude within the team should be gotten rid of. Reason being, a small performance issue can further snowball and can lead to complete change in design / architecture if not fixed at the right time. Also if performance issues are resolved at an early stage in the software development lifecycle then very few performance issues are identified in the later stages of lifecycle.
Pre-optimization is a hybrid term here used to signify optimization (prioritization due to budget constraints) in tests even before the tests begin. Usually this is not a good practice to follow since a lot of operational issues in testing and functional & nonfunctional issues in the app aren’t identified even before the tests. That doesn’t mean that there should be no plan. Planning is must but an intelligent plan is what one must seek for. Also in AGILE you always have the option of changing plans and updating backlogs which can come in handy in situations when plans are impacted due to unavoidable circumstances.
However experienced you are and whatever knowledge you have gained about the technology and the system, you can never guess where the next performance issue is or might come from. Hence the wise approach would always be to test and retest the application with every important build.
It is very important for the performance tester to completely understand the infrastructure of the production environment and have it on the back of his mind. He should always aspire to have a similar environment (a replica of production) for performance testing. This will help him create a production like load on the performance test environment and thus produce (or identify) issues that one would have got in production.
Scaling up infrastructure: As you proceed in your app development and start noticing performance issues, there would often be instances when you would feel like scaling the infrastructure / adding the hardware to increase the capacity and thus the performance of the system. However this may not be the best approach to take ahead since with increase in user (customer) load further, you would again wish to add the servers and at one point of time it would not remain cost effective. At that point, it would be too late to fix the performance issues. Imagine if Facebook would have followed this approach. How much infrastructure would it have needed?
The app works in production continuously. And the tests usually run from 1-2 hours. There is a possibility that many of the issues that the app might notice when it is subjected to load for extended durations may not be noticed in these tests of 2 hours. It is hence necessary to periodically (not in every release thought) test the app under load for extended durations. This would help the testers identify issues like those of memory leaks and help the developers fix those in time.
This post is also available in: Anglais