Step 1. Use consistent and reproducible environments
There are some obvious things you need to do.
- Use the same:
- PC
- web server
- application server
- communications channel and method
- Perform exactly the same actions on each test.
- Make your test like a real end-user’s experience in order to better identify significant improvements you can make.
- Use a PC that is configured like an ‘average’ end-user’s PC.
It would be ideal if your web server, application server and communications channel were always under the same load. Practically this is rarely the case, so you need to be able to compensate for this. For example performing a test at 6:00am, in an empty office, and the comparing the elapsed times with another test taken at 10:30am, when everyone is working flat out, would be silly.
Normally you can make sure your PC is under the same load, but sometimes something as simple as receiving a large e-mail during a test can invalidate your results.
Define exactly what your PC, web server and application server will be on paper for later reference.