7 Ways to Improve Defect Reports
As a software tester, it is easy to get carried away when digging into a new piece of software; the unexplored area of newly developed software calls to us "Test me, test me!".
If the development has been poor, and several defects are found at once, it is often easy to skip one of the most important tasks the tester does; create good reports. In a hurry, several failures have to be transformed into defect reports, and to make sure none is forgotten while the first ones are written, it is quite common to type in the short version, which often lack important information.
When we test, we tend to be engaged in what we do, and we forget that the developer who'll receive the defect won't be able to understand what situation the failure occured: What test data was used. Which environment was the test performed, and what kind of known/business data was loaded into the database. Did we do any workarounds to be able to reach the frontend, and what other parameters did we enter.
If the software is in an early phase of the project, the development team might have quite a different version in their environment, so the screens may look completely different. There may have been changes to the code, or even just a dataload may cause the software to behave differently in the development environment, so it is of great importance to deliver good test reports.
Defects and test execution reports are not only for the development team; they can be used to show the evolution of the software as the project goes along, and if a failure occurs at a later stage (e.g. in Acceptance Test or even in production), the test team can refer to the test report and tell if the feature (and/or surrounding area) has been tested before.
There are two main reasons for making the test execution reports and the defect logs as good and descriptive as possible:
- The developer needs to understand what the failure is, and how it is caused. The better the description is, the easier it is for the developer to understand what's causing it.
- When the defect is fixed, and the development team has released it to the test environment, it is very likely that some time has passed by. It may have been days, or even weeks, since that time when the tester was really engaged and digging into that part of the software, and thus the tester won't neccesarily remember how the failure was caused (if the report is poorly written, that is)
So, what are the steps to write a good defect report..? Here are some tips:
- Start with a short description of what the failure is. In a short sentence, sum it all up so that the developer might go straight to fixing the problem, if he understands right away what's causing it. Even though the description is short though, it should be precise and meaningfull, so that it is possible to understand what the problem is.
- Next, write down a summary of what was done. This can be written as a step by step guide saying what you did, or you can write it as more as an essay kind of text. It is still important to remember though: You want to keep the attention of the reader. Too much text, and nobody will read what you've done, and that's a good reason (maybe not good, but still very common indeed) for not getting the bug fixed. The developer will send the defect back to you and say he can't reproduce the failure.
- Attach scripts for loading test data, and logs from the inserts you did. The developer will probably try to reproduce the failure in her environment, so it is important to attach data so that the preconditions for the test are the same. Also, when the defect is fixed and returned to you, you might get a different result than expected, so you'll want to try it with the exact same script as you did the first time. If the test data is stored in e.g. an excel spreadsheet, and then generated into sql, you don't want to regenerate the data if you have to be 100% sure that no one have made changes to the data.
- Screenshots are of great value when logging a defect; they are proof of what the situation look liked when the defect was logged, and they are a quick way for the developer to see where the problem is located.
- Data extract. After code has been executed, what happened with the data? If it has any value, retrieve the altered data from the database, and attach it to your defect. If records were supposed to be created in a table, and were either incomplete or not there at all, show this by attaching it (it can even be a screenshot from the table in e.g. Toad).
- If you have time to do some debugging, or have a vague idea about where the defect is located, it's a nice thing to help the developer by describing your analysis about the bugs whereabouts.
- Finally, you'll want to add the general information, such as which environment and version did you perform the test, what part of the functionality were you testing at the time. What test (test case number) were you executing, and so on.
