requirement discussion on test report


Hake Huang
 

Hi All,

 

To follow up the topic of this weekly meeting, we are seeking for a better tools to present our test report

We are trying to gather requirements for this tool sets, please help to comments, Thanks.

1.      The tool scripts are open source.

2.      The tool scripts can take several junit xml files and generate a comparison diagram in html format.

3.      The comparison diagram in html format better have sorting and filterring function embedded.

4.      Test result tacking diagram can be generated such as we can see whether an issue is:

a)       Newly added case.

b)       Regression issue.

c)        Long existing issue.

d)       Randon failure from time to time.

 

Here is the background of this topic

Seeking tools to better summarize the test result.

a)       Current test report is generated by junit via ` junit2html results.xml output.html`.

                     i.            Pros:

1.        Easy to generate all report.

2.        Open source. No complext database required, easy to integrate to CI

                    ii.            Cons:

1.        The summary report is too long to read.

2.        Can not sort/ filter dynamically.

3.        No historic data analysis.

b)       Target:

We need to build up a good scripting system, combine with several opensource project solution, which can do below:

1.        Open source easy to integrate to CI.

2.        Easy to generate report.

3.        Test result can be sorted and filterred.(we need scripts to summary several release RC, and use css/in line script to support this)

4.        Further test data minerring. (scritps that can do test data analysis), release snapshot analysis can be generated by this tools and put to wiki, and anyone can setup same analysis environment by own.

a)       https://rapidminer.com/get-started/

b)       https://www.knime.com/

c)        https://datamelt.org/