In cases where a test result is not found, for example when a panic
happened, a dummy failing test is added to the report. Since we don't
know exactly what failed, its error message was simply "Run error". This
has been renamed to "Runtime error".
The testdata directory contains various inputs and the resulting
report.xml that is expected to be generated. Making changes that result
in differences in the generated XML reports requires the report.xml
files to be manually updated. This is time consuming and error prone.
The generate-golden.go script (re)generates all report.xml files in the
testdata directory from the corresponding input. This can be done by
simply running `go generate` from within the testdata directory. Make
sure the generated reports contain the changes you expected before
committing them.
As part of this change, all reports have been regenerated. The diffs in
the report.xml files are caused by the following recent changes:
- XML attributes now appear in the order as defined in the structs in
the junit package.
- Failure and error messages are now wrapped in <![CDATA[]]>.
- Package names are no longer truncated to the last path component.
When running `go test` without the `-v` flag, the output may not contain
everything that we expect. For example, no output is generated for
passing tests. Even for a failing test the output does not contain a
`=== RUN` line.
Currently, this resulted in us ignoring the test result since we
couldn't find an existing test to assign this result to. We should
however handle this situation gracefully, and just assume a test exists
when only encountering a test result line.
References #109
Sometimes, the text after "# " shows the name of the test binary
("<package>.test") in addition to the package
e.g.: "# package/name [package/name.test]".
This change ensures that the packages' build outputs are associated
correctly in this case.
Mainly:
* Moving the averaging/merging of benchmarks from the parser to the formatter package
* Tightening up the regex so it only captures the numeric values (no more of trimming spaces and the ns/op)
* Deleting the writing up in xml file the benchmark memory sections of B/op and Allocs/op
Also added a test case for parseNanoseconds().
