96 Commits

Author SHA1 Message Date
Joël Stemmer
0b843fc2b7 gtr,parser/gotest: Move ReportBuilder to gotest package
The ReportBuilder was made specifically for building a gtr.Report from
parsed go test output. At this point it's unclear whether this will be
reusable if we ever introduce other parser. For now let's move it closer
to where it's used and let's not make it part of the public API yet.
2022-03-27 23:55:59 +01:00
Joël Stemmer
26cb3d5eb3 Print JSON instead of raw Go structs for -debug.print-events 2022-03-22 22:41:50 +00:00
Joël Stemmer
985100c8e8 junit: Move test output within the XML tag for skipped tests
For consistency with failed tests and tests with unknown status.
2022-03-22 22:05:23 +00:00
Joël Stemmer
1c7fff6728 junit: Rename "Run error" to "Runtime error" for dummy error tests
In cases where a test result is not found, for example when a panic
happened, a dummy failing test is added to the report. Since we don't
know exactly what failed, its error message was simply "Run error". This
has been renamed to "Runtime error".
2022-03-22 22:05:23 +00:00
Joël Stemmer
e0a4faed09 fixup: add skipped attr to testsuites 2022-03-22 22:05:23 +00:00
Joël Stemmer
4b87b03e55 junit: Use the timestamp from the gtr.Report when creating Testsuites 2022-03-22 22:05:23 +00:00
Joël Stemmer
b0a9864d1e parser/gotest: Add Option to override the timestamp function
By default, the current local time is used when generating a Report. The
TimestampFunc Option makes it possible to override this default
behaviour. This can for example be used in tests to make sure the
timestamp is a fixed point in time.
2022-03-22 22:05:23 +00:00
Joël Stemmer
3190f85fe3 gtr: Add Timestamp field to Package
When using the ReportBuilder, the Timestamp will by default be set to
the current local time.
2022-03-22 22:05:23 +00:00
Joël Stemmer
c78e04707f gtr,parser/gotest: move Event and building a report to parser/gotest
The Parse method now directly returns a report, rather than a list of
events that then need to be converted into a report. As part of this
change, the Event struct has also been moved to the gotest package. It's
now the responsibility of the parser to construct a gtr.Report.
2022-03-22 22:05:23 +00:00
Joël Stemmer
832cc97037 parser/gotest: refactor parser so Parse is no longer a top level func
Making Parse a method on a Parser struct makes it possible to later
define an common parser interface.
2022-03-22 22:05:23 +00:00
Joël Stemmer
0e7d095a28 gtr: add documentation to exported types and methods 2022-03-22 22:05:23 +00:00
Joël Stemmer
5e492f8212 gtr: rename Package.AddProperty to Package.SetProperty
Properties are stored in a map, so SetProperty better reflects its
behaviour.
2022-03-22 22:05:23 +00:00
Joël Stemmer
24c2ee41ce parser/gotest: refactor tests to run one test per input line 2022-03-22 22:05:23 +00:00
Joël Stemmer
10affc0da1 Store arbitrary key/value properties in Report 2022-03-22 22:05:23 +00:00
Joël Stemmer
9aa9bd94d8 gtr: Remove unnecessary TODO
There's no need (yet) to check if output is potentially a build error,
as build errors are currently handled separately by the parser/gotest
package.
2022-03-22 22:05:23 +00:00
Joël Stemmer
a77bfe0f1c gtr,junit: move creation of JUnit testsuites from gtr to junit
Package gtr shouldn't need to know about the existence of different
output formats like junit.
2022-03-22 22:05:23 +00:00
Joël Stemmer
334044509c Remove custom properties parsed from test output
In the future it will be possible to add your own properties via
commandline flags.
2022-03-22 22:05:23 +00:00
Joël Stemmer
a70d508a2e parser/gotest: support parsing test summaries when there were no tests 2022-03-20 22:35:50 +00:00
Joël Stemmer
458fe89a9a Omit empty <properties> tag when there are no properties.
Go doesn't omit empty parent tags for empty values[1], so we'll work
around this for now by creating a pointer to the property slice.

[1]: https://github.com/golang/go/issues/7233.
2022-03-20 22:35:50 +00:00
Joël Stemmer
43c784a63b Set non-zero exit code for build/run errors when using -set-exit-code 2022-03-20 22:35:50 +00:00
Joël Stemmer
c8de8e11aa gtr: Remove unused findBenchmark function 2022-03-20 22:35:50 +00:00
Joël Stemmer
0c57c726ce junit: Add documentation to the junit package 2022-03-20 22:35:50 +00:00
Joël Stemmer
016e21634a gtr: Add documentation to ReportBuilder 2022-03-20 22:35:50 +00:00
Joël Stemmer
6b85350845 parser/gotest: Refactor tests to test each line type specifically
This makes the tests more focused on each distinct type of line that is
recognized. Maintaining and extending these tests will also be less work
going forward.
2022-03-20 22:35:50 +00:00
Joël Stemmer
e2a7c1a394 parser/gotest: Add TODO to remove the build_output type 2022-03-20 22:35:50 +00:00
Joël Stemmer
bab26d76b5 gtr: Write test output to JUnit SystemOut
Fixes #108
2022-03-20 22:35:50 +00:00
Joël Stemmer
76f68922a2 junit: Output SystemOut and SystemErr contents as CDATA 2022-03-20 22:35:50 +00:00
Joël Stemmer
7d75448298 gtr: Remove debug print 2022-03-20 22:35:50 +00:00
Joël Stemmer
817a23b10d gtr: Handle end_test events without corresponding run_test event
When running `go test` without the `-v` flag, the output may not contain
everything that we expect. For example, no output is generated for
passing tests. Even for a failing test the output does not contain a
`=== RUN` line.

Currently, this resulted in us ignoring the test result since we
couldn't find an existing test to assign this result to. We should
however handle this situation gracefully, and just assume a test exists
when only encountering a test result line.

References #109
2022-03-20 22:35:50 +00:00
Joël Stemmer
014828bef4 junit: Correctly count skipped tests in junit Testsuite
Fixes #116
2022-03-20 22:35:50 +00:00
Joël Stemmer
dde9ef5310 junit: Change JUnit Result Data field to cdata
The Result Data field contains the test output that was read from stdin.
This used to be a chardata field, which escapes many characters in the
resulting XML. Writing this output with <![CDATA[]]> tags makes the raw
XML output much easier to read in e.g. a text editor.
2022-03-20 22:35:50 +00:00
Joël Stemmer
fc0a210dc9 junit: Make failures and errors required fields
Fixes #101
2022-03-20 22:35:50 +00:00
Joël Stemmer
2ece8eae1a gtr: Set hostname and timestamp fields when creating JUnit Testsuite
Fixes #117
2022-03-20 22:35:50 +00:00
Joël Stemmer
c3acdf13c2 parser/gotest: Remove unnecessary string concatenation in regexSummary 2022-03-20 22:35:50 +00:00
Joël Stemmer
1069a04b9f parser/gotest: Fix failing build error test 2022-03-20 22:35:50 +00:00
Joël Stemmer
cb7f773314 gtr: Add build error test case 2022-03-20 22:35:50 +00:00
Joël Stemmer
80a9b62b38 gtr: Fix prefix trimming of test output 2022-03-20 22:35:50 +00:00
Joël Stemmer
506a968ade gtr: Handle failed summary when no tests failed 2022-03-20 22:35:50 +00:00
Joël Stemmer
4fdb6ca564 gtr: Move tests from build and runtime errors to end of junit report 2022-03-20 22:35:50 +00:00
Joël Stemmer
5007397e33 gtr: Reset active test when encountering a status line
This is to ensure that we don't append output lines that follow after a
status line to the last active test.
2022-03-20 22:35:50 +00:00
Joël Stemmer
f7ae0905a2 gtr: Add support for pause/continue tests 2022-03-20 22:35:50 +00:00
Joël Stemmer
c845dfac27 gtr: Correctly handle packages without tests 2022-03-20 22:35:50 +00:00
Joël Stemmer
9167a5d41d gtr: Add support for runtime/init errors 2022-03-20 22:35:50 +00:00
Joël Stemmer
ffc33941fa gtr, parser/gotest: Add support for build errors 2022-03-20 22:35:50 +00:00
Joël Stemmer
b1b88456c1 gtr: Refactor ReportBuilder flush 2022-03-20 22:35:50 +00:00
Joël Stemmer
52a5393fd3 gtr: Refactor ReportBuilder 2022-03-20 22:35:50 +00:00
Joël Stemmer
3f9d5b62db gtr, parser/gotest: Move output formatting out of parser 2022-03-20 22:35:50 +00:00
Joël Stemmer
08a21eb096 parser/gotest: Fix output trimming 2022-03-20 22:35:50 +00:00
Joël Stemmer
1563e51b7c gtr: Fix testsuite duration 2022-03-20 22:35:50 +00:00
Joël Stemmer
d2d65ebe03 gtr: Fix skipped test output 2022-03-20 22:35:50 +00:00