Commit Graph

11 Commits

Author SHA1 Message Date
Bobby R. Bruce
421f963f9d tests,arch-alpha: Removing ALPHA ISA from testlib config
Change-Id: Icded5f4aec7bc212a818a97c4de5d7b8f8757121
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/26823
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2020-03-17 15:35:47 +00:00
Giacomo Travaglini
d1e2f18230 ext: Add failure node to JUnit xml file
"failure" is a child of the testcase node:

https://llg.cubic.org/docs/junit/

It allows xml parsers to understand which testcase failed the run.
Otherwise CI frameworks like jenkins wouldn't be able to classify every
single testcase. Prior to this patch testlib was using
testsuites.failures and testsuite.failures only. These are simply
reporting the number of failures.

Change-Id: I0d498eca029c3232f2a588b153b6b6829b789394
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/25083
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Ciro Santilli <ciro.santilli@arm.com>
2020-02-13 16:40:49 +00:00
Giacomo Travaglini
5ab5d07d71 tests: Add --host tag
A host tag has been added to take into consideration the host ISA which
is running gem5 (default is X86).
There might be regressions which are supposed to be run on a particular
host machine only. This could be the case of dynamically linked
regressions which require dynamic linker/loader + shared libraries of
the same ISA as the target.

Change-Id: I4c4044a4f1b8899f443856340df302df7c1aaf8e
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Ciro Santilli <ciro.santilli@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/24527
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2020-02-10 09:57:56 +00:00
Giacomo Travaglini
454ba32c51 tests: Adding --bin-path option to select tests bin directory
So far lots of tests will download binaries inside the gem5 directory.
The path is also specific to the test being run.
This doesn't play well with an environment where gem5 is cloned from
scratch for every build, or if several gem5 are cloned in a single
machine.
Binaries will be automatically downloaded every time this happens.

This patch is adding a --bin-path option, so that it's possible to
setup a fixed directory with all pre-downloaded binaries.
By default it is set to None to preserve original behaviour.

Change-Id: I42fb25e3ce0a495c73672b15a097b1bd2607795c
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Ciro Santilli <ciro.santilli@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/24525
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2020-01-21 09:13:07 +00:00
Nikos Nikoleris
8282b7408b tests: Refactor the Gem5Fixture to derive from UniqueFixture
Gem5Fixture is used to define a fixture for building the gem5
binary. Most tests are expected to define their own Gem5Fixture,
however, as some might depend on the same binary (e.g.,
./build/ARM/gem5.opt), they will try to re-define a fixture for the
same target. This patchset changes Gem5Fixture to derive from
UniqueFixture.

In addition, this patchset changes the way global fixtures are
discovered to work with the new Gem5Fixture class. Instead of
enumerating them when test definitions are loaded, we do so after the
tests have been filtered according to specified tags (e.g., include
opt variant, exclude fast, debug variants).

Change-Id: Ie868a7e18ef6c3271f3c8a658229657cd43997cb
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19251
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Maintainer: Giacomo Travaglini <giacomo.travaglini@arm.com>
2019-08-09 14:03:21 +00:00
Jason Lowe-Power
76d9c83887 ext,tests: Add back failing exceptions
Change-Id: Idf4ba8a2a3888787abf33d1a4ac52fcf146ce732
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/17452
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
2019-03-21 15:57:10 +00:00
Jason Lowe-Power
4c28149ffa tests,ext: Add skip_cleanup implementation for TempdirFixture
Change-Id: Idc5ec9309a4ef3c0ad0c7e8b2df47294acc97ec4
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/17451
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
2019-03-21 15:57:10 +00:00
Jason Lowe-Power
fced86b061 ext,tests: Make return code based on test results
This patch also fixes a spelling mistake.

Change-Id: I8635216e512c10913a9cda54541d7e31e0d22a40
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/17450
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
2019-03-21 15:57:10 +00:00
Jason Lowe-Power
f871fd33e1 ext,test: Provide default terminal size
When creating the separator for printing things to the terminal (=.*) we
use an ioctl that isn't supported in some sandboxed environments. When
running on the Google jenkins server (kokoro) it errors with an IOError.

Change-Id: I148dd87cffe6e93d6723a386aecf9a9ea6c5b455
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/17449
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
2019-03-21 15:57:10 +00:00
Sean Wilson
c7c0678cc6 testlib: No catch of custom exceptions in sandbox
Custom exceptions are not always properly pickled which could lead to
the sandbox test executor to crash when it tries to re __init__ pickled
exceptions thrown from the failed test.

Change-Id: I4e2ffe5802dda668b5d61c5a16e0989717121a04
Signed-off-by: Sean Wilson <spwilson27@gmail.com>
Reviewed-on: https://gem5-review.googlesource.com/12167
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
2018-09-05 01:04:43 +00:00
Sean Wilson
07ce662bd2 tests,ext: Add a new testing library proposal
The new test library is split into two parts: The framework which resides
in ext/, and the gem5 helping components in /tests/gem5.

Change-Id: Ib4f3ae8d7eb96a7306335a3e739b7e8041aa99b9
Signed-off-by: Sean Wilson <spwilson2@wisc.edu>
Reviewed-on: https://gem5-review.googlesource.com/4421
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
2018-08-17 16:37:42 +00:00