Currently it just diffs the stdout and ignores other reference files.
It also doesn't filter out noise in the diffs from non test related
simulator messages. These include startup messages, messages when the
simulator finishes executing, and some non-standard warnings, etc.
Change-Id: Idcb19edd893cd8818423c2c5ebb6cbfb278baffa
Reviewed-on: https://gem5-review.googlesource.com/12054
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
This just checks whether gem5 exited of its own accord with a
successful error code, or in other words that it didn't hang or crash.
More checking will need to be added to verify the output against the
golden reference.
Change-Id: I1ddef56aa73b5f700743830bd6212804531c484f
Reviewed-on: https://gem5-review.googlesource.com/12053
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Add -j and --timeout options to the execute phase of verify.py.
The --timeout option is implemented using the timeout utility program
which is assumed to be available on the host system. Python 3.3 added
a timeout argument to the subprocess module which is an alternative
approach, but then we would be dependent on python 3.3.
-j is implemented using the standard multiprocess.pool.ThreadPool
class.
Change-Id: I15b92f2b14de6710e2027a6a19984b2644b2a8df
Reviewed-on: https://gem5-review.googlesource.com/12051
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Now that we bit the bullet and stubbed out sc_simcontext and related
functions a little bit, we can enable a couple more tests. This change
also adds in some functions the new tests expect sc_simcontext to have.
Change-Id: I00b5cc0c6eb658eb689b9c85ed171f290009768d
Reviewed-on: https://gem5-review.googlesource.com/11356
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
This type is deprecated, but some tests still rely on it. This change
adds just enough of it to satisfy the tests, and also the several
different mechanisms for retrieving the sc_process_b which refers to
the currently active process.
Change-Id: Id122ae5df23744b5de1e1c97573412f97a73b77a
Reviewed-on: https://gem5-review.googlesource.com/11351
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Make the --list option of verify.py print a count of how many tests it
printed. Because --list respects the filter options, this is an easy
way to check how many tests have certain properties.
Change-Id: I03fac349a946631c20c8e6b49e0ad8934872898e
Reviewed-on: https://gem5-review.googlesource.com/11288
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
One would never have worked as far as I can tell, and the others depend
on a feature that breaks building the systemc library.
Change-Id: Ia3db31a15675ca6df2db16362ee2d9c8700cd0af
Reviewed-on: https://gem5-review.googlesource.com/11287
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Not all of the tests this filter accepts actually work, but we haven't
yet explicitly decided not to try to make them tests work yet.
Recording this in a file makes test runs more repeateable, and creates
a place to record why certain tests or groups of tests are being
excluded.
Change-Id: I91d44b6500a3e8ff5d5808222ce03f138374cf8b
Reviewed-on: https://gem5-review.googlesource.com/11258
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
The --filter option is very useful, but can get to be long and complex
and a bit too cumbersome to use from the command line. This change
adds a --filter-file option which is mutually exclusive with --filter
and which reads the filter expression from a file instead of accepting
it on the command line.
Change-Id: I381c92ddf0d9fe62acd20432fa4868e2121405b8
Reviewed-on: https://gem5-review.googlesource.com/11257
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
The src_dir method of the Test class was using the wrong attribute of
the test from the json file. It should use path, but was using src_dir.
Change-Id: Iaaaf31c31b11b68ecd7dd98807e6b78597cebb99
Reviewed-on: https://gem5-review.googlesource.com/12029
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
The tests were referring to constants defined in the Accellera systemc
implementation which identified various log message types. This change
replaces those (sometimes quite long) string constants with their
actual value. This doesn't make that interface any more fragile since
the constant value won't track between the Accellera version and this
one, but it does make its fragility more explicit by using the value
directly.
If in the future we decide to hide the magical nature of those
particular string constants, we could make them into our own constants
in the implementation. Regardless, the tests shouldn't refer to
internal details of the implementation.
Change-Id: I116a7407fcac49367649b250683117f3ccf4f9ed
Reviewed-on: https://gem5-review.googlesource.com/11181
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Some more tests were including Accellera headers specifically, even
after including systemc.h which would bring those headers in alongside
all the others.
Change-Id: Ib2db0a410e0ecea0035b636e8dd04c85045b8578
Reviewed-on: https://gem5-review.googlesource.com/11179
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
The sc_vector class is available through systemc.h already, so there's
no reason to include an internal path which is only there on Accellera
by coincidence of the implementation.
Change-Id: I41cab4711c0837cd9b20c21871b79be5165bf498
Reviewed-on: https://gem5-review.googlesource.com/11176
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
The only required option for the script is the path to a build
directory, for instance build/ARM. It uses that when running scons and
when referring to generated files (like the built versions of the
tests). It uses the location of the script itself to find source files,
like the "golden" version of outputs.
This script runs tests in three phases, compilation, running, and
then verification. By default, all three phases are run for all tests,
but which phases run and how they work can be customized by specifying
them explicitly on the command line with the --phase option.
Each time the --phase option shows up on the command line, it should
be followed by the name of the phase being specified. That both
declares that that phase should be run, and also that any subsequent
options apply to that phase, at least until the next --phase option.
Currently, only the "compile" and "execute" phases are implemented, and
only the "compile" phase has been tested at all or has any options
defined. The "compile" phase simply takes all the options it was given
and pass them on to scons. The "verify" phase simply prints a message
which says what it would have verified.
The script can list the available tests with the --list option, as
determined by the json systemc test manifest file which scons can
generate, and which the script can refresh if passed the --update-json
option. You can also specify what "flavor" of build you want to do, ie
opt, debug, fast, etc., with the --flavor option. Only one flavor can
be tested at a time, but that should be the desired behavior most of
the time. If multiple flavors need to be tested, the script can be
invoked multiple times.
Finally, there is a --filter option which accepts a python expression
as a string. That expression is evaluated in the namespace of the
properties of each test in the json file, and if it returns true then
that test is run. For instance, to verify only compile_only tests, you
would use:
systemc/tests/verify.py build/ARM --filter 'compile_only'
To test only compile_only tests named bob, you would run this:
systemc/tests/verify.py build/ARM --filter \
'compile_only && name ="bob"'
Also included is a simple config.py which is the beginning of a config
which will run the systemc regression tests. Right now, all it knows
how to do is run sc_main.
Change-Id: I62666be8b1622d1355153e623b4274a939507e44
Reviewed-on: https://gem5-review.googlesource.com/10975
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Otherwise, having hundreds of statically linked gem5s takes up a huge
amount of space, and all those repeated linkings brings the mightiest
workstation to its knees with sufficient parallelism, or will take
forever without it.
Change-Id: I4c358b1a50c5e2b0027ac72423f887e24c786b19
Reviewed-on: https://gem5-review.googlesource.com/10959
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
This mechanism scans the systemc test directories as described in their
original distribution. It tells scons how to build each test
executable, and also how to build a json manifest file which
lists all the tests and some properties about them.
Change-Id: I8ebc748c1aed71f0bb76e04a2040f15abd2837d9
Reviewed-on: https://gem5-review.googlesource.com/10956
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
