Adds a third argument to the weekly.sh script, allowing for a different
number of "run threads" (maximum number of tests that can be run at
once) and "compile threads" (threads to allocate to the running of
gem5). This is useful for cases where a high number of compilation
threads is wanted (e.g., 50 or so), but that number of tests running at
once is undesirable.
Change-Id: Ifd67c5826c6f3602db1546a8a48e615cb03ba820
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/69241
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
The weekly test script was implicitly assuming that no m5out
directory existed in the folder where the script was run.
However, if a prior test ran and failed, it would not clean up
its m5out directory, causing the weekly tests to fail.
This commit resolves this by removing the m5out directory before
trying to run any tests in the weekly script. Moreover, we also
update the weekly script to explicitly remove this m5out directory
at the end of the script.
Change-Id: If10c59034528e171cc2c5dacb928b3a81d6b8c50
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67198
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
As of https://gem5-review.googlesource.com/c/public/gem5/+/64177 we
support version 22.04. This patch therefore updates the testing
infrastructure (kokoro/quick, nightly/long, weekly/extra-long) to use
the Ubuntu 22.04 docker image.
The "jenkins/gem5art-tests.sh" test script has been updated to no longer
require the `pip upgrade`. This was needed for Ubuntu 20.04 as it
utilized an older version of pip which did not have all the dependencies
these tests requried. As of Ubuntu 20.04 this is no longer required.
Change-Id: Ia8f8b1b2c62ad5d5a8419cb31b6a1d2b6dff7ac9
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64291
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
If the pre-commit could not be installed the compilation would continue
as the exit code from running the pre-commit install script was not
read or processed. This commit adds a check. If the install is
unsuccessful the users is asked whether they want to continue the
compilation or not.
This check can be ignored with the '--ignore-style'. The tests have been
updated to include this flag in all cases we compile gem5 to ensure
tests remain automated and uninterrupted on Kokoro/Jenkins.
Change-Id: Iaf4db71300883b828b00d77784c9bb46b2698f89
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63012
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
The following weekly tests were failing:
https://jenkins.gem5.org/job/weekly/53
This appears to be due to the "lulesh" tests failing with the docker
container running out of memory. This can be recreated with:
```
git clone -b develop https://gem5.googlesource.com/public/gem5
cd gem5
git clone -b develop https://gem5.googlesource.com/public/gem5-resources
docker pull gcr.io/gem5-test/gcn-gpu:latest
docker build -t hacc-test-weekly gem5-resources/src/gpu/halo-finder
docker run --rm -u $UID:$GID --volume $(pwd):$(pwd) -w \
$(pwd) --memory="18g" hacc-test-weekly bash -c \
"scons build/GCN3_X86/gem5.opt -j`nproc`"
docker run --rm --volume $(pwd):$(pwd) -w \
"$(pwd)/gem5-resources/src/gpu/lulesh" \
-u $UID:$GID --memory="18g" hacc-test-weekly make
docker run --rm -u $UID:$GID --volume "$(pwd)":"$(pwd)" -w \
"$(pwd)" --memory="18g" \
hacc-test-weekly build/GCN3_X86/gem5.opt configs/example/apu_se.py -n3 \
--mem-size=8GB --reg-alloc-policy=dynamic \
--benchmark-root="$(pwd)/gem5-resources/src/gpu/lulesh/bin" -c lulesh
```
It is currently unknown as to why this process consumes so much memory
or how it can be reduced, but increasing the docker container limit to
24GB for the Weekly tests does appear to fix the issue.
Change-Id: Ifcd65d4ab8bf6a12ddf7a45d61c779e691619072
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/60009
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
GCN3_X86 and VEGA_X86 can use the same test binaries to run tests for
that build configuration. This adds an option to the nightly and weekly
test scripts to select which GPU ISA to test. By default the GCN3_X86
ISA is tested which was previously the only test. No behavior is changed
from current testing.
Change-Id: Ifedcdea5d7c1c8e9994559a20aea5e1954180772
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/54083
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Without this verbose flag, there is no information output on the tests
currently running. Output is only given on the pass/fail status of a
test after completion. This is unhelpful if the tests fail due to a
timeout (e.g., a test has stalled).
The '-vv' verbose flags sets up the TestLib package to output when a
test has started execution. This information can help in figuring out
which test caused a timeout event to occur.
Change-Id: I14e6752d40a7c8d7189211584073ba79c88ed7d5
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/53305
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
The GPU models support a simple register allocation policy (1 WF/CU at a
time) and a dynamic register allocation policy (up to max WF/CU at a
time). By default, the simple policy is used. However, the dynamic
policy is much more realistic relative to real hardware and thus much
more important to ensure it works in the regressions. This commit
updates the nightly and weekly regressions accordingly to run the
dynamic register allocation policy.
Change-Id: Id263d3d5e19e4ff47f0eb6d9b08cbafdf2177fb9
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/52163
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Add the Pannotia benchmarks to the weekly regression suite. These
applications do a good job of testing the GPU support for irregular
access patterns of various kinds. All inputs have been sized to use
relatively small graphs to avoid increasing runtime too much. However,
even with small input sizes Pannotia does run for a while.
Note that the Pannotia benchmarks also use m5ops in them. Thus, this
commit also adds support into the weekly regression for compiling the
m5ops (for x86, since that is what the GPU model assumes for the CPU).
Change-Id: I1f68b02b38ff24505a2894694b7544977024f8fa
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/51968
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
66a056b8 changed the weekly regression to use a single docker for
all GPU tests, to reduce how many times gem5 needed to be compiled.
However, in my local testing of that patch, gem5-resources was not
deleted until after the docker was created -- which causes a problem
when gem5-resources does not exist already from a prior run, since
the creation of the dockerfile requires it for HACC. This commit
fixes this problem by moving the pull of gem5-resources to be before
anything else related to the GPU happens.
Change-Id: I006860204d03807d95628aa5dcf6e82d202fef9c
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/51907
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
This commit adds HACC (a GPU HPC workload) to the weekly regression
tests. HACC requires a number of environment variables to be set, so
to avoid setting all of them manually, we use a specific Dockerfile for
it. To avoid compiling gem5 once for this docker and once for the other
GPU tests in the weekly regression, this commit also updates the weekly
regression such that all GPU weekly regression tests use HACC's docker
for their tests.
Change-Id: I9adabbca01537f031cbc491ddf1d3e7dd155f3f2
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/51708
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
DNNMark and LULESH were both cloning and removing gem5-resources as part
of their tests, since they were committed separately/in parallel. Clean
this up so we only remove and pull gem5-resources once now in the weekly
regression script.
Change-Id: I5ab1410b0934bf20ed817e379f4e494aa53bfa44
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/51707
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
DNNMark is representative of several simple (fast) layers within ML
applications, which are heavily used in modern GPU applications. Thus,
we want to make sure support for these applications are tested. This
commit updates the weekly regression to run three variants: fwd_softmax,
bwd_bn, and fwd_pool -- ensuring we test both inference and training as
well as a variety of ML layers.
Change-Id: I38bfa9bd3a2817099ece46afc2d6132ce346e21a
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/51187
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
LULESH is a popular GPU HPC application that acts as a good test
for several memory and compute patterns. Thus, including it in
the weekly regressions will help verify correctness and
functionality for code that affects the GPU. The default LULESH
input runs 10 iterations and takes 3-4 hours. Hence, it is not
appropriate for nightly regressions.
Change-Id: Ic1b73ab32fdd5cb1b973f2676b272adb91b2a98e
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/50952
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
