This commit contains the rest of the base 2 vs base 10 cache/memory
size clarifications. It also changes the warning message to use
warn(). With these changes, the warning message should now no
longer show up during a fresh compilation of gem5.
Change-Id: Ia63f841bdf045b76473437f41548fab27dc19631
This allows for multiple gem5 simulations to be spawned from a single
parent gem5 process, as defined in a simgle gem5 configuration. In this
design _all_ the `Simulator`s are defined in the simulation script and
then added to the mutlisim module. For example:
```py
from gem5.simulate.Simulator import Simulator
import gem5.utils.multisim as multisim
# Construct the board[0] and board[1] as you wish here...
simulator1 = Simulator(board=board[0], id="board-1")
simulator2 = Simulator(board=board[1], id="board-2")
multisim.add_simulator(simulator1)
multisim.add_simulator(simulator2)
```
This specifies that two simulations are to be run in parallel in
seperate threads: one specified by `simulator1` and another by
`simulator2`. They are then added to MultiSim via the
`multisim.add_simulator` function. The user can specify an id via the
Simulator constructor. This is used to give each process a unique id and
output directory name. Given this, the id should be a helpful name
describing the simulation being specified. If not specified one is
automatically given.
To run these simulators we use `<gem5 binary> -m gem5.utils.multisim
<script> -p <num_processes>`. Note: multisim is an executable module in
gem5. This is the same module we input into our scripts to add the
simulators. This is an intentionally modular encapsulated design. When
the module processes a script it will schedule multiple gem5 jobs and,
dependent on the number of processes specified, will create child gem5
processes to processes tjese jobs (jobs are just gem5 simulations in
this case). The `--processes` (`-p`) argument is optional and if not
specified the max number of processes which can be run concurrently will
be the number of available threads on the host system.
The id for each process is used to create a subdirectory inside the
`outputdor` (`m5out`) of that id name. E.g, in the example above the
ID's are `board-1` and `board-2`. Therefore the m5 out directory will
look as follows:
```sh
- m5out
- board-1
- stats.txt
- config.ini
- config.json
- terminal.out
- board-2
- stats.txt
- config.ini
- config.json
- terminal.out
```
Each simulations output is encapsulated inside the subdirectory of the
id name.
If the multisim configuation script is passed directly to gem5 (like a
traditional gem5 configuraiton script, i.e.: `<gem5 binary> <script>`),
the user may run a single simulation specified in that script by passing
its id as an argument. E.g. `<gem5 binary> <script> board-1` will run
the `board-1` simulation specified in `script`. If no argument is passed
an Exception is raised asking the user to either specify or use the
MultiSim module if multiprocessing is needed.
If the user desires a list of ids of the simulations specified in a
given MultiSim script, they can do so by passing the `--list` (`-l`)
parameter to the config script. I.e., `<gem5 binary> <script> --list`
will list all the IDs for all the simulations specified in`script`.
This change comes with two new example scripts found in
'configs/example/gem5_library/multsim" to demonstrate multisim in both
an SE and FS mode simulation. Tests have been added which run these
scripts as part of gem5' Daily suite of tests.
Notes
=====
* **Bug fixed**: The `NoCache` classic cache hierarchy has been modified
so the Xbar is no longet set with a `__func__` call. This interfered
with MultiProcessing as this structure is not serializable via Pickle.
This was quite bad design anyway so should be changed
* **Change**: `readfile_contents` parameter previously wrote its value
to a file called "readfile" in the output dorectory. This has been
changed to write to a file called "readfile_{hash}" with "{hash}" being
a hash of the `readfile_contents`. This ensures that, during multisim
running, this file is not overwritten by other processes.
* **Removal note**: This implementation supercedes the functionality
outlined in 'src/python/gem5/utils/multiprocessing'. As such, this code
has been removed.
Limitations/Things to Fix/Improve
=================================
* Though each Simulator process has its own output directory (a
subdirectory within m5out, with an ID set by the user unique to that
Simulator), the stdout and stderr are still output to the terminal, not
the output directory. This results in: 1. stdout and stderr data lost
and not recorded for these runs. 2. An incredibly noisy terminal output.
* Each process uses the same cached resources. While there are locks on
resources when downloading, each processes will hash the resources they
require to ensure they are valid. This is very inefficient in cases
where resources are common between processes (e.g., you may have 10
processes each using the same disk image with each processes hashing the
disk images independently to give the same result to validate the
resources).
Change-Id: Ief5a3b765070c622d1f0de53ebd545c85a3f0eee
---------
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
As X86 and RISCV are relying on a Table Walker cache, we
change their stdlib configs to use the newly defined
PrivateL1PrivateL2WalkCacheHierarchy
Change-Id: I63c3f70a9daa3b2c7a8306e51af8065bf1bea92b
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
This `mkdir` is problematic as it doesn't create the directory
recursively. This casues errors if `dir` is `X/Y/Z` and both `Y` and `Z`
has not been created. An error will be returned (`No such file or
directory`).
This issue was fixed with: https://github.com/gem5/gem5/pull/263. The
checkpointing code already recursively creates directories as needed.
Ergo was can remove this `mkdir` statement.
Change-Id: Ibae38267c8ee1eba76d7834367aa1c54013365bc
* stdlib,configs,tests: Remove `Resource` class use
This class is deprecated, but was still used in various example
configuration scriots and tests. This patch replaces it with the
`obtain_resource` function.
Change-Id: I0c89bf17783ccaaafc18072aaeefb5d1e207bc55
* configs: Remove `CustomDiskImageResource` use
The class is deprecated but was still used in the SPEC example scripts.
This patch replaces it with the `DiskImageResource` class.
Change-Id: Ie0697fe59a3d737b05eb45ff3bc964f42b0387e0
* configs,tests: Remove `CustomResource` use
This class is deprecated but was still used in example scripts and
mentioned, incorrectly, in comments in the pyunit tests. This patch
removes these.
Change-Id: Icb6d02f47a5b72cd58551e5dcd59cc72d6a91a01
* stdlib: Remove '\' in Workload docstring example
This example shows how to use the Workload. The backslash is not correct Python and would fail if used in this way.
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
---------
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
This patches does the following:
- Adds 'SimpointResource' which encapsulates Simpoint data and
functionality. It replaces the old 'gem5.util.simpoint.SimPoint'
class. Simpoints can be loaded from gem5-resources using the
`obtain_resource` function.
- Adds 'SimpointDirectoryResource'. This inherits form
'SimpointResource'. While 'SimpointResource' takes raw Simpoint data
via parameters, 'SimpointDirectoryResource' assumes the data exists
in files, in a directory.
- Updates the
"configs/example/gem5_library/checkpoints/simpoints-se-checkpoint.py"
and
"configs/example/gem5_library/checkpoints/simpoints-se-restory.py"
example files to utilize this new Simpoint resource classes.
**Note**: While the old "SimPoint" class
("src/python/gem5/util/simpoint.py") is marked as deprecated, it may be
difficult to utilize given updates to the APIs in the gem5 stdlib Cores
and Simulator modules.
Change-Id: I9bed5c643ffc735838c9f22a58c53547941010e7
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67339
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
This patch:
- Makes this function private.
- Updates the function's documentation.
- Changes the 'init' parameter to 'board_initialized'.
It doesn't make much sense for this function to be exposed directly to
the user as it requires knowing whether the board is initialized or not.
In addition to this I believe it makes more sense for the 'init' logic
to be flipped and renamed "board_initialized' so that this value is True
if the board has been initialized.
The documentation for this function has been updated.
Change-Id: I016c65bde88357111d3e648d7aa99aeb6e31f410
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64833
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
It never made much sense to set checkpoint via the Simulator module as
Checkpoints are very tightly coupled with the Workload being run. This
change therefore moves the checkpoint to the set_workload functions.
Setting checkpoints via the Simulator is deprecated and will be removed
in a future release.
Change-Id: I24d2133b38a86423d3553ec888c917c5fe47b93d
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64571
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
The
'configs/example/gem5_library_checkpoints/simpoints-se-checkpoint.py'
example would dump the savepoint in the CWD. This is fine when running
as an example, but we also run this script as a test. In this case the
checkpoint litters the repository.
To fix this, an optional 'checkpoint-path' argument is added to this
example which specifies where the checkpoint is to be saved. In the
tests, the checkpoint is saved to 'tests/gem5/resources'. This is our
default location for resources needed/produced by tests and is ignored
by git.
Change-Id: I3dccc574b9e64d32386fd822ed7248ee365a0a08
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64092
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The
'configs/example/gem5_library_example_tests/test_gem5_library_examples.py'
example would dump the savepoint in the CWD. This is fine when running
as an example, but we also run this script as a test. In this case the
checkpoint litters the repository.
To fix this, an optional 'checkpoint-path' argument is added to this
example which specifies where the checkpoint is to be saved. In the
tests, the checkpoint is saved to 'tests/gem5/resources'. This is our
default location for resources needed/produced by tests and is ignored
by git.
Change-Id: Ib985433786c99c37794a1c67cc4337a5dfd0498d
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64091
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
simpoints-se-checkpoint.py & simpoints-se-restore.py:
These are two example scripts to show how to use SimPoints functions with
the stdlib.
se_binary_workload.py:
Allow se_binary_workload to take in SimPoint Class item and schedule
SimPoint exit events.
exit_event.py:
Added SIMPOINT_BEGIN and MAX_INSTS exit events.
simulator.py:
Added SIMPOINT_BEGIN and MAX_INSTS exit event scheduling functions.
They can schedule exit events before or during the simulation.
Jira Issue: https://gem5.atlassian.net/browse/GEM5-1259
Change-Id: Iaa07a83de9dddc293b9f1a230aba8e35d4f5af6c
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63154
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
This change consists of two scripts,
- riscv-hello-save-checkpoint.py: runs the first million ticks of the
simulation and save a checkpoint.
- riscv-hello-load-checkpoint.py: loads the above checkpoint, and runs
the rest of the simulation.
This change also adds the two scripts as part of quick tests.
Change-Id: I7bd97ba953fab52f298cbbcf213f2ea5c185cc38
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/58829
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
