This commit contains the rest of the base 2 vs base 10 cache/memory
size clarifications. It also changes the warning message to use
warn(). With these changes, the warning message should now no
longer show up during a fresh compilation of gem5.
Change-Id: Ia63f841bdf045b76473437f41548fab27dc19631
This change adds a new utility function for processing Spatter traces
into SpatterKernels under parse_kernels.
Additionally, it adds documentation for all the utility functions in
spatter_kernel.py.
Lastly, it adds an example script for running one spatter trace using
SpatterGenerator to the examples.
This allows for multiple gem5 simulations to be spawned from a single
parent gem5 process, as defined in a simgle gem5 configuration. In this
design _all_ the `Simulator`s are defined in the simulation script and
then added to the mutlisim module. For example:
```py
from gem5.simulate.Simulator import Simulator
import gem5.utils.multisim as multisim
# Construct the board[0] and board[1] as you wish here...
simulator1 = Simulator(board=board[0], id="board-1")
simulator2 = Simulator(board=board[1], id="board-2")
multisim.add_simulator(simulator1)
multisim.add_simulator(simulator2)
```
This specifies that two simulations are to be run in parallel in
seperate threads: one specified by `simulator1` and another by
`simulator2`. They are then added to MultiSim via the
`multisim.add_simulator` function. The user can specify an id via the
Simulator constructor. This is used to give each process a unique id and
output directory name. Given this, the id should be a helpful name
describing the simulation being specified. If not specified one is
automatically given.
To run these simulators we use `<gem5 binary> -m gem5.utils.multisim
<script> -p <num_processes>`. Note: multisim is an executable module in
gem5. This is the same module we input into our scripts to add the
simulators. This is an intentionally modular encapsulated design. When
the module processes a script it will schedule multiple gem5 jobs and,
dependent on the number of processes specified, will create child gem5
processes to processes tjese jobs (jobs are just gem5 simulations in
this case). The `--processes` (`-p`) argument is optional and if not
specified the max number of processes which can be run concurrently will
be the number of available threads on the host system.
The id for each process is used to create a subdirectory inside the
`outputdor` (`m5out`) of that id name. E.g, in the example above the
ID's are `board-1` and `board-2`. Therefore the m5 out directory will
look as follows:
```sh
- m5out
- board-1
- stats.txt
- config.ini
- config.json
- terminal.out
- board-2
- stats.txt
- config.ini
- config.json
- terminal.out
```
Each simulations output is encapsulated inside the subdirectory of the
id name.
If the multisim configuation script is passed directly to gem5 (like a
traditional gem5 configuraiton script, i.e.: `<gem5 binary> <script>`),
the user may run a single simulation specified in that script by passing
its id as an argument. E.g. `<gem5 binary> <script> board-1` will run
the `board-1` simulation specified in `script`. If no argument is passed
an Exception is raised asking the user to either specify or use the
MultiSim module if multiprocessing is needed.
If the user desires a list of ids of the simulations specified in a
given MultiSim script, they can do so by passing the `--list` (`-l`)
parameter to the config script. I.e., `<gem5 binary> <script> --list`
will list all the IDs for all the simulations specified in`script`.
This change comes with two new example scripts found in
'configs/example/gem5_library/multsim" to demonstrate multisim in both
an SE and FS mode simulation. Tests have been added which run these
scripts as part of gem5' Daily suite of tests.
Notes
=====
* **Bug fixed**: The `NoCache` classic cache hierarchy has been modified
so the Xbar is no longet set with a `__func__` call. This interfered
with MultiProcessing as this structure is not serializable via Pickle.
This was quite bad design anyway so should be changed
* **Change**: `readfile_contents` parameter previously wrote its value
to a file called "readfile" in the output dorectory. This has been
changed to write to a file called "readfile_{hash}" with "{hash}" being
a hash of the `readfile_contents`. This ensures that, during multisim
running, this file is not overwritten by other processes.
* **Removal note**: This implementation supercedes the functionality
outlined in 'src/python/gem5/utils/multiprocessing'. As such, this code
has been removed.
Limitations/Things to Fix/Improve
=================================
* Though each Simulator process has its own output directory (a
subdirectory within m5out, with an ID set by the user unique to that
Simulator), the stdout and stderr are still output to the terminal, not
the output directory. This results in: 1. stdout and stderr data lost
and not recorded for these runs. 2. An incredibly noisy terminal output.
* Each process uses the same cached resources. While there are locks on
resources when downloading, each processes will hash the resources they
require to ensure they are valid. This is very inefficient in cases
where resources are common between processes (e.g., you may have 10
processes each using the same disk image with each processes hashing the
disk images independently to give the same result to validate the
resources).
Change-Id: Ief5a3b765070c622d1f0de53ebd545c85a3f0eee
---------
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
This change sets the `release` of the ARM board at the config file
instead of overriding the release on the ArmBoard. This change partially
solves issue 932 as the system taking and restoring the checkpoint is
consistent across KVM and timing CPUs respectively.
Signed-off-by: Kaustav Goswami <kggoswami@ucdavis.edu>
As X86 and RISCV are relying on a Table Walker cache, we
change their stdlib configs to use the newly defined
PrivateL1PrivateL2WalkCacheHierarchy
Change-Id: I63c3f70a9daa3b2c7a8306e51af8065bf1bea92b
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
This change adds support to use KVM cores on the ARM board. The board
simulates gic to enable KVM, similar to the gem5 ARM FS configs. The
limitation is that it only supports VExpress_GEM5_V1.
Signed-off-by: Kaustav Goswami <kggoswami@ucdavis.edu>
- updated the x86-npb-benchmarks.py to use npb workloads and suites.
The suites and workloads are not in the database are also waiting
feedback. I am attaching the JSON file here.
[npb_workloads_suite.json](https://github.com/gem5/gem5/files/13431116/npb_workloads_suite.json)
To run the x86-npb-benchmarks.py script use the
GEM5_RESOURCE_JSON_APPEND env variable. The full command is:
```
GEM5_RESOURCE_JSON_APPEND=[path to npb_workloads_suite.json] ./build/X86/gem5.opt configs/example/gem5_library/x86-npb-benchmarks.py --benchmark [benchmark]
```
Change-Id: I248e6452ea4122e9260e34e4368847660edae577
Recent breaking changes in the DRAMSys API require user code to be
updated. These updates have been applied to the gem5 integration.
Furthermore, as DRAMSys started to use CMake dependency management,
it is no longer sensible to maintain two separate build systems for
DRAMSys. The use of the DRAMSys integration in gem5 will therefore
from now on require that CMake is installed on the target machine.
Additionally, support for snapshots have been implemented into DRAMSys
and coupled with gem5's checkpointing API.
This `mkdir` is problematic as it doesn't create the directory
recursively. This casues errors if `dir` is `X/Y/Z` and both `Y` and `Z`
has not been created. An error will be returned (`No such file or
directory`).
This issue was fixed with: https://github.com/gem5/gem5/pull/263. The
checkpointing code already recursively creates directories as needed.
Ergo was can remove this `mkdir` statement.
Change-Id: Ibae38267c8ee1eba76d7834367aa1c54013365bc
This patch introduces a new category called "suite".
A suite is a collection of workloads.
Each workload in a SuiteResource has a tag that can be narrowed down
through the function with_input_group.
Also, the set of input groups can be seen through list_input_groups.
Added unit tests to test all functions of SuiteResource class.
Change-Id: Iddda5c898b32b7cd874987dbe694ac09aa231f08
Co-authored-by: Kunal Pai <kunpai@ucdavis.edu>
- Added deprecated warnings to Workload and Abstract workload.
- Added comments to the classes changed.
Change-Id: I671daacf5ef455ea65103bd96aa442486142a486
- Added WrokloadResource in resource.py.
- depricated Workload and CustomWorkload.
- changed iscvmatched-fs.py with obtain resource for workload to test.
Change-Id: I2267c44249b96ca37da3890bf630e0d15c7335ed
Note: change example files back to original
* stdlib,configs,tests: Remove `Resource` class use
This class is deprecated, but was still used in various example
configuration scriots and tests. This patch replaces it with the
`obtain_resource` function.
Change-Id: I0c89bf17783ccaaafc18072aaeefb5d1e207bc55
* configs: Remove `CustomDiskImageResource` use
The class is deprecated but was still used in the SPEC example scripts.
This patch replaces it with the `DiskImageResource` class.
Change-Id: Ie0697fe59a3d737b05eb45ff3bc964f42b0387e0
* configs,tests: Remove `CustomResource` use
This class is deprecated but was still used in example scripts and
mentioned, incorrectly, in comments in the pyunit tests. This patch
removes these.
Change-Id: Icb6d02f47a5b72cd58551e5dcd59cc72d6a91a01
* stdlib: Remove '\' in Workload docstring example
This example shows how to use the Workload. The backslash is not correct Python and would fail if used in this way.
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
---------
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
* cpu-kvm: Add a variable signifying whether we are using perf
Change-Id: Iaa081e364f85c863f781723b5524d267724ed0e4
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
* cpu-kvm: Making it clear the functionalities are specific to KVM
Change-Id: I982426f294d90655227dc15337bf73c42a260ded
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
* cpu-kvm: Make perf optional
Change-Id: I8973c2a96575383976cea7ca3fda478f83e95c3f
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
* configs: Add an example config of using KVM without perf
Change-Id: Ic69fa7dac4f1a2c8fe23712b0fa77b5b22c5f2df
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
* Apply suggestions from code review
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
* misc: Add an example to the panic
Change-Id: Ic1fdfb955e5d8b9ad1d4f0a2bf30fa8050deba70
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
* misc: Add warning of not using perf when using KVM CPU
Change-Id: I96c0832fb48c63a79773665ca6228da778ef0497
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
* misc: Fix stuff
Change-Id: Ib407ae7407955b695f0e0f2718324f41bb0d768f
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
* misc: style fix
Change-Id: I7275942e43f46140fdd52c975f76abb3c81b8b0a
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
---------
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
I modified the spec-2017 benchmark file to include a
processor.switch() statement (which had been removed
in an earlier commit). I also replaced the use of
get_roi_ticks() with get_tick_stopwatch() as this
particular workload is not annotated with "workbegin"
and "workend" annotations.
Lastly, I fixed a minor bug in the spec-2006 file
which printed the total simulated ticks as zero due
to incorrect indexing of get_tick_stopwatch().
Change-Id: If0b6f88d6701a11f32e2e69994582524e0d7097b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/69537
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Ayaz Akram <yazakram@ucdavis.edu>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
The call to processor switch from KVM to TIMING was
removed in an earlier commit. This change fixes that.
Also, get_roi_ticks() doesn't work because spec2006
does not have work_begin and work_exit annotations.
This change uses get_tick_stopwatch() to calculate
the roi ticks.
Change-Id: I55efe28ebd686cb4e6c88a528533127fb73c88ed
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/69357
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Add DRAMSys as a new AbstractMemorySystem to the gem5 stdlib.
Also, provide convenient subclasses with predefined DRAMSys
configurations.
Add two new stdlib examples:
- dramsys-traffic.py: Demonstrates the usage of DRAMSys
using the stdlib TrafficGenerators
- arm-hello-dramsys.py: A variant of the arm-hello.py
script that uses DRAMSys as it's memory.
These DRAMSys memory components are only compiled into the standard
library if DRAMSys is not compiled into gem5.
Change-Id: I9db87c41fbd9c28bc44e9d6bde13fc225dc16be9
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/62914
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
This change refactors the Looppoint files. While functionally
equivalent, this classes have been moved and altered to be easier to
handle going forward. The following changes have been made:
- New classes have been added to represent the data structure of the
Looppoint JSON. This simplifies the parsing of JSON files and makes it
handle Looppoint data structures. Ultimately this is hidden from the
user via the new 'gem5.resources.Looppoint' class which will be the
front-facing class for Looppoint interactions.
- The `LooppointCheckpoint` class has been replaced with
`LooppointCsvLoader`. This new class takes in a CSV pintpoints file
to load necessary looppoint data.
- The `LoopPointRestore` class has been replaced by
`LooppointJsonLoader`.
- All Looppoint classes have been moved to `gem5.resources`. This will
make it easier when we add Looppoints as specific gem5 resources.
Change-Id: I11dd1fe8f76658db220320584270d57cb37a3c62
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67611
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
This patches does the following:
- Adds 'SimpointResource' which encapsulates Simpoint data and
functionality. It replaces the old 'gem5.util.simpoint.SimPoint'
class. Simpoints can be loaded from gem5-resources using the
`obtain_resource` function.
- Adds 'SimpointDirectoryResource'. This inherits form
'SimpointResource'. While 'SimpointResource' takes raw Simpoint data
via parameters, 'SimpointDirectoryResource' assumes the data exists
in files, in a directory.
- Updates the
"configs/example/gem5_library/checkpoints/simpoints-se-checkpoint.py"
and
"configs/example/gem5_library/checkpoints/simpoints-se-restory.py"
example files to utilize this new Simpoint resource classes.
**Note**: While the old "SimPoint" class
("src/python/gem5/util/simpoint.py") is marked as deprecated, it may be
difficult to utilize given updates to the APIs in the gem5 stdlib Cores
and Simulator modules.
Change-Id: I9bed5c643ffc735838c9f22a58c53547941010e7
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67339
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
The "test-gem5-library-example-riscvmatched-fs" test, which runs
"configs/example/gem5_library/riscvmatched-fs.py", was running the
script in full. This takes a very long time. Given we already have boot
tests for RISCV, it's better to just run this configuration to just the
end of the Linux boot (significantly faster than a full OS boot). This
patch adds this feature to the config script and modifies the test to
utilize it.
Change-Id: I1e37a26aab5e9a127ebd64590be79fbc16fe53aa
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65853
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
This patch:
- Makes this function private.
- Updates the function's documentation.
- Changes the 'init' parameter to 'board_initialized'.
It doesn't make much sense for this function to be exposed directly to
the user as it requires knowing whether the board is initialized or not.
In addition to this I believe it makes more sense for the 'init' logic
to be flipped and renamed "board_initialized' so that this value is True
if the board has been initialized.
The documentation for this function has been updated.
Change-Id: I016c65bde88357111d3e648d7aa99aeb6e31f410
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64833
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
The "config/example/gem5_library/arm-ubuntu-boot-exit.py" script is
renamed to "config/example/gem5_library/arm-ubuntu-run.py". This makes
it more consistent with similar scripts in the
"config/example/gem5_library" directory: "x86-ubuntu-run.py" and
"riscv-ubuntu-run.py".
Change-Id: I9d96fd68e122f2841573b1717b0969cd44972771
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65132
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
