Previously, when passing the -re option while using multisim, the files
simerr.txt and simout.txt would be redirected into the m5out directory
instead of the correct subdirectory. They would also have a name of the
format
Spawn_gem5PoolWorker-some-integer_(simout|simerr).txt, which doesn't
indicate which simulation the files correspond to.
This commit fixes these issues by redirecting simerr.txt and simout.txt
into the correct subdirectory.
Change-Id: I0a25a9fd8dc672949f5f85fc5ca6452529301a73
According to the pybind documentation, "When combining *args or **kwargs
with Keyword arguments you should not include py::arg tags for the
py::args and py::kwargs arguments."
In the current implementation of gem5, if you use the cxxMethod
decorator on a function that has *args or **kwargs, gem5 will
incorrectly add these variables to the pybind generated declaration.
I.e., def f(arg1, arg2, *args, **kwargs): -> .def("f", &f,
py::arg("arg1"), py::arg("arg2"), py::arg("*args"), py::arg("**kwargs"))
which is incorrect pybind code.
To fix this problem, we should ignore variables in the generator if they
are *args or **kwargs. This change skips these variables when creating
the pybind declaration.
Change-Id: I44a1e0eb0b5fc5c1e1d423ba145d456bff92c6b8
- Updated search query so that resources that are not compatible with
the gem5 version are still downloaded and used but a warning is thrown
instead of returning an error.
In `get_default_kernel_root_val()`, now prioiritizes
the explicit disk_device passed from the user over the
default implemented by the board.
Also adjusts syntax for selecting this value in
`set_kernel_disk_workload()` for consistency.
Change-Id: Icddcf438f5b96c2288c3cc608782f191df2c394e
Added documentation for `strided_generator.py` and
`strided_generator_core.py.`
Updated clarity of documentation for `linear_generator.py`,
`linear_generator_core.py`, `random_generator.py`, and
`random_generator_core.py`.
Made `max_addr` exclusive instead of inclusive for strided and linear
traffic generation in `strided_gen.cc` and `linear_gen.cc`.
This commit moves the requires() call that checks the cache coherence
protocol above the imports. This change was made for the chi private l1,
ruby mesi three level, mesi two level, and mi example cache hierarchies.
This ensures that a clear error message about having the wrong coherence
protocol is printed, rather than a less useful message.
Change-Id: I3bac1ffcb1f8a9d94e486237f880cf248e442ba8
- gem5 was querying the full version of gem5 that is `24.0.0.0` while
searching for resources.
This was causing an error to find resources on staging branch.
This change trims the gem5 version to be just the major.minor version.
Change-Id: I30c3a1b38c631981f797ef0fd2b616e6a66ca18e
This change adds a new utility function for processing Spatter traces
into SpatterKernels under parse_kernels.
Additionally, it adds documentation for all the utility functions in
spatter_kernel.py.
Lastly, it adds an example script for running one spatter trace using
SpatterGenerator to the examples.
This change extend the AbstractMemory class to add a getter method that
allows other components to get the memory's range without interleaving.
This method will be useful if other components in the system need to
interleave the memory range different to the way the memory has
interleaved them.
This change fixes the way indices are generated in a multi generator
setup.
It changes it from all cores generating the same trace of indices for
accessing the index array to each core generating an interleaved subset
of indices.
For an example look below for traces (indices to index array) in a 2
core setup.
Before:
core_0: 0, 1, 2, 3, 4, 5, 6, 7, ...
core_1: 0, 1, 2, 3, 4, 5, 6, 7, ...
After:
core_0: 0, 1, 2, 3, 8, 9, 10, 11, ...
core_1: 4, 5, 6, 7, 12, 13, 14, 15, ...
Additionally, this change fixes the SpatterKernel class in the standard
library to comply with the change in the SpatterGen source code.
Often, you want to add another argument to the default kernel arguments.
This function allows you to do that on the `kernel_disk_workload` board
mixin.
This allows for multiple gem5 simulations to be spawned from a single
parent gem5 process, as defined in a simgle gem5 configuration. In this
design _all_ the `Simulator`s are defined in the simulation script and
then added to the mutlisim module. For example:
```py
from gem5.simulate.Simulator import Simulator
import gem5.utils.multisim as multisim
# Construct the board[0] and board[1] as you wish here...
simulator1 = Simulator(board=board[0], id="board-1")
simulator2 = Simulator(board=board[1], id="board-2")
multisim.add_simulator(simulator1)
multisim.add_simulator(simulator2)
```
This specifies that two simulations are to be run in parallel in
seperate threads: one specified by `simulator1` and another by
`simulator2`. They are then added to MultiSim via the
`multisim.add_simulator` function. The user can specify an id via the
Simulator constructor. This is used to give each process a unique id and
output directory name. Given this, the id should be a helpful name
describing the simulation being specified. If not specified one is
automatically given.
To run these simulators we use `<gem5 binary> -m gem5.utils.multisim
<script> -p <num_processes>`. Note: multisim is an executable module in
gem5. This is the same module we input into our scripts to add the
simulators. This is an intentionally modular encapsulated design. When
the module processes a script it will schedule multiple gem5 jobs and,
dependent on the number of processes specified, will create child gem5
processes to processes tjese jobs (jobs are just gem5 simulations in
this case). The `--processes` (`-p`) argument is optional and if not
specified the max number of processes which can be run concurrently will
be the number of available threads on the host system.
The id for each process is used to create a subdirectory inside the
`outputdor` (`m5out`) of that id name. E.g, in the example above the
ID's are `board-1` and `board-2`. Therefore the m5 out directory will
look as follows:
```sh
- m5out
- board-1
- stats.txt
- config.ini
- config.json
- terminal.out
- board-2
- stats.txt
- config.ini
- config.json
- terminal.out
```
Each simulations output is encapsulated inside the subdirectory of the
id name.
If the multisim configuation script is passed directly to gem5 (like a
traditional gem5 configuraiton script, i.e.: `<gem5 binary> <script>`),
the user may run a single simulation specified in that script by passing
its id as an argument. E.g. `<gem5 binary> <script> board-1` will run
the `board-1` simulation specified in `script`. If no argument is passed
an Exception is raised asking the user to either specify or use the
MultiSim module if multiprocessing is needed.
If the user desires a list of ids of the simulations specified in a
given MultiSim script, they can do so by passing the `--list` (`-l`)
parameter to the config script. I.e., `<gem5 binary> <script> --list`
will list all the IDs for all the simulations specified in`script`.
This change comes with two new example scripts found in
'configs/example/gem5_library/multsim" to demonstrate multisim in both
an SE and FS mode simulation. Tests have been added which run these
scripts as part of gem5' Daily suite of tests.
Notes
=====
* **Bug fixed**: The `NoCache` classic cache hierarchy has been modified
so the Xbar is no longet set with a `__func__` call. This interfered
with MultiProcessing as this structure is not serializable via Pickle.
This was quite bad design anyway so should be changed
* **Change**: `readfile_contents` parameter previously wrote its value
to a file called "readfile" in the output dorectory. This has been
changed to write to a file called "readfile_{hash}" with "{hash}" being
a hash of the `readfile_contents`. This ensures that, during multisim
running, this file is not overwritten by other processes.
* **Removal note**: This implementation supercedes the functionality
outlined in 'src/python/gem5/utils/multiprocessing'. As such, this code
has been removed.
Limitations/Things to Fix/Improve
=================================
* Though each Simulator process has its own output directory (a
subdirectory within m5out, with an ID set by the user unique to that
Simulator), the stdout and stderr are still output to the terminal, not
the output directory. This results in: 1. stdout and stderr data lost
and not recorded for these runs. 2. An incredibly noisy terminal output.
* Each process uses the same cached resources. While there are locks on
resources when downloading, each processes will hash the resources they
require to ensure they are valid. This is very inefficient in cases
where resources are common between processes (e.g., you may have 10
processes each using the same disk image with each processes hashing the
disk images independently to give the same result to validate the
resources).
Change-Id: Ief5a3b765070c622d1f0de53ebd545c85a3f0eee
---------
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
This PR adds source code for C++ implementation of SpatterGen as well as
SpatterKernel. SpatterGen uses a PyBindMethod to add kernels to the
backend code. This way the process of processing json files could be
offloaded to python. In addition it adds standard library components for
SpatterGenCore and SpatterGen. These two components follow the same
structure as AbstractCore and AbstractProcessor. In addition
spatter_kernel.py adds a definition for SpatterKernel in python to make
adding kernels to C++ easier. Also it adds utility functions for parsing
dictionaries read from json as well as partitioning traces for multicore
setups.
This change adds code for SpatterGenCore and SpatterGen as well
as SpatterKernel to the standard library. SpatterGenCore and
SpatterGen follow the same structure as AbstractCore and
AbstractProcessor. spatter_kernel.py adds utility functions
to parse dictionaries as well as partition a list into
multiple lists through interleaving to be used when setting up
a multicore SpatterGen.
Change-Id: I003553e97f901c0724f5feac0bb6e21a020bd6ad
Bigs fixed of note:
1. The 'find' method has been fixed to work. This involved making
'children' a class implemented per-subclass as required.
2. The 'get_all_stats_of_name' method has been removed. This was not
working at all correctly and is largely doing what 'find' does.
2. The functionality to get an element in a vector via an attribute call
(i.e., self.vector1 == self.vector[1]) has been implemented this
maintaining backwards compatibility with the regular Python stats.
Change-Id: I31a4ccc723937018a3038dcdf491c82629ddbbb2
The SimStat Object is nothing more than a group of other SimStats and is
therefore logically a group. With this, functionality can be shared more
easily.
Change-Id: I5dce23a02d5871e640b422654ca063e590b1429a
1. Adds newly added PyStat classes to "__init__.py", ensuring they can
all be accessed via a `m5.ext.pystats` import.
2. Simplifies the layout out "__init__.py" to just import all classes
from all files.
Change-Id: I43bfc5e7ff1aec837e661905304c6fb10b00c90e
This commit addresses Jason's comment
(https://github.com/gem5/gem5/pull/996#discussion_r1613870880) which
highlighted putting the `_m5.stats.processDumpQueue` call in the
iteration through the `root` object in `get_simstat` caused this
function be potentially called many times when it only needs to be
called once. This chance moved this call to just before this iteration
and will tehrefore only be called once (if required) per `get_simstat`
execution.
Change-Id: I16908b6dee063a0df7877a19e215883963bfb081
As Distribution inherits from Vector, it should be constructed with
a Dictionary of scalars (in our implementation, a dictionary mapping the
vector position's unique id for each bin and the value of that bin).
Change-Id: Ie603c248e5db4b6dd7f71cc453eebd78793f69a3
The big thing missing from the Vector stats was that each position in
the vector could have it's own unique id (a str, float, or int) and each
position in the vector can have its own description. Therefore, to add
this the Vector is represented as a dictionary mapping the unique ID to
a Pystat Scaler (whcih can have it's own unique description.
Change-Id: I3a8634f43298f6491300cf5a4f9d25dee8101808
This isn't a true Base class, it's just a Vector. In gem5 all Vectors
are Scalar Vectors. This change simplfies the naming.
Change-Id: Ib8881d854ab18de6acbf0fb200db2de6a43621a7
The SIMPOINT_BEGIN should do nothing by default since it might be used
in various cases.
In
[https://www.mail-archive.com/gem5-users@gem5.org/msg22383.html](mailing
list), a user discovered a bug with the current
`simpoints-se-restore.py` example.
The bug is caused by the default behavior of the SIMPOINT_BEGIN exit
event.
When taking a checkpoint with `simpoints-se-checkpoint.py`, it stores
the future exit event scheduled at the beginning of the simulation. I
did not notice this when I wrote and tested the example script due to
the long print out log and my custom handler of the SIMPOINT_BEGIN exit
event.
In the restoring, the SIMPOINT_BEGIN exit event was triggered right
before the region end, so it resets the stats before the final stats
dump. Therefore, the simulation time is 0 as the user discovered.
This patch should fix this bug.
Change-Id: I800dfbd28d7b2c842864a1ab7d84b8f8e17b9b3c
- There was a bug in JSONClient when searching
for resoruces. The id was not checked and
the booleans were not set to true when
optional search queries like resource_version
and gem5_version are not passed.
Change-Id: I4aa7c5388035144ec6864d57130ad09e6709692e
This commit adds more detailed instruction types for RISC-V Vector.
Concretely, it substitutes VectorIntegerArith, VectorFloatArith,
VectorIntegerReduce and VectorFloatReduce with more specific types
related to the operation that each instruction (e.g., VectorIntegerAdd
or VectorIntegerMult).
Additionaly, fixes two RISC-V instruction types (VectorXXX) that were
used in ARM SVE, placing the proper SimdXXX ones.
Change-Id: I31774fa6a7cd249abfffec68d11d3d77f08ad70b
CC @adriaarmejach
One of things we miss in gem5 is the capability to neatly compose the
cache hierarchy of CPUs and clusters of CPUs. The BaseCPU
addPrivateSplitL1Caches and addTwoLevelCacheHierarchy APIs have
historically been used to bind cache levels together.
These APIs have been superseded by the introduction of the Cache
hierarchy abstraction in the standard library. The standard library
makes it cleaner for a user to quickly instantiate a hierarchy of caches
with few lines of code. While this removes a lot of complexity for a
user, the Hierarchy objects still have little information about their
internal topology.
To address this problem, this patch adds a tree data structure to the
AbstractCacheHierarchy class, where every node of the tree represent
a cache in the hierarchy. In this way we will expose APIs for traversing
and querying the tree.
For example a 2 CPUs system with private L1, private L2 and shared L3
will contain the following tree:
[root]
|
[L3]
/\
/ \
[L2] [L2]
| |
[L1] [L1]
When the `statistics::nozero` flag is set gem5 does not output that stat
if its value is zero. This was not the case for SimStats which output in
this case. This patch correct this behavior.
This will just call the _m5.range.isSubset method
Change-Id: If747819a008a8ed20796b4efd42a42e5c3a8d7d9
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
This change sets the `release` of the ARM board at the config file
instead of overriding the release on the ArmBoard. This change partially
solves issue 932 as the system taking and restoring the checkpoint is
consistent across KVM and timing CPUs respectively.
Signed-off-by: Kaustav Goswami <kggoswami@ucdavis.edu>
Now that we are able to provide a view of the cache hierarchy from
the python world, we can start generating DTB entries for caches
and more specifically to properly fill the next-level-cache and
cache-level properties
Change-Id: Iba9ea08fe605f77a353c9e64d62b04b80478b4e2
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
One of things we miss in gem5 is the capability to neatly compose the
cache hierarchy of CPUs and clusters of CPUs. The BaseCPU
addPrivateSplitL1Caches and addTwoLevelCacheHierarchy APIs have
historically been used to bind cache levels together.
These APIs have been superseeded by the introduction of the Cache
hierarchy abstraction in the standard library. The standard library
makes it cleaner for a user to quickly instantiate a hierarchy of caches
with few lines of code. While this removes a lot of complexity for a
user, the Hierarchy objects still have little information about their
internal topology.
To address this problem, this patch adds a tree data structure to the
AbstractCacheHierarchy class, where every node of the tree represent
a cache in the hierarchy. In this way we will expose APIs for traversing
and querying the tree.
For example a 2 CPUs system with private L1, private L2 and shared L3
will contain the following tree:
[root]
|
[L3]
/\
/ \
[L2] [L2]
| |
[L1] [L1]
Change-Id: I78fe6ad094f0938ff9bed191fb10b9e841418692
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
This commit updates cpu by removing VectorXXX types and updates
FUs according to the newer SimdXXX ones. This is part of the
homogenization of RISCV Vector instruction types, which moved
from VectorXXX to SimdXXX.
Change-Id: I84baccd099b73a11cf26dd714487a9f272671d3d
This PR fixes#948 in which running KVM CPUs through the updated gem5
interface in SE mode causes an immediate crash.
To fix this, I added a check to set_se_binary_workload that checks if
any of the cores are KVM, and if so, sets a couple of knobs for the
board and process that are required to make KVM work. The depecated
se.py script, which sets these knobs, is able to run KVM in SE mode just
fine, so doing the same here fixed the bug.
