Commit Graph

1123 Commits

Author SHA1 Message Date
Harshil Patel
452a600c49 New function to kernel_disk_workload to allow new disk device location (#151)
Added a parameter (_disk_device) to kernel_disk_workload which allows
users to change the disk device location. get_disk_device() now chooses
between the parameter and, if no parameter was passed, it calls a new
function _get_default_disk_device() which is implemented by each board
and has a default disk device according to each board, eg /dev/hda in
the x86_board. The previous way of setting a disk device still exists as
a default, however, with the new function users can now override this
default
2023-10-09 10:33:45 -07:00
Harshil Patel
79f40ffdab stdlib: Del comment stating SE mode limited to single thread (#402)
This comment was left in the codebase in error. The
`set_se_binary_workload` function works fine with multi-threaded
applications. This hasn't been a restriction for some time.
2023-10-09 10:30:32 -07:00
David Schall
edf9092fee cpu: Restructure BTB
- A new abstract BTB class is created to enable different BTB
  implementations. The new BTB class gets its own parameter
  and stats.
- An enum is added to differentiate branch instruction types.
  This enum is used to enhance statistics and BPU management.
- The existing BTB is moved into `simple_btb` as default.
- An additional function is added to store the static instruction in
  the BTB. This function is used for the decoupled front-end.
- Update configs to match new BTB parameters.

Change-Id: I99b29a19a1b57e59ea2b188ed7d62a8b79426529
Signed-off-by: David Schall <david.schall@ed.ac.uk>
2023-10-09 14:37:47 +00:00
Hoa Nguyen
6f8b74ece8 dev,arch-riscv: Mark gem5's 8250 UART as 16550a compatible
8250 UART is supposed to be compatible to 16550a UART.

This enables OpenSBI to print things to UART as OpenSBI only
prints if the UART is 16550a compatible [1].

There is a similar change from gem5 gerrit [2] pointing out
that this also enables bbl to print things to UART. This is
confirmed :)

[1] https://github.com/riscv-software-src/opensbi/blob/v1.3.1/lib/utils/serial/fdt_serial_uart8250.c#L29
[2] https://gem5-review.googlesource.com/c/public/gem5/+/68481

Signed-off-by: Hoa Nguyen <hn@hnpl.org>
2023-10-06 00:48:12 -07:00
Hoa Nguyen
e8fd8303fb stdlib: Add chosen node to the device tree of RISC-V board
This enables two things,
- /chosen/stdout-path is now default to uart@10000000, meaning
the linux kernel's boot console will be redirected to uart.
- /chosen/bootargs now contains the boot arguments obtained from
gem5's library. This allows passing the boot arguments to the
linux kernel via the device tree.

Change-Id: I53821d85f619e6276da85f41c972c041eaaf3280
Signed-off-by: Hoa Nguyen <hn@hnpl.org>
2023-10-06 00:33:45 -07:00
Bobby R. Bruce
4db748a507 resources, stdlib: Adding 'suite' category to gem5 (#191) 2023-10-05 13:26:58 -07:00
Bobby R. Bruce
f75c0fca8a stdlib: Del comment stating SE mode limited to single thread
This comment was left in the codebase in error. The
`set_se_binary_workload` function works fine with multi-threaded
applications. This hasn't been a restriction for some time.

Change-Id: I1b1d27c86f8d9284659f62ae27d752bf5325e31b
2023-10-05 10:20:55 -07:00
Leo Redivo
98a6cd6ee2 misc: changed call get_default_disk_device to get_disk_device
Change-Id: I240da78a658208211ede6648547dfa4c971074a1
2023-10-04 13:32:35 -07:00
Harshil Patel
3af3c1121b stdlib, resources: Addressed requested changes
Change-Id: I22abdc3bdcdde52301ed10cb3113e8925159c245
Co-authored-by: Kunal Pai <kunpai@users.noreply.github.com>
2023-10-02 23:27:32 -07:00
Harshil Patel
7301d4bd19 python: Add importer to standalone gem5py_m5 (#369)
I believe the point of this binary was to allow people to use the m5
objects without the entire gem5 binary. However, without adding the
importer call, this did not work. Unfortunately, with the importer call
there is a circular dependence on the original gem5py.cc file.
Therefore, this change creates a new file that has the importer call.

Now, with the `gem5py_m5` binary you can run python code that references
modules in `src/python`. Note that `_m5` is not available, so anything
that depends on the gem5 SimObjects' implementation will not work.
However, this can still be useful for things like getting Resources,
processing stats, etc.
2023-10-02 14:28:45 -07:00
Harshil Patel
f9781af6e5 mem: fix bug in 3-level cache (#265)
The L3 cache did not work due to argument type mismatch in the call to
the constructor `DMAController`. The second argument is expecting a
`RubySystem` type but the code passes in a `cache_line_size` variable.
After I change the second argument to `self.ruby_system` everything
works.
2023-09-29 10:59:18 -07:00
Harshil Patel
8182f8084b stdlib, resources, tests: Introduce Suite of Workloads
This patch introduces a new category called "suite".
A suite is a collection of workloads.
Each workload in a SuiteResource has a tag that can be narrowed down
through the function with_input_group.
Also, the set of input groups can be seen through list_input_groups.
Added unit tests to test all functions of SuiteResource class.

Change-Id: Iddda5c898b32b7cd874987dbe694ac09aa231f08

Co-authored-by: Kunal Pai <kunpai@ucdavis.edu>
2023-09-29 10:50:09 -07:00
Jason Lowe-Power
aaad79cf51 python: Add importer to standalone gem5py_m5
I believe the point of this binary was to allow people to use the m5
objects without the entire gem5 binary. However, without adding the
importer call, this did not work. Unfortunately, with the importer call
there is a circular dependence on the original gem5py.cc file.
Therefore, this change creates a new file that has the importer call.

Now, with the `gem5py_m5` binary you can run python code that references
modules in `src/python`. Note that `_m5` is not available, so anything
that depends on the gem5 SimObjects' implementation will not work.
However, thic can still be useful for things like getting Resources,
processing stats, etc.

Change-Id: I5c0e5d1a669fe5ce491458df916f2049c81292eb
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
2023-09-28 11:22:33 -07:00
Bobby R. Bruce
62d34ef374 misc: 'sim{out/err}' -> 'sim{out/err}.txt' (#250)
By default, the `--stderr-file` and `--stdout-file` arguments were
directing the simulator to output files named "simerr" and "simout"
respectively if an output redirect was requested.

A small annoyance is these files lack an extension meaning programs
refuse to open them, or don't do so withou additional effort. On many
systems they are assumed to scripts.

This patch adds the `.txt` extension to both, thus clearly indicating to
other programs these are text files and can be opened and read as such.
2023-09-27 17:36:03 -07:00
Bobby R. Bruce
5d254ffb02 stdlib, resources: Added pretty printing resource (#323)
- Implemented a __str__ for AbstractResource __str__ prints resource
category, id and version.
link to resources website is also printed.
2023-09-27 17:32:35 -07:00
Bobby R. Bruce
074fa4c604 misc,ext,tests: Automatically split CI TestLib tests across GitHub Action jobs (#263)
This PR utilizes GitHub Action's matrix's to automatically distribute
the CI testlib gem5 build and test jobs across available GitHub Action
Runners.

The CI tests (the `quick` testlib tests, i.e. those run with `./main.py
run`) are distributed across the runners on a per directory basis ---
all directories under "tests/gem5" are run as their own jobs.

The necessary gem5 builds for each workflow are now automatically
inferred via the introduction of `./main.py list`'s `--build-targets`
flag which returns the gem5 build target for a given test or collection
of tests. E.g., `./main.py list --build-targets` will return the build
targets for all the `quick` testlib tests and `./main.py list
--build-target --uid=<id>` will return the build targets the test suite
`<id>` requires.

Moving from monolithic jobs to fine-grained ones will make the locaiton
of test failures more obvious. Each job has it's own artifact containing
"test/testing-results" for the tests run in that job. In addition,
maintenance of these files should become less burdensome due to less
hardcoding.
2023-09-27 14:32:16 -07:00
Harshil Patel
633bdc08f2 stdlib: Addressed requested changes
- Added mulitline string for print message

- Added get_category_name method instead of having category as variable

Change-Id: I51e0e14a70e802453c21070711b200bc47994ba3
2023-09-27 11:36:51 -07:00
Bobby R. Bruce
391f62b213 misc: 'sim{out/err}' -> 'sim{out/err}.txt'
By default, the --stderr-file and --stdout-file arguments were
directing the simulator output to files named "simerr" and
"simout" respectively if an output redirect was requested.

A small annoyance is these files lack an extension meaning programs
refuse to open them, or to do so without some additional effort. On
many systems they are assumed to scripts.

This patch adds the .txt extension to both, thus clearly indicating
to other programs these are text files and can be opened to be read
as such.

Change-Id: Iff5af4a9e6966b4467d005a029dbf401099fbd35
2023-09-21 12:57:43 -07:00
Bobby R. Bruce
3f9afe96c6 python,util: Add Python MyPy Stubgen to enable Pylance IntelliSense (#307)
This allows us to generate stubs for the modules in gem5. The output
will be a "typings" directory which can be used by Pylance (Python
IntelliSense) to infer typings in Visual Studio Code.

Note: A "typings" directory in the root of the workspace is the default
location for Pylance to look for typings. This can be changed via
`python.analysis.stubPath` in "settings.json".

Usage
=====

```
pip3 install -r requirements.txt
scons build/ALL/gem5.opt -j$(nproc)
./build/ALL/gem5.opt util/gem5-stubgen.py
```
2023-09-21 11:52:16 -07:00
Leo Redivo
83374bdf99 misc: changed name get_default_disk_device to get_disk_device
Change-Id: Ida9673445a4426ddedc8221010204bd2b71103a5
2023-09-20 15:28:49 -07:00
Pu (Luke) Yi
3c38d4952a mem: fix bug in 3-level cache
Change-Id: I5b875908ac8f81180d781e609869e2f6fe1a8dc4
2023-09-20 12:15:33 -07:00
Bobby R. Bruce
6921d94373 python: Recursively create checkpoint dir
While there was code present in "serialize.cc" to create the checkpoint
directory, it did not do recursively. This patch ensures all the
directories are created in a path to the checkpoint directory.

Change-Id: Ibcf7f800358fd89946f550b8cfb0cef8b51fceac
2023-09-19 15:48:11 -07:00
Harshil Patel
7225da4ac6 stdlib, resources: Removed unused import
Change-Id: Iee54cc695c7c8ce146719ef583be424b792e2232
2023-09-15 10:41:45 -07:00
Harshil Patel
f3ce343a26 stdlib, resources: Added pretty printing resource
- Implemented a __str__  for AbstractResource
__str__ prints resource category, id and version.
link to resources website is also printed.

Change-Id: Iad5825ff7d8d505ceb236e00dc49bb56055fc8f0
2023-09-15 10:21:27 -07:00
Bobby R. Bruce
a101b1aba3 stdlib: Add 'to_path' arg to obtain_resource
This allows for a user to specify the exact path they want a resource to
be downloaded to. This differs from 'resource_direcctory' in that a user
may specify the file/directory name of the resource (using just the
'resource_directory' will have the resource as its ID in that directory.

Change-Id: I887be6216c7607c22e49cf38226a5e4600f39057
2023-09-14 15:33:17 -07:00
Bobby R. Bruce
b12f28af96 stdlib: Add 'quiet' option to obtain_resource func
Change-Id: I15d3be959ba7ab8af328fc6ec2912a8151941a1e
2023-09-14 15:33:17 -07:00
Leo Redivo
020bc05928 misc: moved logic of get_disk_device to workload.command_line
Change-Id: I5313bb381d5d8983b050047849fae61ea7dfc63b
2023-09-14 11:47:19 -07:00
Bobby R. Bruce
39f0bcd9af python: Mimic Python 3's -P flag in gem5
Python 3's `-P` flag, when set, means `sys.path` is not prepended with
potentially unsafe paths:
https://docs.python.org/3/using/cmdline.html#cmdoption-P

This patch allows gem5 to mimic this. This is necesssary when using
`mypy.stubgen` as it expects the Python Interpreter to have the `-P`
flag.

Change-Id: I456c8001d3ee1e806190dc37142566d50d54cc90
2023-09-12 14:51:59 -07:00
Bobby R. Bruce
eb5ae35341 resources,stdlib: Add workload to resource specialization and deprecate workload.py (#212) 2023-09-07 12:45:45 -07:00
Harshil Patel
7042d74ac2 stdlib, resources: Moved parsing params.
- moved parsing from WorkloadResource __init__ to obtain_resource

Change-Id: I9ed2aebb23af7f359bc1e5fff8ebe616a0da1374
2023-08-31 09:19:24 -07:00
KUNAL PAI
d52c7ce87f resources, stdlib: Add support for local files in obtain_resource
This patch allows a local JSON file to specify a local path
in the JSON object of a Resource, through the "url" field.

Local paths can be entered with the prefix "file:" in "url".
All File URI scheme formats are supported.

This behavior is the same as using specific Resource classes
(ex. BinaryResource) and passing a local_path into the function.

But, the above infrastructure does not allow simultaneous
creation of Resources and Workloads of those Resources.

With this patch, someone can use a local JSON, specify the location
of local Resources and create a Workload from those Resources and
test both together.

Also, this patch adds pyunit tests to check the functionality
of the function used to convert the "url" field into a path.

Change-Id: I1fa3ce33a9870528efd7751d7ca24c27baf36ad4
2023-08-29 09:47:03 -07:00
Harshil Patel
7da0aeee7d stdlib, resources: Updated warn messages
- Updated warn messages to be more clear.

Change-Id: I2a8922e96b5b544f2f229b01b3c51fc5e79995b4
2023-08-29 09:08:44 -07:00
Harshil Patel
328d140c70 stdlib, resources: Added warn msgs and commets.
- Added deprecated warnings to Workload and Abstract workload.

- Added comments to the classes changed.

Change-Id: I671daacf5ef455ea65103bd96aa442486142a486
2023-08-23 13:50:08 -07:00
Harshil Patel
a18b4b17ed stdlib, resources: depricated workload
- Added WrokloadResource in resource.py.

- depricated Workload and CustomWorkload.

- changed iscvmatched-fs.py with obtain resource for workload to test.

Change-Id: I2267c44249b96ca37da3890bf630e0d15c7335ed
Note: change example files back to original
2023-08-18 13:56:12 -07:00
Harshil Patel
ea5951a467 stdlib, resources: skeleton for workload resouce
Change-Id: I5017ac479ad61c767ede36fae195105e0519304f
2023-08-18 08:57:31 -07:00
Bobby R. Bruce
c0216dbe48 stdlib: Allow passing of func as Exit Event generator
In this case the function is turned into a generator with the
"yield" of the generator the return the function's execution.

Change-Id: I4b06d64c5479638712a11e3c1a2f7bd30f60d188
2023-08-17 16:48:33 -07:00
Leo Redivo
576e8c1897 misc: Move inform to get_default_kernel_args() and fix formatting
Change-Id: I788b630d811f8268da0e87923741cf9afdef0a3e
2023-08-11 15:07:41 -07:00
Leo Redivo
cf1678f43f misc: Fix pre-commit formatting issues
Change-Id: I50e71cfc21d43c2c17da52cf2f40591599907548
2023-08-04 13:15:55 -07:00
Bobby R. Bruce
2bef8efb94 stdlib, resources: Fixed keyerror: 'is_zipped' bug (#153)
Change-Id: I68fffd880983ebc225ec6fc8c7f8d509759b581d
2023-08-03 16:01:07 -07:00
Harshil Patel
a25ca04851 stdlib, resources: Fixed keyerror: 'is_zipped' bug
Change-Id: I68fffd880983ebc225ec6fc8c7f8d509759b581d
2023-08-03 10:59:11 -07:00
Harshil Patel
32b7ffc454 stdlib: fixed warning message
Change-Id: I04ef23529d7afc5d46fbba7558279ec08acd629a
Co-authored-by: paikunal <kunpai@ucdavis.edu>
2023-08-01 17:22:35 -07:00
leoredivo
052c870058 New function to kernel_disk_workload to allow new disk device location
Added a parameter to kernel_disk_workload which allows users to change the disk device location. Maintained the previous way of setting a disk device as the default, however added a function to allow users to override this default
2023-08-01 16:51:36 -07:00
Harshil Patel
d96df40253 stdlib: Added support for JSON via env variables.
Change-Id: I5791e6d51b3b9f68eb212a46c4cd0add23668340
Co-authored-by: Kunal Pai <kunpai@ucdavis.edu>
2023-08-01 16:22:44 -07:00
Bobby R. Bruce
01623fac68 stdlib,configs,tests: Remove deprecated Resource classes usage (#102)
* stdlib,configs,tests: Remove `Resource` class use

This class is deprecated, but was still used in various example
configuration scriots and tests. This patch replaces it with the
`obtain_resource` function.

Change-Id: I0c89bf17783ccaaafc18072aaeefb5d1e207bc55

* configs: Remove `CustomDiskImageResource` use

The class is deprecated but was still used in the SPEC example scripts.
This patch replaces it with the `DiskImageResource` class.

Change-Id: Ie0697fe59a3d737b05eb45ff3bc964f42b0387e0

* configs,tests: Remove `CustomResource` use

This class is deprecated but was still used in example scripts and
mentioned, incorrectly, in comments in the pyunit tests. This patch
removes these.

Change-Id: Icb6d02f47a5b72cd58551e5dcd59cc72d6a91a01

* stdlib: Remove '\' in Workload docstring example

This example shows how to use the Workload. The backslash is not correct Python and would fail if used in this way.

Co-authored-by: Jason Lowe-Power <jason@lowepower.com>

---------

Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
2023-07-20 23:08:39 -07:00
Kunal Pai
3c6563d6f7 stdlib: Change resource compatibility warning (#91)
* stdlib: Change resource compatibility warning

If the gem5 version is "develop", the warning will not
be thrown.

Change-Id: Id2be1c4323c6ca06c5503c2885c1608f8d119420

* stdlib: Change resource compatibility warning

If the gem5 version is "develop", the warning will not
be thrown.

Change-Id: Id2be1c4323c6ca06c5503c2885c1608f8d119420

* tests: Edit obtain_resources warning test

Since we are editing the warning message for
the develop branch, the test removes the
warning message as well.

Change-Id: I90882340188360bb3435344cdc14b324412c6c0e

---------

Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
2023-07-20 12:00:49 -07:00
Lingkang
573523c07a python: fix fatal in main.py (github #78) (#93)
* python: fix fatal in main.py (github #78)

Issue-On: https://github.com/gem5/gem5/issues/78

* python: fix fatal in main.py (github #78)

Issue-On: https://github.com/gem5/gem5/issues/78
Change-Id: I80855b05168a067ddd7706ad9fd7e71e75bfd3b1

---------

Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
2023-07-18 14:00:57 -07:00
Jason Lowe-Power
442923c414 Add feature to output citations automatically based on configuration (#90)
This change adds a new file to m5out which is citations.bib.
This file will contain the citations to the papers which describe the
aspects of the gem5 simulator that the simulation uses. In other words,
each simulation configuration could generate a different bib file
referencing different works.

Each SimObject can now have a set of citations associated with it. After
the system is built (in `instantiate`), the citations.bib file is
created by parsing all SimObjects that have been instantiated and taking
the union of their associated citations.

This commit is not meant to add all citations, but to act as an example
for others to add more citations to gem5.

Change-Id: Icd5c46fd9ee44adbeec1fea162657f5716f7e5ef
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
2023-07-17 10:41:51 -07:00
Mahyar Samani
b2fcc558d8 stdlib: Deviding range for linear multicore. (#63)
This patch changes the way memory ranges are devided when using
multiple cores for linear traffic. The current state assigns the
same range to multiple linear generators so all the cores start
generating the same trace. This patch devides the overall range
assigned to the generator ([min_addr:max_addr]) between the cores.

Change-Id: I49f69b3d61b590899f8d54ee3be997ad22d7fa9b

Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
Co-authored-by: mkjost0 <50555529+mkjost0@users.noreply.github.com>
Co-authored-by: Bobby R. Bruce <bbruce@ucdavis.edu>
2023-07-14 07:33:21 -07:00
Luming Wang
c634b23305 sim,python: follow the new CPython startup sequence
Currently, gem5 suffers from several bugs related
to Python interpreter's locale encoding issues.
gem5 will crash when the working directory contains
Non-ASCII characters.

The reason is that Python 3.8+ introduces a new
interpreter startup sequence [1]. The startup
sequence consists of three phases:

1. Python core runtime preinitialization
2. Python core runtime initialization
3. Main interpreter configuration

Stage 1 determining the encodings used for system
interfaces.

However, gem5 doesn't preinitialize the Python
interpreter. Thus, the locale settings do not take
effect. This patch preinitialize the Python for
Python 3.8+.

Also, this patch avoid the use of `Py_SetProgramName`,
which is deprecated since Python 3.11[3].

[1] https://peps.python.org/pep-0432/
[2] https://peps.python.org/pep-0587/
[3] https://docs.python.org/3/c-api/init.html#c.Py_SetProgramName

Change-Id: I08a2ec6ab2b39a95ab194909932c8fc578c745ce
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/70898
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Roger Chang <rogerycchang@google.com>
2023-07-10 23:00:31 +00:00
Bobby R. Bruce
a43f8e6904 stdlib: Allow passing of func list as exit_event generator
Allows for a passing of functions to specify actions to execute on an
exit event via the Simulator module in the stdlib.

The list of functions must have no manditory arguments and return True
if the Simulation is to exit upon the function call's completion.

Issue-on: https://gem5.atlassian.net/browse/GEM5-1126
Change-Id: Ia88caf2975227e78243763627acab9e9f89e2a7d
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/62691
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2023-07-10 22:56:54 +00:00