Commit Graph

1079 Commits

Author SHA1 Message Date
Bobby R. Bruce
c4156b06fb python: Fix base logic in MetaSimObject
This ensures `class Foo` is considered equivalent to `class
Foo(object)`.

Change-Id: I65a8aec27280a0806308bbc9d32281dfa6a8f84e
2023-10-10 21:47:08 -07:00
Bobby R. Bruce
298119e402 misc,python: Run pre-commit run --all-files
Applies the `pyupgrade` hook to all files in the repo.

Change-Id: I9879c634a65c5fcaa9567c63bc5977ff97d5d3bf
2023-10-10 21:47:07 -07:00
Bobby R. Bruce
3f5d7d647a misc: Run pre-commit autoupdate (#419)
1. Runs `pre-commit autoupdate`.
2. Runs `pre-commit run --all-files`.
3. Adds (2.) to ".git-blame-ignore-rev".
2023-10-10 21:41:33 -07:00
Bobby R. Bruce
d559c24ac2 stdlib: Improve handing of errors in Atlas request failures (#404)
Now:

* The Atlas Client will attempt a connection 4 times, using an
exponential backoff approach between attempts.
* When a failure does arise a rich output is given so problems can be
easily diagnosed.

Addresses: #340
2023-10-10 21:34:24 -07:00
Harshil Patel
bbc301f2f0 stdlib, tests: Fixed bugs and tests
- Fixed bugs rekated to retrying on request faliure.
- Updated the pyunit tests.

Change-Id: Ia484690267bf27018488324f3408f7e47c59bef3
2023-10-10 15:54:20 -07:00
Bobby R. Bruce
ddf6cb88e4 misc: Run pre-commit run --all-files
This is reflect the updates made to black when running `pre-commit
autoupdate`.

Change-Id: Ifb7fea117f354c7f02f26926a5afdf7d67bc5919
2023-10-10 14:01:58 -07:00
root
05ebbd2184 stdlib: Fix use internal _hashlib in md5_utils.py
Removes the use of the internal _hashlib, which is an
internal Python API

Change-Id: Id4541a143adb767ca7d942c0fd8a1cf1a08a04ab
2023-10-10 06:18:59 +00:00
Bobby R. Bruce
51c881d0f1 stdlib: Improve handing of errors in Atlas request failures
Now:

* The Atlas Client will attempt a connection 4 times, using an
  exponential backoff approach between attempts.
* When a failure does arise a rich output is given so problems can be
  easily diagnosed.

Change-Id: I3df332277c33a040c0ed734b9f3e28f38606af44
2023-10-09 16:30:02 -07:00
Harshil Patel
452a600c49 New function to kernel_disk_workload to allow new disk device location (#151)
Added a parameter (_disk_device) to kernel_disk_workload which allows
users to change the disk device location. get_disk_device() now chooses
between the parameter and, if no parameter was passed, it calls a new
function _get_default_disk_device() which is implemented by each board
and has a default disk device according to each board, eg /dev/hda in
the x86_board. The previous way of setting a disk device still exists as
a default, however, with the new function users can now override this
default
2023-10-09 10:33:45 -07:00
Harshil Patel
79f40ffdab stdlib: Del comment stating SE mode limited to single thread (#402)
This comment was left in the codebase in error. The
`set_se_binary_workload` function works fine with multi-threaded
applications. This hasn't been a restriction for some time.
2023-10-09 10:30:32 -07:00
David Schall
edf9092fee cpu: Restructure BTB
- A new abstract BTB class is created to enable different BTB
  implementations. The new BTB class gets its own parameter
  and stats.
- An enum is added to differentiate branch instruction types.
  This enum is used to enhance statistics and BPU management.
- The existing BTB is moved into `simple_btb` as default.
- An additional function is added to store the static instruction in
  the BTB. This function is used for the decoupled front-end.
- Update configs to match new BTB parameters.

Change-Id: I99b29a19a1b57e59ea2b188ed7d62a8b79426529
Signed-off-by: David Schall <david.schall@ed.ac.uk>
2023-10-09 14:37:47 +00:00
Bobby R. Bruce
4db748a507 resources, stdlib: Adding 'suite' category to gem5 (#191) 2023-10-05 13:26:58 -07:00
Bobby R. Bruce
f75c0fca8a stdlib: Del comment stating SE mode limited to single thread
This comment was left in the codebase in error. The
`set_se_binary_workload` function works fine with multi-threaded
applications. This hasn't been a restriction for some time.

Change-Id: I1b1d27c86f8d9284659f62ae27d752bf5325e31b
2023-10-05 10:20:55 -07:00
Leo Redivo
98a6cd6ee2 misc: changed call get_default_disk_device to get_disk_device
Change-Id: I240da78a658208211ede6648547dfa4c971074a1
2023-10-04 13:32:35 -07:00
Harshil Patel
3af3c1121b stdlib, resources: Addressed requested changes
Change-Id: I22abdc3bdcdde52301ed10cb3113e8925159c245
Co-authored-by: Kunal Pai <kunpai@users.noreply.github.com>
2023-10-02 23:27:32 -07:00
Harshil Patel
7301d4bd19 python: Add importer to standalone gem5py_m5 (#369)
I believe the point of this binary was to allow people to use the m5
objects without the entire gem5 binary. However, without adding the
importer call, this did not work. Unfortunately, with the importer call
there is a circular dependence on the original gem5py.cc file.
Therefore, this change creates a new file that has the importer call.

Now, with the `gem5py_m5` binary you can run python code that references
modules in `src/python`. Note that `_m5` is not available, so anything
that depends on the gem5 SimObjects' implementation will not work.
However, this can still be useful for things like getting Resources,
processing stats, etc.
2023-10-02 14:28:45 -07:00
Harshil Patel
f9781af6e5 mem: fix bug in 3-level cache (#265)
The L3 cache did not work due to argument type mismatch in the call to
the constructor `DMAController`. The second argument is expecting a
`RubySystem` type but the code passes in a `cache_line_size` variable.
After I change the second argument to `self.ruby_system` everything
works.
2023-09-29 10:59:18 -07:00
Harshil Patel
8182f8084b stdlib, resources, tests: Introduce Suite of Workloads
This patch introduces a new category called "suite".
A suite is a collection of workloads.
Each workload in a SuiteResource has a tag that can be narrowed down
through the function with_input_group.
Also, the set of input groups can be seen through list_input_groups.
Added unit tests to test all functions of SuiteResource class.

Change-Id: Iddda5c898b32b7cd874987dbe694ac09aa231f08

Co-authored-by: Kunal Pai <kunpai@ucdavis.edu>
2023-09-29 10:50:09 -07:00
Jason Lowe-Power
aaad79cf51 python: Add importer to standalone gem5py_m5
I believe the point of this binary was to allow people to use the m5
objects without the entire gem5 binary. However, without adding the
importer call, this did not work. Unfortunately, with the importer call
there is a circular dependence on the original gem5py.cc file.
Therefore, this change creates a new file that has the importer call.

Now, with the `gem5py_m5` binary you can run python code that references
modules in `src/python`. Note that `_m5` is not available, so anything
that depends on the gem5 SimObjects' implementation will not work.
However, thic can still be useful for things like getting Resources,
processing stats, etc.

Change-Id: I5c0e5d1a669fe5ce491458df916f2049c81292eb
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
2023-09-28 11:22:33 -07:00
Bobby R. Bruce
62d34ef374 misc: 'sim{out/err}' -> 'sim{out/err}.txt' (#250)
By default, the `--stderr-file` and `--stdout-file` arguments were
directing the simulator to output files named "simerr" and "simout"
respectively if an output redirect was requested.

A small annoyance is these files lack an extension meaning programs
refuse to open them, or don't do so withou additional effort. On many
systems they are assumed to scripts.

This patch adds the `.txt` extension to both, thus clearly indicating to
other programs these are text files and can be opened and read as such.
2023-09-27 17:36:03 -07:00
Bobby R. Bruce
5d254ffb02 stdlib, resources: Added pretty printing resource (#323)
- Implemented a __str__ for AbstractResource __str__ prints resource
category, id and version.
link to resources website is also printed.
2023-09-27 17:32:35 -07:00
Bobby R. Bruce
074fa4c604 misc,ext,tests: Automatically split CI TestLib tests across GitHub Action jobs (#263)
This PR utilizes GitHub Action's matrix's to automatically distribute
the CI testlib gem5 build and test jobs across available GitHub Action
Runners.

The CI tests (the `quick` testlib tests, i.e. those run with `./main.py
run`) are distributed across the runners on a per directory basis ---
all directories under "tests/gem5" are run as their own jobs.

The necessary gem5 builds for each workflow are now automatically
inferred via the introduction of `./main.py list`'s `--build-targets`
flag which returns the gem5 build target for a given test or collection
of tests. E.g., `./main.py list --build-targets` will return the build
targets for all the `quick` testlib tests and `./main.py list
--build-target --uid=<id>` will return the build targets the test suite
`<id>` requires.

Moving from monolithic jobs to fine-grained ones will make the locaiton
of test failures more obvious. Each job has it's own artifact containing
"test/testing-results" for the tests run in that job. In addition,
maintenance of these files should become less burdensome due to less
hardcoding.
2023-09-27 14:32:16 -07:00
Harshil Patel
633bdc08f2 stdlib: Addressed requested changes
- Added mulitline string for print message

- Added get_category_name method instead of having category as variable

Change-Id: I51e0e14a70e802453c21070711b200bc47994ba3
2023-09-27 11:36:51 -07:00
Bobby R. Bruce
391f62b213 misc: 'sim{out/err}' -> 'sim{out/err}.txt'
By default, the --stderr-file and --stdout-file arguments were
directing the simulator output to files named "simerr" and
"simout" respectively if an output redirect was requested.

A small annoyance is these files lack an extension meaning programs
refuse to open them, or to do so without some additional effort. On
many systems they are assumed to scripts.

This patch adds the .txt extension to both, thus clearly indicating
to other programs these are text files and can be opened to be read
as such.

Change-Id: Iff5af4a9e6966b4467d005a029dbf401099fbd35
2023-09-21 12:57:43 -07:00
Bobby R. Bruce
3f9afe96c6 python,util: Add Python MyPy Stubgen to enable Pylance IntelliSense (#307)
This allows us to generate stubs for the modules in gem5. The output
will be a "typings" directory which can be used by Pylance (Python
IntelliSense) to infer typings in Visual Studio Code.

Note: A "typings" directory in the root of the workspace is the default
location for Pylance to look for typings. This can be changed via
`python.analysis.stubPath` in "settings.json".

Usage
=====

```
pip3 install -r requirements.txt
scons build/ALL/gem5.opt -j$(nproc)
./build/ALL/gem5.opt util/gem5-stubgen.py
```
2023-09-21 11:52:16 -07:00
Leo Redivo
83374bdf99 misc: changed name get_default_disk_device to get_disk_device
Change-Id: Ida9673445a4426ddedc8221010204bd2b71103a5
2023-09-20 15:28:49 -07:00
Pu (Luke) Yi
3c38d4952a mem: fix bug in 3-level cache
Change-Id: I5b875908ac8f81180d781e609869e2f6fe1a8dc4
2023-09-20 12:15:33 -07:00
Bobby R. Bruce
6921d94373 python: Recursively create checkpoint dir
While there was code present in "serialize.cc" to create the checkpoint
directory, it did not do recursively. This patch ensures all the
directories are created in a path to the checkpoint directory.

Change-Id: Ibcf7f800358fd89946f550b8cfb0cef8b51fceac
2023-09-19 15:48:11 -07:00
Harshil Patel
7225da4ac6 stdlib, resources: Removed unused import
Change-Id: Iee54cc695c7c8ce146719ef583be424b792e2232
2023-09-15 10:41:45 -07:00
Harshil Patel
f3ce343a26 stdlib, resources: Added pretty printing resource
- Implemented a __str__  for AbstractResource
__str__ prints resource category, id and version.
link to resources website is also printed.

Change-Id: Iad5825ff7d8d505ceb236e00dc49bb56055fc8f0
2023-09-15 10:21:27 -07:00
Bobby R. Bruce
a101b1aba3 stdlib: Add 'to_path' arg to obtain_resource
This allows for a user to specify the exact path they want a resource to
be downloaded to. This differs from 'resource_direcctory' in that a user
may specify the file/directory name of the resource (using just the
'resource_directory' will have the resource as its ID in that directory.

Change-Id: I887be6216c7607c22e49cf38226a5e4600f39057
2023-09-14 15:33:17 -07:00
Bobby R. Bruce
b12f28af96 stdlib: Add 'quiet' option to obtain_resource func
Change-Id: I15d3be959ba7ab8af328fc6ec2912a8151941a1e
2023-09-14 15:33:17 -07:00
Leo Redivo
020bc05928 misc: moved logic of get_disk_device to workload.command_line
Change-Id: I5313bb381d5d8983b050047849fae61ea7dfc63b
2023-09-14 11:47:19 -07:00
Bobby R. Bruce
39f0bcd9af python: Mimic Python 3's -P flag in gem5
Python 3's `-P` flag, when set, means `sys.path` is not prepended with
potentially unsafe paths:
https://docs.python.org/3/using/cmdline.html#cmdoption-P

This patch allows gem5 to mimic this. This is necesssary when using
`mypy.stubgen` as it expects the Python Interpreter to have the `-P`
flag.

Change-Id: I456c8001d3ee1e806190dc37142566d50d54cc90
2023-09-12 14:51:59 -07:00
Bobby R. Bruce
eb5ae35341 resources,stdlib: Add workload to resource specialization and deprecate workload.py (#212) 2023-09-07 12:45:45 -07:00
Harshil Patel
7042d74ac2 stdlib, resources: Moved parsing params.
- moved parsing from WorkloadResource __init__ to obtain_resource

Change-Id: I9ed2aebb23af7f359bc1e5fff8ebe616a0da1374
2023-08-31 09:19:24 -07:00
KUNAL PAI
d52c7ce87f resources, stdlib: Add support for local files in obtain_resource
This patch allows a local JSON file to specify a local path
in the JSON object of a Resource, through the "url" field.

Local paths can be entered with the prefix "file:" in "url".
All File URI scheme formats are supported.

This behavior is the same as using specific Resource classes
(ex. BinaryResource) and passing a local_path into the function.

But, the above infrastructure does not allow simultaneous
creation of Resources and Workloads of those Resources.

With this patch, someone can use a local JSON, specify the location
of local Resources and create a Workload from those Resources and
test both together.

Also, this patch adds pyunit tests to check the functionality
of the function used to convert the "url" field into a path.

Change-Id: I1fa3ce33a9870528efd7751d7ca24c27baf36ad4
2023-08-29 09:47:03 -07:00
Harshil Patel
7da0aeee7d stdlib, resources: Updated warn messages
- Updated warn messages to be more clear.

Change-Id: I2a8922e96b5b544f2f229b01b3c51fc5e79995b4
2023-08-29 09:08:44 -07:00
Harshil Patel
328d140c70 stdlib, resources: Added warn msgs and commets.
- Added deprecated warnings to Workload and Abstract workload.

- Added comments to the classes changed.

Change-Id: I671daacf5ef455ea65103bd96aa442486142a486
2023-08-23 13:50:08 -07:00
Harshil Patel
a18b4b17ed stdlib, resources: depricated workload
- Added WrokloadResource in resource.py.

- depricated Workload and CustomWorkload.

- changed iscvmatched-fs.py with obtain resource for workload to test.

Change-Id: I2267c44249b96ca37da3890bf630e0d15c7335ed
Note: change example files back to original
2023-08-18 13:56:12 -07:00
Harshil Patel
ea5951a467 stdlib, resources: skeleton for workload resouce
Change-Id: I5017ac479ad61c767ede36fae195105e0519304f
2023-08-18 08:57:31 -07:00
Bobby R. Bruce
c0216dbe48 stdlib: Allow passing of func as Exit Event generator
In this case the function is turned into a generator with the
"yield" of the generator the return the function's execution.

Change-Id: I4b06d64c5479638712a11e3c1a2f7bd30f60d188
2023-08-17 16:48:33 -07:00
Leo Redivo
576e8c1897 misc: Move inform to get_default_kernel_args() and fix formatting
Change-Id: I788b630d811f8268da0e87923741cf9afdef0a3e
2023-08-11 15:07:41 -07:00
Leo Redivo
cf1678f43f misc: Fix pre-commit formatting issues
Change-Id: I50e71cfc21d43c2c17da52cf2f40591599907548
2023-08-04 13:15:55 -07:00
Bobby R. Bruce
2bef8efb94 stdlib, resources: Fixed keyerror: 'is_zipped' bug (#153)
Change-Id: I68fffd880983ebc225ec6fc8c7f8d509759b581d
2023-08-03 16:01:07 -07:00
Harshil Patel
a25ca04851 stdlib, resources: Fixed keyerror: 'is_zipped' bug
Change-Id: I68fffd880983ebc225ec6fc8c7f8d509759b581d
2023-08-03 10:59:11 -07:00
Harshil Patel
32b7ffc454 stdlib: fixed warning message
Change-Id: I04ef23529d7afc5d46fbba7558279ec08acd629a
Co-authored-by: paikunal <kunpai@ucdavis.edu>
2023-08-01 17:22:35 -07:00
leoredivo
052c870058 New function to kernel_disk_workload to allow new disk device location
Added a parameter to kernel_disk_workload which allows users to change the disk device location. Maintained the previous way of setting a disk device as the default, however added a function to allow users to override this default
2023-08-01 16:51:36 -07:00
Harshil Patel
d96df40253 stdlib: Added support for JSON via env variables.
Change-Id: I5791e6d51b3b9f68eb212a46c4cd0add23668340
Co-authored-by: Kunal Pai <kunpai@ucdavis.edu>
2023-08-01 16:22:44 -07:00
Bobby R. Bruce
01623fac68 stdlib,configs,tests: Remove deprecated Resource classes usage (#102)
* stdlib,configs,tests: Remove `Resource` class use

This class is deprecated, but was still used in various example
configuration scriots and tests. This patch replaces it with the
`obtain_resource` function.

Change-Id: I0c89bf17783ccaaafc18072aaeefb5d1e207bc55

* configs: Remove `CustomDiskImageResource` use

The class is deprecated but was still used in the SPEC example scripts.
This patch replaces it with the `DiskImageResource` class.

Change-Id: Ie0697fe59a3d737b05eb45ff3bc964f42b0387e0

* configs,tests: Remove `CustomResource` use

This class is deprecated but was still used in example scripts and
mentioned, incorrectly, in comments in the pyunit tests. This patch
removes these.

Change-Id: Icb6d02f47a5b72cd58551e5dcd59cc72d6a91a01

* stdlib: Remove '\' in Workload docstring example

This example shows how to use the Workload. The backslash is not correct Python and would fail if used in this way.

Co-authored-by: Jason Lowe-Power <jason@lowepower.com>

---------

Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
2023-07-20 23:08:39 -07:00