This allows us to generate stubs for the modules in gem5. The output
will be a "typings" directory which can be used by Pylance (Python
IntelliSense) to infer typings in Visual Studio Code.
Note: A "typings" directory in the root of the workspace is the default
location for Pylance to look for typings. This can be changed via
`python.analysis.stubPath` in "settings.json".
Usage
=====
```
pip3 install -r requirements.txt
scons build/ALL/gem5.opt -j$(nproc)
./build/ALL/gem5.opt util/gem5-stubgen.py
```
While there was code present in "serialize.cc" to create the checkpoint
directory, it did not do recursively. This patch ensures all the
directories are created in a path to the checkpoint directory.
Change-Id: Ibcf7f800358fd89946f550b8cfb0cef8b51fceac
- Implemented a __str__ for AbstractResource
__str__ prints resource category, id and version.
link to resources website is also printed.
Change-Id: Iad5825ff7d8d505ceb236e00dc49bb56055fc8f0
This allows for a user to specify the exact path they want a resource to
be downloaded to. This differs from 'resource_direcctory' in that a user
may specify the file/directory name of the resource (using just the
'resource_directory' will have the resource as its ID in that directory.
Change-Id: I887be6216c7607c22e49cf38226a5e4600f39057
Python 3's `-P` flag, when set, means `sys.path` is not prepended with
potentially unsafe paths:
https://docs.python.org/3/using/cmdline.html#cmdoption-P
This patch allows gem5 to mimic this. This is necesssary when using
`mypy.stubgen` as it expects the Python Interpreter to have the `-P`
flag.
Change-Id: I456c8001d3ee1e806190dc37142566d50d54cc90
This patch allows a local JSON file to specify a local path
in the JSON object of a Resource, through the "url" field.
Local paths can be entered with the prefix "file:" in "url".
All File URI scheme formats are supported.
This behavior is the same as using specific Resource classes
(ex. BinaryResource) and passing a local_path into the function.
But, the above infrastructure does not allow simultaneous
creation of Resources and Workloads of those Resources.
With this patch, someone can use a local JSON, specify the location
of local Resources and create a Workload from those Resources and
test both together.
Also, this patch adds pyunit tests to check the functionality
of the function used to convert the "url" field into a path.
Change-Id: I1fa3ce33a9870528efd7751d7ca24c27baf36ad4
- Added deprecated warnings to Workload and Abstract workload.
- Added comments to the classes changed.
Change-Id: I671daacf5ef455ea65103bd96aa442486142a486
- Added WrokloadResource in resource.py.
- depricated Workload and CustomWorkload.
- changed iscvmatched-fs.py with obtain resource for workload to test.
Change-Id: I2267c44249b96ca37da3890bf630e0d15c7335ed
Note: change example files back to original
In this case the function is turned into a generator with the
"yield" of the generator the return the function's execution.
Change-Id: I4b06d64c5479638712a11e3c1a2f7bd30f60d188
Added a parameter to kernel_disk_workload which allows users to change the disk device location. Maintained the previous way of setting a disk device as the default, however added a function to allow users to override this default
* stdlib,configs,tests: Remove `Resource` class use
This class is deprecated, but was still used in various example
configuration scriots and tests. This patch replaces it with the
`obtain_resource` function.
Change-Id: I0c89bf17783ccaaafc18072aaeefb5d1e207bc55
* configs: Remove `CustomDiskImageResource` use
The class is deprecated but was still used in the SPEC example scripts.
This patch replaces it with the `DiskImageResource` class.
Change-Id: Ie0697fe59a3d737b05eb45ff3bc964f42b0387e0
* configs,tests: Remove `CustomResource` use
This class is deprecated but was still used in example scripts and
mentioned, incorrectly, in comments in the pyunit tests. This patch
removes these.
Change-Id: Icb6d02f47a5b72cd58551e5dcd59cc72d6a91a01
* stdlib: Remove '\' in Workload docstring example
This example shows how to use the Workload. The backslash is not correct Python and would fail if used in this way.
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
---------
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
* stdlib: Change resource compatibility warning
If the gem5 version is "develop", the warning will not
be thrown.
Change-Id: Id2be1c4323c6ca06c5503c2885c1608f8d119420
* stdlib: Change resource compatibility warning
If the gem5 version is "develop", the warning will not
be thrown.
Change-Id: Id2be1c4323c6ca06c5503c2885c1608f8d119420
* tests: Edit obtain_resources warning test
Since we are editing the warning message for
the develop branch, the test removes the
warning message as well.
Change-Id: I90882340188360bb3435344cdc14b324412c6c0e
---------
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
This change adds a new file to m5out which is citations.bib.
This file will contain the citations to the papers which describe the
aspects of the gem5 simulator that the simulation uses. In other words,
each simulation configuration could generate a different bib file
referencing different works.
Each SimObject can now have a set of citations associated with it. After
the system is built (in `instantiate`), the citations.bib file is
created by parsing all SimObjects that have been instantiated and taking
the union of their associated citations.
This commit is not meant to add all citations, but to act as an example
for others to add more citations to gem5.
Change-Id: Icd5c46fd9ee44adbeec1fea162657f5716f7e5ef
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
This patch changes the way memory ranges are devided when using
multiple cores for linear traffic. The current state assigns the
same range to multiple linear generators so all the cores start
generating the same trace. This patch devides the overall range
assigned to the generator ([min_addr:max_addr]) between the cores.
Change-Id: I49f69b3d61b590899f8d54ee3be997ad22d7fa9b
Co-authored-by: Jason Lowe-Power <jason@lowepower.com>
Co-authored-by: mkjost0 <50555529+mkjost0@users.noreply.github.com>
Co-authored-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Currently, gem5 suffers from several bugs related
to Python interpreter's locale encoding issues.
gem5 will crash when the working directory contains
Non-ASCII characters.
The reason is that Python 3.8+ introduces a new
interpreter startup sequence [1]. The startup
sequence consists of three phases:
1. Python core runtime preinitialization
2. Python core runtime initialization
3. Main interpreter configuration
Stage 1 determining the encodings used for system
interfaces.
However, gem5 doesn't preinitialize the Python
interpreter. Thus, the locale settings do not take
effect. This patch preinitialize the Python for
Python 3.8+.
Also, this patch avoid the use of `Py_SetProgramName`,
which is deprecated since Python 3.11[3].
[1] https://peps.python.org/pep-0432/
[2] https://peps.python.org/pep-0587/
[3] https://docs.python.org/3/c-api/init.html#c.Py_SetProgramName
Change-Id: I08a2ec6ab2b39a95ab194909932c8fc578c745ce
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/70898
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Roger Chang <rogerycchang@google.com>
This patch includes several changes to the gem5 tools interface to the
gem5-resources infrastructure. These are:
* The old download and JSON query functions have been removed from the
downloader module. These functions were used for directly downloading
and inspecting the resource JSON file, hosted at
https://resources.gem5.org/resources. This information is now obtained
via `gem5.client`. If a resources JSON file is specified as a client,
it should conform to the new schema:
https//resources.gem5.org/gem5-resources-schema.json. The old schema
(pre-v23.0) is no longer valid. Tests have been updated to reflect
this change. Those which tested these old functions have been removed.
* Unused imports have been removed.
* For the resource query functions, and those tasked with obtaining the
resources, the parameter `gem5_version` has been added. In all cases
it does the same thing:
* It will filter results based on compatibility to the
`gem5_version` specified. If no resources are compatible the
latest version of that resource is chosen (though a warning is
thrown).
* By default it is set to the current gem5 version.
* It is optional. If `None`, this filtering functionality is not
carried out.
* Tests have been updated to fix the version to “develop” so the
they do not break between versions.
* The `gem5_version` parameters will filter using a logic which will
base compatibility on the specificity of the gem5-version specified in
a resource’s data. If a resource has a compatible gem5-version of
“v18.4” it will be compatible with any minor/hotfix version within the
v18.4 release (this can be seen as matching on “v18.4.*.*”.) Likewise,
if a resource has a compatible gem5-version of “v18.4.1” then it’s
only compatible with the v18.4.1 release but any of it’s hot fix
releases (“v18.4.1.*”).
* The ‘list_resources’ function has been updated to use the
“gem5.client” APIs to get resource information from the clients
(MongoDB or a JSON file). This has been designed to remain backwards
compatible to as much as is possible, though, due to schema changes,
the function does search across all versions of gem5.
* `get_resources` function was added to the `AbstractClient`. This is a
more general function than `get_resource_by_id`. It was
primarily created to handle the `list_resources` update but is a
useful update to the API. The `get_resource_by_id` function has been
altered to function as a wrapped to the `get_resources` function.
* Removed “GEM5_RESOURCE_JSON” code has been removed. This is no longer
used.
* Tests have been cleaned up a little bit to be easier to read.
* Some docstrings have been updated.
Things that are left TODO with this code:
* The client_wrapper/client/abstract_client abstractions are rather
pointless. In particular the client_wrapper and client classes could
be merged.
* The downloader module no longer does much and should have its
functions merged into other modules.
* With the addition of the `get_resources` function, much of the code in
the `AbstractClient` could be simplified.
Change-Id: I0ce48e88b93a2b9db53d4749861fa0b5f9472053
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71506
Reviewed-by: Kunal Pai <kunpai@ucdavis.edu>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
(cherry picked from commit 82587ce71b)
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71739
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
The cache is modeled after an AMD EPYC cache, but not exactly
like AMD EPYC cache.
- K cores per core complex (CCD), each core has one private split L1,
and one private L2.
- K cores in the same CCD share 1 slice of L3 cache, which is not
a victim cache.
- There can be multiple CCDs, which communicate with each other via
Cross-CCD router. The Cross-CCD rounter is also connected to
directory controllers and dma controllers.
- All links latency are set to 1.
Change-Id: Ib64248bed9155b8e48e5158ffdeebf1f2d770754
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71598
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
This patch includes several changes to the gem5 tools interface to the
gem5-resources infrastructure. These are:
* The old download and JSON query functions have been removed from the
downloader module. These functions were used for directly downloading
and inspecting the resource JSON file, hosted at
https://resources.gem5.org/resources. This information is now obtained
via `gem5.client`. If a resources JSON file is specified as a client,
it should conform to the new schema:
https//resources.gem5.org/gem5-resources-schema.json. The old schema
(pre-v23.0) is no longer valid. Tests have been updated to reflect
this change. Those which tested these old functions have been removed.
* Unused imports have been removed.
* For the resource query functions, and those tasked with obtaining the
resources, the parameter `gem5_version` has been added. In all cases
it does the same thing:
* It will filter results based on compatibility to the
`gem5_version` specified. If no resources are compatible the
latest version of that resource is chosen (though a warning is
thrown).
* By default it is set to the current gem5 version.
* It is optional. If `None`, this filtering functionality is not
carried out.
* Tests have been updated to fix the version to “develop” so the
they do not break between versions.
* The `gem5_version` parameters will filter using a logic which will
base compatibility on the specificity of the gem5-version specified in
a resource’s data. If a resource has a compatible gem5-version of
“v18.4” it will be compatible with any minor/hotfix version within the
v18.4 release (this can be seen as matching on “v18.4.*.*”.) Likewise,
if a resource has a compatible gem5-version of “v18.4.1” then it’s
only compatible with the v18.4.1 release but any of it’s hot fix
releases (“v18.4.1.*”).
* The ‘list_resources’ function has been updated to use the
“gem5.client” APIs to get resource information from the clients
(MongoDB or a JSON file). This has been designed to remain backwards
compatible to as much as is possible, though, due to schema changes,
the function does search across all versions of gem5.
* `get_resources` function was added to the `AbstractClient`. This is a
more general function than `get_resource_by_id`. It was
primarily created to handle the `list_resources` update but is a
useful update to the API. The `get_resource_by_id` function has been
altered to function as a wrapped to the `get_resources` function.
* Removed “GEM5_RESOURCE_JSON” code has been removed. This is no longer
used.
* Tests have been cleaned up a little bit to be easier to read.
* Some docstrings have been updated.
Things that are left TODO with this code:
* The client_wrapper/client/abstract_client abstractions are rather
pointless. In particular the client_wrapper and client classes could
be merged.
* The downloader module no longer does much and should have its
functions merged into other modules.
* With the addition of the `get_resources` function, much of the code in
the `AbstractClient` could be simplified.
Change-Id: I0ce48e88b93a2b9db53d4749861fa0b5f9472053
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71506
Reviewed-by: Kunal Pai <kunpai@ucdavis.edu>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This patch makes changes to the stdlib based on the gem5 Vision project.
Firstly, a MongoDB database is supported.
A JSON database's support is continued.
The JSON can either be a local path or a raw GitHub link.
The data for these databases is stored in src/python
under "gem5-config.json".
This will be used by default.
However, the configuration can be overridden:
- by providing a path using the GEM5_CONFIG env variable.
- by placing a gem5-config.json file in the current working directory.
An AbstractClient is an abstract class that implements
searching and sorting relevant to the databases.
Clients is an optional list that can be passed
while defining any Resource class and obtain_resource.
These databases can be defined in the config JSON.
Resources now have versions. This allows for a
single version, e.g., 'x86-ubuntu-boot', to have
multiple versions. As such, the key of a resource is
its ID and Version (e.g., 'x86-ubuntu-boot/v2.1.0').
Different versions of a resource might be compatible
with different versions of gem5.
By default, it picks the latest version compatible with the gem5 Version
of the user.
A gem5 resource schema now has additional fields.
These are:
- source_url: Stores URL of GitHub Source of the resource.
- license: License information of the resource.
- tags: Words to identify a resource better, like hello for hello-world
- example_usage: How to use the resource in a simulation.
- gem5_versions: List of gem5 versions that resource is compatible with.
- resource_version: The version of the resource itself.
- size: The download size of the resource, if it exists.
- code_examples: List of objects.
These objects contain the path to where a resource is
used in gem5 example config scripts,
and if the resource itself is used in tests or not.
- category: Category of the resource, as defined by classes in
src/python/gem5/resources/resource.py.
Some fields have been renamed:
- "name" is changed to "id"
- "documentation" is changed to "description"
Besides these, the schema also supports resource specialization.
It adds fields relevant to a specific resource as specified in
src/python/gem5/resources/resource.py
These changes have been made to better present
information on the new gem5 Resources website.
But, they do not affect the way resources are used by a gem5 user.
This patch is also backwards compatible.
Existing code doesn't break with this new infrastructure.
Also, refs in the tests have been changed to match this new schema.
Tests have been changed to work with the two clients.
Change-Id: Ia9bf47f7900763827fd5e873bcd663cc3ecdba40
Co-authored-by: Kunal Pai <kunpai@ucdavis.edu>
Co-authored-by: Parth Shah <helloparthshah@gmail.com>
Co-authored-by: Harshil Patel <harshilp2107@gmail.com>
Co-authored-by: aarsli <arsli@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71278
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
