derek/gem5
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

misc: Minor release v23.0.1.0 (#174)

Browse Source
This commit is contained in:
Bobby R. Bruce
2023-08-11 02:08:41 -07:00
committed by GitHub
parent c8239cca6d 1b67297efc
commit 6835f06657
69 changed files with 2132 additions and 710 deletions
Download Patch File Download Diff File Expand all files Collapse all files

789
CONTRIBUTING.md
View File

@@ -1,539 +1,390 @@
If you've made changes to gem5 that might benefit others, we strongly encourage
you to contribute those changes to the public gem5 repository. There are
several reasons to do this:
* Share your work with others, so that they can benefit from new functionality.
* Support the scientific principle by enabling others to evaluate your
suggestions without having to guess what you did.
* Once your changes are part of the main repo, you no longer have to merge
them back in every time you update your local repo. This can be a huge time
saving!
* Once your code is in the main repo, other people have to make their changes
work with your code, and not the other way around.
* Others may build on your contributions to make them even better, or extend
them in ways you did not have time to do.
* You will have the satisfaction of contributing back to the community.
This document serves as a guide to contributing to gem5.
The following subsections outline, in order, the steps involved in contributing
to the gem5 project.
The main method for contributing code to gem5 is via our code review website:
https://gem5-review.googlesource.com/. This documents describes the details of
how to create code changes, upload your changes, have your changes
reviewed, and finally push your changes to gem5. More information can be found
from the following sources:
* http://gem5.org/contributing
* https://gerrit-review.googlesource.com/Documentation/index.html
* https://git-scm.com/book
## Determining what you can contribute
The easiest way to see how you can contribute to gem5 is to check our Jira
issue tracker: <https://gem5.atlassian.net> or GitHub issue tracker:
<https://github.com/gem5/gem5/issues>.
High-level flow for submitting changes
======================================
Browse these open issues and see if there are any which you are capable of
handling. When you find a task you are happy to carry out, verify no one else
is presently assigned, then leave a comment asking if you may assign yourself
this task. Though not mandatory, we
advise first-time contributors do this so developers more familiar with the
task may give advice on how best to implement the necessary changes.
+-------------+
| Make change |
+------+------+
|
|
v
+-------------+
| Run tests |<--------------+
+------+------+ |
| |
| |
v |
+------+------+ |
| Post review | |
+------+------+ |
| |
v |
+--------+---------+ |
| Wait for reviews | |
+--------+---------+ |
| |
| |
v |
+----+----+ No +------+------+
|Reviewers+--------->+ Update code |
|happy? | +------+------+
+----+----+ ^
| |
| Yes |
v |
+----+-----+ No |
|Maintainer+----------------+
|happy? |
+----+-----+
|
| Yes
v
+------+------+
| Submit code |
+-------------+
Once a developers has replied to your comment (and given any advice they may
have), you may officially assign yourself the task. This helps the gem5
development community understand which parts of the project are presently being
worked on.
After creating your change to gem5, you can post a review on our Gerrit
code-review site: https://gem5-review.googlesource.com. Before being able to
submit your code to the mainline of gem5, the code is reviewed by others in the
community. Additionally, the maintainer for that part of the code must sign off
on it.
**If, for whatever reason, you stop working on a task, please unassign
yourself from the task.**
Cloning the gem5 repo to contribute
===================================
## Obtaining the git repo
If you plan on contributing, it is strongly encouraged for you to clone the
repository directly, and checkout the `develop` branch from our gerrit instance
at https://gem5.googlesource.com/.
The gem5 git repository is hosted at <https://github.com/gem5/gem5>.
**Please note: contributions made to other gem5 repos
will not be considered. Please contribute to <https://github.com/gem5/gem5>
exclusively.**
To clone the gem5 repository:
To pull the gem5 git repo:
```
git clone https://gem5.googlesource.com/public/gem5
```sh
git clone https://github.com/gem5/gem5
```
By default, the stable branch is checked out. The stable branch contains the
latest released version of gem5. To obtain code still under-development (and
which contributions can be made):
If you wish to use gem5 and never contribute, this is fine. However, to
contribute, we use the [GitHub Pull-Request model](https://docs.github.com/en/pull-requests), and therefore recommend [Forking the gem5 repository](https://docs.github.com/en/get-started/quickstart/fork-a-repo) prior to contributing.
```
cd gem5
git checkout --track origin/develop
### Forking
Please consult the [GitHub documentation on Forking a GitHub repository](https://docs.github.com/en/get-started/quickstart/fork-a-repo).
As we will be working atop the `develop` branch, please ensure you Fork all the repository's branches, not just the `stable` branch.
This will create your own forked version of the gem5 repo on your own GitHub account.
You may then obtain it locally using:
```sh
git clone https://github.com/{your github account}/gem5
```
Changes should be made to this develop branch. Changes to the stable branch
will be blocked. Once a change on the develop branch is properly incorporated
into the gem5 repo it will be merged into the stable branch upon the next
release of gem5. New releases of gem5 occur three times a year. Ergo, changes
made to the develop branch should appear on the stable branch within three to
four months as part of a stable release.
### stable / develop branch
Other gem5 repositories
-----------------------
When cloned the git repo will have the `stable` branch checked-out by default. The
`stable` branch is the gem5 stable release branch. I.e., the HEAD
of this branch contains the latest stable release of gem5. (execute `git tag`
on the `stable` branch to see the list of stable releases. A particular
release may be checked out by executing `git checkout <release>`). As the
`stable` branch only contains officially released gem5 code **contributors
should not develop changes on top of the `stable` branch** they should instead
**develop changes on top of the `develop` branch**.
There are a few repositories other than the main gem5 development repository.
To switch to the `develop` branch:
* public/m5threads: The code for a pthreads implementation that works with
gem5's syscall emulation mode.
* public/gem5-resources: Resources to enable computer architecture research
with gem5. See the README.md file in the gem5-resources repository for more
information.
* public/gem5-website: The gem5.org website source. See the README.md file in
the gem5-website repository for more information.
```sh
git switch develop
```
Making changes to gem5
======================
The develop `branch` is merged into the `stable` branch upon a gem5 release.
Therefore, any changes you make exist on the develop branch until the next release.
It is strongly encouraged to use git branches when making changes to gem5.
Additionally, keeping changes small and concise and only have a single logical
change per commit.
We strongly recommend creating your own local branches to do changes.
The flow of development works best if `develop` and `stable` are not modified directly.
This helps keep your changes organized across different branches in your forked repository.
The following example will create a new branch, from `develop`, called `new-feature`:
Unlike our previous flow with Mercurial and patch queues, when using git, you
will be committing changes to your local branch. By using separate branches in
git, you will be able to pull in and merge changes from mainline and simply
keep up with upstream changes.
```sh
git switch -c new-feature
```
We use a rebase-always model for contributions to the develop branch of gem5.
In this model, the changes are rebased on top of the tip of develop instead of
merged. This means that to contribute, you will have to frequently rebase any
feature branches on top of develop. If you see a "merge conflict" in gerrit, it
can often be solved with a simple rebase. To find out more information about
rebasing and git, see the [git book].
## Making modifications
[git book]: https://git-scm.com/book/en/v2/Git-Branching-Rebasing
### C/CPP
Different tasks will require the project to be modified in different ways.
Though, in all cases, our style-guide must be adhered to. The full C/C++ style
guide is outlined [here](/documentation/general_docs/development/coding_style).
Setting up pre-commit
---------------------
As a high-level overview:
To help ensure the gem5 style guide is maintained, we use [pre-commit](
https://pre-commit.com) to run checks on changes to be contributed.
* Lines must not exceed 79 characters in length.
* There should be no trailing white-space on any line.
* Indentations must be 4 spaces (no tab characters).
* Class names must use upper camel case (e.g., `ThisIsAClass`).
* Class member variables must use lower camel case (e.g.,
`thisIsAMemberVariable`).
* Class member variables with their own public accessor must start with an
underscore (e.g., `_variableWithAccessor`).
* Local variables must use snake case (e.g., `this_is_a_local_variable`).
* Functions must use lower camel case (e.g., `thisIsAFunction`)
* Function parameters must use snake case.
* Macros must be in all caps with underscores (e.g., `THIS_IS_A_MACRO`).
* Function declaration return types must be on their own line.
* Function brackets must be on their own line.
* `for`/`if`/`while` branching operations must be followed by a white-space
before the conditional statement (e.g., `for (...)`).
* `for`/`if`/`while` branching operations' opening bracket must be on the
same line, with the closing bracket on its own line (e.g.,
`for (...) {\n ... \n}\n`). There should be a space between the condition(s)
and the opening bracket.
* C++ access modifies must be indented by two spaces, with method/variables
defined within indented by four spaces.
To setup pre-commit, run the following in your gem5 directory to install the
pre-commit and commit message hooks.
Below is a simple toy example of how a class should be formatted:
```C++
#DEFINE EXAMPLE_MACRO 7
class ExampleClass
{
private:
int _fooBar;
int barFoo;
public:
int
getFooBar()
{
return _fooBar;
}
int
aFunction(int parameter_one, int parameter_two)
{
int local_variable = 0;
if (true) {
int local_variable = parameter_one + parameter_two + barFoo
+ EXAMPLE_MACRO;
}
return local_variable;
}
}
```
### Python
We use [Python Black](https://github.com/psf/black) to format our Python code
to the correct style. To install:
```sh
pip install black
```
Then run on modified/added python files using:
```sh
black <files/directories>
```
For variable/method/etc. naming conventions, please follow the [PEP 8 naming
convention recommendations](
https://peps.python.org/pep-0008/#naming-conventions). While we try our best to
enforce naming conventions across the gem5 project, we are aware there are
instances where they are not. In such cases please **follow the convention
of the code you are modifying**.
### Using pre-commit
To help enforce our style guide we use use [pre-commit](
https://pre-commit.com). pre-commit is a git hook and, as such, must be
explicitly installed by a gem5 developer.
To install the gem5 pre-commit checks, execute the following in the gem5
directory:
```sh
pip install pre-commit
pre-commit install -t pre-commit -t commit-msg
pre-commit install
```
The hooks are also automatically installed when gem5 is compiled.
Once installed pre-commit will run checks on modified code prior to running the
`git commit` command (see [our section on committing](#committing) for more
details on committing your changes). If these tests fail you will not be able to
commit.
When you run a `git commit` command the pre-commit hook will run checks on your
committed code. The commit will be blocked if a check fails.
These same pre-commit checks are run as part our CI checks (those
which must pass in order for a change to be merged into the develop branch). It
is therefore strongly recommended that developers install pre-commit to catch
style errors early.
The same checks are run as part of Gerrit's CI tests (those required to obtain
a Verified label, necessary for a change to be accepted to the develop branch).
Therefore setting up pre-commit in your local gem5 development environment is
recommended.
## Compiling and running tests
You can automatically format files to pass the pre-commit tests by running:
The minimum criteria for a change to be submitted is that the code is
compilable and the test cases pass.
The following command both compiles the project and runs our "quick"
system-level checks:
```sh
pre-commit run --files <files to format>
cd tests
./main.py run
```
Requirements for change descriptions
------------------------------------
To help reviewers and future contributors more easily understand and track
changes, we require all change descriptions be strictly formatted.
**Note: These tests can take several hours to build and execute. `main.py` may
be run on multiple threads with the `-j` flag. E.g.: `python main.py run
-j6`.**
A canonical commit message consists of three parts:
* A short summary line describing the change. This line starts with one or
more keywords (found in the MAINTAINERS file) separated by commas followed
by a colon and a description of the change. This short description is
written in the imperative mood, and should say what happens when the patch
is applied. Keep it short and simple. Write it in sentence case preferably
not ending in a period. This line should be no more than 65 characters long
since version control systems usually add a prefix that causes line-wrapping
for longer lines.
* (Optional, but highly recommended) A detailed description. This describes
what you have done and why. If the change isn't obvious, you might want to
motivate why it is needed. Lines need to be wrapped to 72 characters or
less. Leave a blank line between the first short summary line and this
detailed description.
* Tags describing patch metadata. You are highly recommended to use
tags to acknowledge reviewers for their work. Gerrit will automatically add
most tags.
The unit tests should also pass. To run the unit tests:
Tags are an optional mechanism to store additional metadata about a patch and
acknowledge people who reported a bug or reviewed that patch. Tags are
generally appended to the end of the commit message in the order they happen.
We currently use the following tags:
* Signed-off-by: Added by the author and the submitter (if different).
This tag is a statement saying that you believe the patch to be correct and
have the right to submit the patch according to the license in the affected
files. Similarly, if you commit someone else's patch, this tells the rest
of the world that you have have the right to forward it to the main
repository. If you need to make any changes at all to submit the change,
these should be described within hard brackets just before your
Signed-off-by tag. By adding this line, the contributor certifies the
contribution is made under the terms of the Developer Certificate of Origin
(DCO) [https://developercertificate.org/].
* Reviewed-by: Used to acknowledge patch reviewers. It's generally considered
good form to add these. Added automatically.
* Reported-by: Used to acknowledge someone for finding and reporting a bug.
* Reviewed-on: Link to the review request corresponding to this patch. Added
automatically.
* Change-Id: Used by Gerrit to track changes across rebases. Added
automatically with a commit hook by git.
* Tested-by: Used to acknowledge people who tested a patch. Sometimes added
automatically by review systems that integrate with CI systems.
* Issue-On: Used to link a commit to an issue in gem5's [issue tracker]. The
format should be https://gem5.atlassian.net/browse/GEM5-<NUMBER>
[issue tracker]: https://gem5.atlassian.net/
Other than the "Signed-off-by", "Issue-On", "Reported-by", and "Tested-by"
tags, you generally don't need to add these manually as they are added
automatically by Gerrit.
It is encouraged for the author of the patch and the submitter to add a
Signed-off-by tag to the commit message. By adding this line, the contributor
certifies the contribution is made under the terms of the Developer Certificate
of Origin (DCO) [https://developercertificate.org/].
If your change relates to a [Jira Issue](https://gem5.atlassian.net), it is
advised that you provide a link to the issue in the commit message (or messages
if the Jira Issue relates to multiple commits). Though optional, doing this
can help reviewers understand the context of a change.
It is imperative that you use your real name and your real email address in
both tags and in the author field of the changeset.
For significant changes, authors are encouraged to add copyright information
and their names at the beginning of the file. The main purpose of the author
names on the file is to track who is most knowledgeable about the file (e.g.,
who has contributed a significant amount of code to the file). The
`util/update-copyright.py` helper script can help to keep your copyright dates
up-to-date when you make further changes to files which already have your
copyright but with older dates.
Note: If you do not follow these guidelines, the gerrit review site will
automatically reject your patch.
If this happens, update your changeset descriptions to match the required style
and resubmit. The following is a useful git command to update the most recent
commit (HEAD).
```
git commit --amend
```sh
scons build/NULL/unittests.opt
```
Running tests
=============
To compile an individual gem5 binary:
Before posting a change to the code review site, you should always run the
quick tests!
See TESTING.md for more information.
Posting a review
================
If you have not signed up for an account on the Gerrit review site
(https://gem5-review.googlesource.com), you first have to create an account.
Setting up an account
---------------------
1. Go to https://gem5.googlesource.com/
2. Click "Sign In" in the upper right corner. Note: You will need a Google
account to contribute.
3. After signing in, click "Generate Password" and follow the instructions.
Submitting a change
-------------------
In gerrit, to submit a review request, you can simply push your git commits to
a special named branch. For more information on git push see
https://git-scm.com/docs/git-push.
There are three ways to push your changes to gerrit.
Push change to gerrit review
----------------------------
```
git push origin HEAD:refs/for/develop
```sh
scons build/ALL/gem5.opt
```
Assuming origin is https://gem5.googlesource.com/public/gem5 and you want to
push the changeset at HEAD, this will create a new review request on top of the
develop branch. More generally,
This compiles a gem5 binary containing "ALL" ISA targets. For more information
on building gem5 please consult our [building documentation](
/documentation/general_docs/building).
```
git push <gem5 gerrit instance> <changeset>:refs/for/<branch>
## Committing
When you feel your change is done, you may commit. Start by adding the changed
files:
```Shell
git add <changed files>
```
See https://gerrit-review.googlesource.com/Documentation/user-upload.html for
more information.
Make sure these changes are being added to your forked repository.
Then commit using:
Pushing your first change
--------------------------
The first time you push a change you may get the following error:
```
remote: ERROR: [fb1366b] missing Change-Id in commit message footer
...
```Shell
git commit
```
Within the error message, there is a command line you should run. For every new
clone of the git repo, you need to run the following command to automatically
insert the change id in the the commit (all on one line).
The commit message must adhere to our style. The first line of the commit is
the "header". The header starts with a tag (or tags, separated by a comma),
then a colon. Which tags are used depend on which components of gem5
you have modified. **Please refer to the [MAINTAINERS.yaml](
https://github.com/gem5/gem5/blob/stable/MAINTAINERS.yaml) for
a comprehensive list of accepted tags**. After this colon a short description
of the commit must be provided. **This header line must not exceed 65
characters**.
After this, a more detailed description of the commit can be added. This is
inserted below the header, separated by an empty line. Including a description
is optional but it's strongly recommended. The description may span multiple
lines, and multiple paragraphs. **No line in the description may exceed 72
characters.**
To improve the navigability of the gem5 project we would appreciate if commit
messages include a link to the relevant Jira issue/issues.
Below is an example of how a gem5 commit message should be formatted:
```
curl -Lo `git rev-parse --git-dir`/hooks/commit-msg \
https://gerrit-review.googlesource.com/tools/hooks/commit-msg ; \
chmod +x `git rev-parse --git-dir`/hooks/commit-msg
test,base: This commit tests some classes in the base component
This is a more detailed description of the commit. This can be as long
as is necessary to adequately describe the change.
A description may spawn multiple paragraphs if desired.
Jira Issue: https://gem5.atlassian.net/browse/GEM5-186
```
If you receive the above error, simply run this command and then amend your
changeset.
If you feel the need to change your commit, add the necessary files then
_amend_ the changes to the commit using:
```
git commit --amend
```sh
git commit --amend
```
Push change to gerrit as a Work In Progress
-------------------------------------------
This will give you opportunity to edit the commit message.
It is acceptable to push commits as "Work In Progress" (WIP) changes within
gerrit. WIP changes are publicly visible though no one will be able to review
the changes or be directly notified they have been submitted. WIP changes can
be useful for backing up code currently under-development or for sharing
incomplete code with the wider community (i.e., the link to the gerrit change
may be shared, and others may download the change, comment on it, and track
alterations over time).
You may continue to add more commits as a chain of commits to be included in the pull-request.
However, we recommend that pull-requests are kept small and focused.
For example, if you wish to add a different feature or fix a different bug, we recommend doing so in another pull requests.
See https://gerrit-review.googlesource.com/Documentation/intro-user.html#wip
for details on WIP gerrit changes.
## Keeping your forked and local repositories up-to-date
To push a change as a WIP:
While working on your contribution, we recommend keeping your forked repository in-sync with the source gem5 repository.
To do so, regularly [Sync your fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork).
This can be done via the GitHub web interface and, if so, you should `git pull` on top of your local `stable` and `develop` branches to ensure your local repository is in-sync.
To do so from the command line:
```
git push origin HEAD:refs/for/develop%wip
```sh
# Add the main gem5 repository as a remote on your local repository. This only
# needs done once.
git remote add upstream https://github.com/gem5/gem5.git
git fetch upstream # Obtain the latest from the gem5 repo.
git switch develop # Switch to the develop branch.
git merge upstream/develop # Merge the latest changes into the develop branch.
git push # Push to develop to your forked repo.
git switch stable # Switch to the stable branch.
git merge upstream/stable # Merge the latest changes into the stable branch.
git push # Push the changes to stable to your forked repo.
```
Once you have pushed your change as a WIP, you can log onto [gerrit](
https://gem5-review.googlesource.com) and view it. Once you're happy with the
change you can add reviewers which shall move your change from WIP status
to be considered for submission by the wider gem5 community. Switching from a
WIP to a regular change does not notify the gem5 community, via the gem5-dev
mailing-list, that a change has been submitted (as would occur if a change were
submitted directly for review). It is therefore important to include reviewers
and CC those who you wish to view the change (they will be notified
automatically via email).
As our local branch work atop the `develop` branch, once we've synced our forked repository, we can rebase our local branch on top of the `develop` branch.
Assuming our local branch is called `new-feature`:
Push change bypassing gerrit
-----------------------------
Only maintainers can bypass gerrit review. This should very rarely be used.
```
git push origin HEAD:refs/heads/develop
```sh
git switch develop # Switching back to the develop branch.
git pull # Ensuring we have the latest from the forked repository.
git switch new-feature # Switching back to our local branch.
git rebase develop # Rebasing our local branch on top of the develop branch.
```
Other gerrit push options
-------------------------
Conflicts may need resolved between your branch and new changes.
There are a number of options you can specify when uploading your changes to
gerrit (e.g., reviewers, labels). The gerrit documentation has more
information.
https://gerrit-review.googlesource.com/Documentation/user-upload.html
## Pushing and creating a pull request
Branches
========
Once you have completed your changes locally, you can push to your forked gem5 repository.
Assuming the branch we are working on is `new-feature`:
By default, contributions to gem5 should be made on the develop branch. The
stable branch is maintained as a stable release branch (i.e., it can be pulled
to obtain the latest official release of gem5). Creation of additional branches
is generally discouraged due to their tendency to bloat git repositories with
abandoned code. However, the creation of new branches is permitted for
development of a specific feature or improvement if one or more of the
following criteria are met:
1. The feature/improvement is likely to be of a large size, consisting of many
commits, with little logic in these commits being contributed separately.
2. The feature/improvement will be developed over a long period of time.
3. There is sufficient reason that a feature/improvement should not be part
of the next gem5 release (e.g., the change should be held within a feature
branch until ready for the next release, at which point it will be merged
into the develop branch).
If a branch is required it can only be created by a project maintainer.
Therefore, if a gem5 contributor desires a separate branch for their work, they
should request one from the maintainer of the component the work relates to
(see MAINTAINERS for the list of maintainers and the components they are
responsible for). **The maintainer shall use their discretion to determine
whether the creation of a branch is necessary**. If approved, the maintainer
shall create the branch which the contributor may then use.
Development on a branch within Gerrit functions in exactly the same way as
contributing to the develop branch. When contributors to a branch are
satisfied, they should create a merge commit into the develop branch. The
maintainer should then be notified that the branch they created can now be
deleted.
**Abandonment of changes within branches may result in these branches being
removed from the repository. All branches within a repo should be under active
development.**
Reviewing patches
=================
Reviewing patches is done on our gerrit instance at
https://gem5-review.googlesource.com/.
After logging in with your Google account, you will be able to comment, review,
and push your own patches as well as review others' patches. All gem5 users are
encouraged to review patches. The only requirement to review patches is to be
polite and respectful of others.
There are multiple labels in Gerrit that can be applied to each review detailed
below.
* Code-review: This is used by any gem5 user to review patches. When reviewing
a patch you can give it a score of -2 to +2 with the following semantics.
* -2: This blocks the patch. You believe that this patch should never be
committed. This label should be very rarely used.
* -1: You would prefer this is not merged as is
* 0: No score
* +1: This patch seems good, but you aren't 100% confident that it should be
pushed.
* +2: This is a good patch and should be pushed as is.
* Maintainer: Currently only PMC members are maintainers. At least one
maintainer must review your patch and give it a +1 before it can be merged.
* Verified: This is automatically generated from the continuous integrated
(CI) tests. Each patch must receive at least a +1 from the CI tests before
the patch can be merged. The patch will receive a +1 if gem5 builds and
runs, and it will receive a +2 if the stats match.
* Style-Check: This is automatically generated and tests the patch against the
gem5 code style
(http://www.gem5.org/documentation/general_docs/development/coding_style/).
The patch must receive a +1 from the style checker to be pushed.
Note: Whenever the patch creator updates the patch all reviewers must re-review
the patch. There is no longer a "Fix it, then Ship It" option.
Once you have received reviews for your patch, you will likely need to make
changes. To do this, you should update the original git changeset. Then, you
can simply push the changeset again to the same Gerrit branch to update the
review request.
```
git push origin HEAD:refs/for/develop
```sh
git switch new-feature # Ensure we are on the 'new-feature' branch.
git push --set-upstream origin new-feature
```
Committing changes
==================
Now, via the GitHub web interface, you can [create a pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) of your changes from your forked repository's branch into the gem5 `develop` branch.
Each patch must meet the following criteria to be merged:
* At least one review with +2
* At least one maintainer with +1
* At least +1 from the CI tests (gem5 must build and run)
* At least +1 from the style checker
## Passing the checks
Once a patch meets the above criteria, the submitter of the patch will be able
to merge the patch by pressing the "Submit" button on Gerrit. When the patch is
submitted, it is merged into the public gem5 branch.
Once you have created a pull request, the gem5 Continuous Integration (CI) tests will run.
These run a series of checks to ensure your changes are valid.
These must pass before your changes can be merged into the gem5 `develop` branch.
Review moderation and guidelines
--------------------------------
In addition to the CI tests, your changes will be reviewed by the gem5 community.
Your pull-request must have the approval of at least one community member prior to being merged.
Once a change is submitted, reviewers shall review the change. This may require
several iterations before a merge. Comments from reviewers may include
questions, and requests for alterations to the change prior to merging. The
overarching philosophy in managing this process is that there should be
politeness and clear communication between all parties at all times, and,
whenever possible, permission should be asked before doing anything that may
inconvenience another party. Included below are some guidelines we expect
contributors and reviewers to follow.
Once your pull-request has passed all the CI tests and has been approved by at least one community member, it will be merged a gem5 maintainer will do a [Merge](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/about-pull-request-merges) on the pull-request.
The gem5 maintainers are individuals granted the ability to merge pull requests into the gem5 `develop` branch.
* In all forms of communication, contributors and reviewers must be polite.
Comments seen as being needlessly hostile or dismissive will not be
tolerated.
* Change contributors should respond to, or act upon, each item of feedback
given by reviewers. If there is disagreement with a piece of
feedback, a sufficiently detailed reason for this disagreement should
be given. Polite discussion, and sharing of information and expertise
is strongly encouraged.
* Contributors are advised to assign reviewers when submitting a change.
Anyone who contributes to gem5 can be assigned as a reviewer. However,
all changes must be accepted by at least one maintainer prior to a
merge, ergo assigning of at least one maintainer as a reviewer is
strongly recommended. Please see MAINTAINERS for a breakdown of
gem5 maintainers and which components they claim responsibility for.
Maintainers should be chosen based on which components the change is
targeting. Assigning of reviewers is not strictly enforced, though not
assigning reviewers may slow the time in which a change is reviewed.
* If a contributor posts a change and does not receive any reviews after two
working days (excluding regional holidays), it is acceptable to "prod"
reviewers. This can be done by adding a reply to the changeset review
(e.g., "Would it be possible for someone to review my change?"). If the
contributor has yet to assign reviewers, they are strongly advised to do so.
Reviewers will get notified when assigned to referee a change.
* By default, the original contributor is assumed to own a change. I.e.,
they are assumed to be the sole party to submit patchsets. If someone
other than the original contributor wishes to submit patchsets to a
change on the original contributor's behalf, they should first ask
permission. If two working days pass without a response, a patchset may be
submitted without permission. Permission does not need to be asked to submit
a patchset consisting of minor, inoffensive, changes such a typo and format
fixes.
* Once a change is ready to merge, it enters a "Ready to Submit" state. The
original contributor should merge their change at this point, assuming they
are content with the commit in its present form. After two working days, a
reviewer may message a contributor to remind them of the change being in a
"Ready to Submit" state and ask if they can merge the change on the
contributors behalf. If a further two working days elapse without a
response, the reviewer may merge without permission. A contributor may keep
a change open for whatever reason though this should be communicated to the
reviewer when asked.
* After a month of inactivity from a contributor on an active change, a
reviewer may post a message on the change reminding the submitter, and
anyone else watching the change, of its active status and ask if they are
still interested in eventually merging the change. After two weeks of no
response the reviewer reserves the right to abandon the change under the
assumption there is no longer interest.
* The final arbiter in any dispute between reviewers and/or contributors
is the PMC (PMC members are highlighted in MAINTAINERS). Disputes requiring
intervention by the PMC are undesirable. Attempts should be made to resolve
disagreements via respectful and polite discourse before being escalated to
this level.
Releases
========
### Making iterative improvements based on feedback
A reviewer will ask questions and post suggestions on GitHub. You should read
these comments and answer these questions. **All communications between
reviewers and contributors should be done in a polite manner. Rude and/or
dismissive remarks will not be tolerated.**
When you understand what changes are required make amendments to the pull
request by adding patches to the same branch and then pushing to the forked repository.
A git "force push" (i.e., `git push --force`) is also acceptable if you wish to alter the commits locally in order to make the changes.
We encourage contributors to help keep our `git log` clean and readable.
We recommend that users rebase their changes frequently on top of the develop branch, squash their commits where appropriate (e.g., in cases where there are many small fix commits to a change in the same PR) then force push changes to keep their PR commits concise.
Once pushed to the forked repository, the pull request will automatically update with your changes.
The reviewer will then re-review your changes and, if necessary, ask for further changes, or approve your pull-request.
## Reviewing other contributions
We encourage all gem5 developers to review other's contributions.
Anyone may review a gem5 change and, if they feel it is ready, approve it.
All pull-requests can be found at <https://github.com/gem5/gem5/pulls>.
When reviewing a pull request we enforce the followings guidelines.
These have been designed to ensure clear and polite communication between all parties:
* In all forms of communication, contributors and reviewers must be polite.
Comments seen as being rude or dismissive will not be tolerated.
* If choosing to not approve a PR, please state clearly why.
When asking for changes, the commits should be specific and actionable.
General criticisms which cannot be addressed or understood by the contributor are unhelpful.
If the contribution needs improvement, reviewers should state what their requested changes are.
If more information is needed for the reviewers to make a decision the reviewer should ask clear questions.
If the PR is generally not seen as a worthwhile contribution, a good justification should be given so the contributor may fairly rebuttal.
* By default, the original contributor is assumed to own a change.
I.e., they are assumed to be the sole party to submit patches to the pull request.
If someone other than the original contributor wishes to submit patches on the original contributors behalf they should first ask permission.
Pull requests which appear abandoned may be adopted by a new contributor as long as there is good enough reason to assume the original contributor is no longer working on the pull request.
* Maintainers have the final say on whether a change is merged.
Your review will be taken into account by the maintainer.
It is expected, in all but the most extreme cases, that the reviewer's concerns must be addressed and for the reviewer to approve the the contribution prior to the maintainer merging the pull request.
We also recommend consulting Google's ["How to write code review comments"](https://google.github.io/eng-practices/review/reviewer/comments.html) for advice on giving feedback to contributors.
## Releases
gem5 releases occur 3 times per year. The procedure for releasing gem5 is as
follows:
@@ -549,8 +400,8 @@ gem5-dev mailing list will be notified that the staging branch will be merged
into the stable branch after two weeks, thus marking the new release.
3. The staging branch will have the full suite of gem5 tests run on it to
ensure all tests pass and the to-be-released code is in a decent state.
4. If a user submits a changeset to the staging branch, it will be considered
and undergo the standard Gerrit review process. However, only alterations that
4. If a user submits a pull request to the staging branch, it will be considered
and undergo the standard github review process. However, only alterations that
cannot wait until the following release will be accepted for submission into
the branch (i.e., submissions to the staging branch for "last minute"
inclusions to the release should be of a high priority, such as a critical bug
@@ -558,8 +409,8 @@ fix). The project maintainers will use their discretion in deciding whether a
change may be submitted directly to the staging branch. All other submissions
to gem5 will continue to be made to the develop branch. Patches submitted
into the staging branch do not need to be re-added to the develop branch.
5. Once signed off by members of the PMC the staging branch shall be merged
into the stable and develop branch. The staging branch will then be deleted.
5. Once the staging branch has been deemed ready for release, the [release procedures](https://www.gem5.org/documentation/general_docs/development/release_procedures/) will be carried out.
This will end with the staging branch being merged into the stable branch.
6. The stable branch shall be tagged with the correct version number for that
release. gem5 conforms to a "v{YY}.{MAJOR}.{MINOR}.{HOTFIX}" versioning system.
E.g., the first major release of 2022 will be "v22.0.0.0", followed by
@@ -569,8 +420,16 @@ the minor release numbers in case this policy changes in the future.
7. The gem5-dev and gem5-user mailing lists shall be notified of the new gem5
release.
Hotfixes
--------
### Exemptions
Due to limitations with GitHub we may update the ".github" directory in the gem5 repo's `stable` branch between gem5 releases.
This is due to certain processes carried out by the GitHub Actions infrastructure which rely on configurations being present on a repository's primary branch.
As the files in ".github" only influence the functionality of our GitHub actions and other GitHub activities, updating these files does not change the functionality of the gem5 in way.
It is therefore safe to do this.
Despite this exemption to our normal procedure we aim to ensure that **the ".github" directory on the `stable` is never "ahead" of that in the `develop` branch**.
Therefore contributors who wish to update files in ".github" should submit their changes to `develop` and then request their changes to be applied to the `stable` branch.
### Hotfixes
There may be circumstances in which a change to gem5 is deemed critical and
cannot wait for an official release (e.g., a high-priority bug fix). In these
@@ -585,7 +444,7 @@ permitted, the following steps will be taken:
1. A new branch with the prefix "hotfix-" will be created from the stable
branch. Only gem5 maintainers can create branches. If a non-maintainer requires
the creation of a hotfix branch then they should contact a gem5 maintainer.
2. The change shall be submitted to the hotfix branch via gerrit. Full review,
2. The change shall be submitted to the hotfix branch via github. Full review,
as with any other change, will be required.
3. Once fully submitted, the hotfix branch shall be merged into both the
develop and the stable branch by a gem5 maintainer.

43
README
View File

@@ -1,43 +0,0 @@
This is the gem5 simulator.
The main website can be found at http://www.gem5.org
A good starting point is http://www.gem5.org/about, and for
more information about building the simulator and getting started
please see http://www.gem5.org/documentation and
http://www.gem5.org/documentation/learning_gem5/introduction.
To build gem5, you will need the following software: g++ or clang,
Python (gem5 links in the Python interpreter), SCons, zlib, m4, and lastly
protobuf if you want trace capture and playback support. Please see
http://www.gem5.org/documentation/general_docs/building for more details
concerning the minimum versions of these tools.
Once you have all dependencies resolved, type 'scons
build/<CONFIG>/gem5.opt' where CONFIG is one of the options in build_opts like
ARM, NULL, MIPS, POWER, SPARC, X86, Garnet_standalone, etc. This will build an
optimized version of the gem5 binary (gem5.opt) with the the specified
configuration. See http://www.gem5.org/documentation/general_docs/building for
more details and options.
The main source tree includes these subdirectories:
- build_opts: pre-made default configurations for gem5
- build_tools: tools used internally by gem5's build process.
- configs: example simulation configuration scripts
- ext: less-common external packages needed to build gem5
- include: include files for use in other programs
- site_scons: modular components of the build system
- src: source code of the gem5 simulator
- system: source for some optional system software for simulated systems
- tests: regression tests
- util: useful utility programs and files
To run full-system simulations, you may need compiled system firmware, kernel
binaries and one or more disk images, depending on gem5's configuration and
what type of workload you're trying to run. Many of those resources can be
downloaded from http://resources.gem5.org, and/or from the git repository here:
https://gem5.googlesource.com/public/gem5-resources/
If you have questions, please send mail to gem5-users@gem5.org
Enjoy using gem5 and please share your modifications and extensions.

90
README.md Normal file
View File

@@ -0,0 +1,90 @@
# The gem5 Simulator
This is the repository for the gem5 simulator. It contains the full source code
for the simulator and all tests and regressions.
The gem5 simulator is a modular platform for computer-system architecture
research, encompassing system-level architecture as well as processor
microarchitecture. It is primarily used to evaluate new hardware designs,
system software changes, and compile-time and run-time system optimizations.
The main website can be found at <http://www.gem5.org>.
## Getting started
A good starting point is <http://www.gem5.org/about>, and for
more information about building the simulator and getting started
please see <http://www.gem5.org/documentation> and
<http://www.gem5.org/documentation/learning_gem5/introduction>.
## Building gem5
To build gem5, you will need the following software: g++ or clang,
Python (gem5 links in the Python interpreter), SCons, zlib, m4, and lastly
protobuf if you want trace capture and playback support. Please see
<http://www.gem5.org/documentation/general_docs/building> for more details
concerning the minimum versions of these tools.
Once you have all dependencies resolved, execute
`scons build/ALL/gem5.opt` to build an optimized version of the gem5 binary
(`gem5.opt`) containing all gem5 ISAs. If you only wish to compile gem5 to
include a single ISA, you can replace `ALL` with the name of the ISA. Valid
options include `ARM`, `NULL`, `MIPS`, `POWER`, `SPARC`, and `X86` The complete
list of options can be found in the build_opts directory.
See https://www.gem5.org/documentation/general_docs/building for more
information on building gem5.
## The Source Tree
The main source tree includes these subdirectories:
* build_opts: pre-made default configurations for gem5
* build_tools: tools used internally by gem5's build process.
* configs: example simulation configuration scripts
* ext: less-common external packages needed to build gem5
* include: include files for use in other programs
* site_scons: modular components of the build system
* src: source code of the gem5 simulator. The C++ source, Python wrappers, and Python standard library are found in this directory.
* system: source for some optional system software for simulated systems
* tests: regression tests
* util: useful utility programs and files
## gem5 Resources
To run full-system simulations, you may need compiled system firmware, kernel
binaries and one or more disk images, depending on gem5's configuration and
what type of workload you're trying to run. Many of these resources can be
obtained from <https://resources.gem5.org>.
More information on gem5 Resources can be found at
<https://www.gem5.org/documentation/general_docs/gem5_resources/>.
## Getting Help, Reporting bugs, and Requesting Features
We provide a variety of channels for users and developers to get help, report
bugs, requests features, or engage in community discussions. Below
are a few of the most common we recommend using.
* **GitHub Discussions**: A GitHub Discussions page. This can be used to start
discussions or ask questions. Available at
<https://github.com/orgs/gem5/discussions>.
* **GitHub Issues**: A GitHub Issues page for reporting bugs or requesting
features. Available at <https://github.com/gem5/gem5/issues>.
* **Jira Issue Tracker**: A Jira Issue Tracker for reporting bugs or requesting
features. Available at <https://gem5.atlassian.net/>.
* **Slack**: A Slack server with a variety of channels for the gem5 community
to engage in a variety of discussions. Please visit
<https://www.gem5.org/join-slack> to join.
* **gem5-users@gem5.org**: A mailing list for users of gem5 to ask questions
or start discussions. To join the mailing list please visit
<https://www.gem5.org/mailing_lists>.
* **gem5-dev@gem5.org**: A mailing list for developers of gem5 to ask questions
or start discussions. To join the mailing list please visit
<https://www.gem5.org/mailing_lists>.
## Contributing to gem5
We hope you enjoy using gem5. When appropriate we advise charing your
contributions to the project. <https://www.gem5.org/contributing> can help you
get started. Additional information can be found in the CONTRIBUTING.md file.

26
RELEASE-NOTES.md
View File

@@ -1,3 +1,29 @@
# Version 23.0.1.0
This minor release incorporates documentation updates, bug fixes, and some minor improvements.
## Documentation updates
* "TESTING.md" has been updated to more accurately reflect our current testing infrastructure.
* "README" has been replaced with "README.md" and includes more up-to-date information on using gem5.
* "CONTRIBUTING.md" has been updated to reflect our migration to GitHub and the changes in policy and proceedures.
* Where needed old references to Gerrit have been removed in favor of GitHub.
## Bug Fixes
* Fixes an assert failure when using ARM which was trigged when `shiftAmt` is 0 for a UQRSH instruction.
* Fixes `name 'fatal' is not defined` being thrown when tracing is off.
* Fixes a bug in ARM in which the TLBIOS instructions were decoded as normal MSR instructions with no effect on the TLBs.
* Fixes invalid `packet_id` value in flit.
* Fixes default CustomMesh for use with Garnet.
## Minor Improvements
* The gem5 resources downloader now outputs more helpful errors in the case of a failure.
* "util/github-runners-vagrant" has been added. This outlines how to setup a GitHub Action's set-hosted runner for gem5.
* The PyUnit tests have been refactored to no longer download large resources during testing.
* Using Perf is now optional when utilizing KVM CPUs.
# Version 23.0.0.1
**[HOTFIX]** Fixes compilation of `GCN3_X86` and `VEGA_X85`.

172
TESTING.md
View File

@@ -7,9 +7,9 @@ gem5's testing infrastructure has the following goals:
* Fast execution in the simple case
* High coverage of gem5 code
# Running unit tests
## Running the CPP unit tests
gem5 comes with unit tests, created using the Google Test framework. These can
gem5 comes with unit tests for CPP, created using the Google Test framework. These can
be built through SCons.
To build and run all the unit tests:
@@ -18,8 +18,8 @@ To build and run all the unit tests:
scons build/ALL/unittests.opt
```
All unit tests should be run prior to posting a patch to
https://gem5-review.googlesource.com
All unit tests should be run prior to creating a pull request at
https://github.com/gem5/gem5/pulls/
To compile and run just one set of tests (e.g. those declared within
`src/base/bitunion.test.cc`):
@@ -41,9 +41,30 @@ To run a specific test function (e.g., BitUnionData.NormalBitfield):
./build/ALL/base/bitunion.test.opt --gtest_filter=BitUnionData.NormalBitfield
```
# Running system-level tests
## Running the Python unit tests
Within the `tests` directory we have system-level tests. These tests run
gem5 comes with Python unit tests.
These are built using the [Python unit testing framework](https://docs.python.org/3/library/unittest.html).
These tests can be found in "tests/gem5/pyunit".
To run these tests a gem5 binary must first be compiled.
We recommend, `build/ALL/gem5.opt`:
```sh
scons build/ALL/gem5.opt -j {number of compilation threads}
```
Then the Pyunit tests may be executed using:
```sh
./build/ALL/gem5.opt tests/run_pyunit.py
```
**Note**: These tests are also run via the 'quick' system-level tests, explained below.
## Running system-level tests
Within the "tests/gem5" directory we have system-level tests. These tests run
the gem5 framework against various hardware configurations, with different
ISAs, then verify the simulations execute correctly. These should be seen as
high-level, coarse-grained tests to compliment the unit-tests.
@@ -60,8 +81,8 @@ cd tests
./main.py run
```
The above is the *minumum* you should run before posting a patch to
https://gem5-review.googlesource.com
The above is the *minumum* you should run before posting a pull request to
https://github.com/gem5/gem5/pulls/
## Running tests from multiple directories
@@ -77,56 +98,37 @@ arguments:
This will load every test in directory1 and directory2 (and their
subdirectories).
## Specifying a subset of tests to run
### 'quick', 'long', and 'very-long' tests
You can use the tag query interface to specify the exact tests you want to run.
For instance, if you want to run only with `gem5.opt`, you can use
There are three categoties of tests which may be run from the "tests" directory:
```shell
./main.py run --variant opt
1. **'quick' tests**. This suite of tests are designed to finish execution in a few hours, inclusive of compilation of gem5.
We run these as part of our continuous integration tests on pull requests made to our repository.
These tests all utilize a binary build `scons build/ALL/gem5.opt`, and thus only rely on a single compilation for the tests to run.
2. **'long' tests**. This suite of tests are designed to finish execution in around 12 hours.
They incorporate longer running tests which are unsuitable to run as part of the 'quick' tests.
We run these daily via a scheduled job.
3. **'very-long' tests**. This suite of tests are designed to finish execution in days.
They incorporate tests which are too long to run frequntly
We run these daily via a scheduled job.
When executing `./main.py run` the 'quick' tests are executed.
To run the 'long' tests execute:
```sh
./main.py run --length=long
```
Or, if you want to just run quick tests with the `gem5.opt` binary:
and to run the 'very-long' tests execute:
```shell
./main.py run --length quick --variant opt
```sh
./main.py run --length=very-long
```
In most cases we recommend running the 'quick' tests for most changes.
Only in some cases, such as contributions which significantly change the codebase, do we recommend running the 'long' or 'very-long' suite.
To view all of the available tags, use
```shell
./main.py list --all-tags
```
The output is split into tag *types* (e.g., isa, variant, length) and the
tags for each type are listed after the type name.
Note that when using the isa tag type, tests were traditionally sorted based
on what compilation it required. However, as tests have switched to all be
compiled under the ALL compilation, which includes all ISAs so one doesn't
need to compile each one individually, using the isa tag for ISAs other than
ALL has become a less optimal way of searching for tests. It would instead
be better to run subsets of tests based on their directories, as described
above.
You can specify "or" between tags within the same type by using the tag flag
multiple times. For instance, to run everything that is tagged "opt" or "fast"
use
```shell
./main.py run --variant opt --variant fast
```
You can also specify "and" between different types of tags by specifying more
than one type on the command line. For instance, this will only run tests with
both the "ALL" and "opt" tags.
```shell
./main.py run --isa All --variant opt
```
## Running tests in batch
### Running tests in batch
The testing infrastructure provides the two needed methods to run tests in
batch. First, you can list all of the tests based on the same tags as above in
@@ -160,7 +162,7 @@ run more than one uid, you must call `./main.py` multiple times.
Currently, you must specify `--skip-build` if you want to run a single suite or
run in batch mode. Otherwise, you will build gem5 for all architectures.
## Rerunning failed tests
### Rerunning failed tests
While developing software a common practice is to run tests, make a change, and
assert that the tests still pass. If tests fail you'll likely want to
@@ -178,7 +180,7 @@ using the `rerun` command.
./main.py rerun
```
## If something goes wrong
### If something goes wrong
The first step is to turn up the verbosity of the output using `-v`. This will
allow you to see what tests are running and why a test is failing.
@@ -186,7 +188,7 @@ allow you to see what tests are running and why a test is failing.
If a test fails, the temporary directory where the gem5 output was saved is kept
and the path to the directory is printed in the terminal.
## Debugging the testing infrastructure
### Debugging the testing infrastructure
Every command takes an option for the verbosity. `-v`, `-vv`, `-vvv` will
increase the verbosity level. If something isn't working correctly, you can
@@ -197,7 +199,7 @@ contains the base code for tests, suites, fixtures, etc. The code in tests/gem5
is *gem5-specific* code. For the most part, the code in tests/gem5 extends the
structures in ext/testlib.
## Common errors
### Common errors
You may see a number of lines of output during test discovery that look like
the following:
@@ -213,45 +215,7 @@ test library executes each python file it finds searching for tests. It's okay
if the file causes an exception. This means there are no tests in that file
(e.g., it's not a new-style test).
## Binary test applications
The code for some test binaries that are run in the gem5 guest during
testing can be found in `tests/test-progs`.
There's one directory per test application.
The source code is under the `source` directory.
You may have a `bin` directory as well.
The `bin` directory is automatically created when running the test case that
uses the test binary.
This is not the case when a test is run via the --bin-path option.
In that scenario a bin directory will be created in the selected path
rather than in `tests/test-progs`.
The binary is downloaded from the gem5 servers the first
time it is referenced by a test.
Some other tests (like Linux-boot) don't have sources inside gem5 and
are simply downloaded from gem5 servers.
## Updating the test binaries
The test infrastructure should check with the gem5 servers to ensure you have
the latest binaries. However, if you believe your binaries are out of date,
simply delete the `bin` directory and they will be re-downloaded to your local
machine.
## Building (new-style) test binaries
In each `src/` directory under `tests/test-progs`, there is a Makefile.
This Makefile downloads a docker image and builds the test binary for some ISA
(e.g., Makefile.x86 builds the binary for x86). Additionally, if you run `make
upload` it will upload the binaries to the gem5 server, if you have access to
modify the binaries. *If you need to modify the binaries for updating a test or
adding a new test and you don't have access to the gem5 server, contact a
maintainer (see MAINTAINERS).*
## Running Tests in Parallel
### Running Tests in Parallel
Whimsy has support for parallel testing baked in. This system supports
running multiple suites at the same time on the same computer. To run
@@ -260,3 +224,25 @@ suites in parallel, supply the `-t <number-tests>` flag to the run command.
For example, to run up to three test suites at the same time::
./main.py run --skip-build -t 3
### Testing resources
By default binaries and testing resources are obtained via the [gem5 resources infrastructure](https://www.gem5.org/documentation/general_docs/gem5_resources/).
The downloaded resources are cached in "tests/gem5/resources".
The resources are cached to avoid re-downloading when tests are run multiple times, though some of these resources, such as disk images, are large.
It is therefore recommended you remove the "tests/gem5/resources" directory when you are done testing.
## Running Tests within GitHub Actions
These tests outlined here are run as part of [GitHub Actions](https://github.com/features/actions).
These are outlined in [workflow files](https://docs.github.com/en/actions/using-workflows/about-workflows), which can be found in the repo's ".github" directory.
Each workflow is made up of individual jobs where ecch job consists of a series of steps which are executed within a [GitHub Runner](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners).
### Adding Tests to GitHub Actions
To ensure tests added are run in GitHub Actions you may need to modify the worklfow files.
For tests run via `./main.py` we split up the tests via the subdirectories in "tests/gem5".
For example, all tests under "test/gem5/cpu_tests" are run as one job.
Therefore tests added to existing directories are likely to be included, but modifications to the workflow files may be needed if new directories are added.
We strongly recommend that when adding or ammending tests, that contributors check the ".github/workflows" files to ensure the tests they specify will be run as intended.

4
build_tools/enum_cc.py
View File

@@ -97,7 +97,7 @@ const char *${name}Strings[static_cast<int>(${name}::Num_${name})] =
)
else:
code(
"""namespace enums
"""namespace ${wrapper_name}
{"""
)
code.indent(1)
@@ -112,7 +112,7 @@ code("};")
if not enum.wrapper_is_struct and not enum.is_class:
code.dedent(1)
code("} // namespace enums")
code("} // namespace ${wrapper_name}")
code("} // namespace gem5")

6
build_tools/infopy.py
View File

@@ -51,7 +51,11 @@ args = parser.parse_args()
code = code_formatter()
for source in args.files:
src = os.path.basename(source)
# We replace the "."s in the file name with underscores to make
# it a valid python identifier. With the dot, "README.md" would generate
# `README.md = "..."` which is not valid as `md` is not a property of
# `README`.
src = os.path.basename(source).replace(".", "_")
with open(source, "r") as f:
data = "".join(f)
code("${src} = ${{repr(data)}}")

2
configs/example/gem5_library/arm-hello.py
View File

@@ -80,7 +80,7 @@ board = SimpleBoard(
board.set_se_binary_workload(
# The `Resource` class reads the `resources.json` file from the gem5
# resources repository:
# https://gem5.googlesource.com/public/gem5-resource.
# https://github.com/gem5/gem5-resources.
# Any resource specified in this file will be automatically retrieved.
# At the time of writing, this file is a WIP and does not contain all
# resources. Jira ticket: https://gem5.atlassian.net/browse/GEM5-1096

100
configs/example/gem5_library/caches/octopi-cache-example.py Normal file
View File

@@ -0,0 +1,100 @@
# Copyright (c) 2023 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
This script boots Ubuntu 20.04 with 8 timing cores in 1 CCD.
Usage
-----
```
scons build/ARM_MESI_Three_Level/gem5.opt -j `nproc`
./build/ARM_MESI_Three_Level/gem5.opt \
configs/example/gem5_library/caches/octopi-cache-example.py
```
"""
from m5.objects import ArmDefaultRelease, VExpress_GEM5_Foundation
from gem5.utils.requires import requires
from gem5.components.boards.arm_board import ArmBoard
from gem5.components.memory import DualChannelDDR4_2400
from gem5.components.processors.simple_processor import SimpleProcessor
from gem5.components.processors.cpu_types import CPUTypes
from gem5.components.cachehierarchies.ruby.caches.mesi_three_level.octopi import (
OctopiCache,
)
from gem5.isas import ISA
from gem5.coherence_protocol import CoherenceProtocol
from gem5.simulate.simulator import Simulator
from gem5.resources.workload import Workload
num_ccds = 1 # CCDs
num_cores_per_ccd = 8 # 8 cores/CCD
# OctopiCache is built on top of gem5's MESI_Three_Level cache coherence
# protocol
requires(coherence_protocol_required=CoherenceProtocol.MESI_THREE_LEVEL)
cache_hierarchy = OctopiCache(
l1i_size="32KiB",
l1i_assoc=8,
l1d_size="32KiB",
l1d_assoc=8,
l2_size="512KiB",
l2_assoc=8,
l3_size="32MiB",
l3_assoc=16,
num_core_complexes=num_ccds,
is_fullsystem=True,
)
memory = DualChannelDDR4_2400(size="16GB")
# The number of cores must be consistent with
# num_core_complexes and num_cores_per_core_complexes
processor = SimpleProcessor(
cpu_type=CPUTypes.TIMING,
isa=ISA.ARM,
num_cores=num_ccds * num_cores_per_ccd,
)
release = ArmDefaultRelease()
platform = VExpress_GEM5_Foundation()
board = ArmBoard(
clk_freq="4GHz",
processor=processor,
memory=memory,
cache_hierarchy=cache_hierarchy,
release=release,
platform=platform,
)
board.set_workload(Workload("arm64-ubuntu-20.04-boot"))
simulator = Simulator(board=board)
simulator.run()

2
configs/example/gem5_library/checkpoints/riscv-hello-save-checkpoint.py
View File

@@ -97,7 +97,7 @@ board = SimpleBoard(
board.set_se_binary_workload(
# The `Resource` class reads the `resources.json` file from the gem5
# resources repository:
# https://gem5.googlesource.com/public/gem5-resource.
# https://github.com/gem5/gem5-resources.
# Any resource specified in this file will be automatically retrieved.
# At the time of writing, this file is a WIP and does not contain all
# resources. Jira ticket: https://gem5.atlassian.net/browse/GEM5-1096

2
configs/example/gem5_library/dramsys/arm-hello-dramsys.py
View File

@@ -74,7 +74,7 @@ board = SimpleBoard(
board.set_se_binary_workload(
# The `Resource` class reads the `resources.json` file from the gem5
# resources repository:
# https://gem5.googlesource.com/public/gem5-resource.
# https://github.com/gem5/gem5-resources.
# Any resource specified in this file will be automatically retrieved.
# At the time of writing, this file is a WIP and does not contain all
# resources. Jira ticket: https://gem5.atlassian.net/browse/GEM5-1096

2
configs/example/gem5_library/riscv-fs.py
View File

@@ -27,7 +27,7 @@
"""
This example runs a simple linux boot. It uses the 'riscv-disk-img' resource.
It is built with the sources in `src/riscv-fs` in [gem5 resources](
https://gem5.googlesource.com/public/gem5-resources).
https://github.com/gem5/gem5-resources).
Characteristics
---------------

138
configs/example/gem5_library/x86-ubuntu-run-with-kvm-no-perf.py Normal file
View File

@@ -0,0 +1,138 @@
# Copyright (c) 2023 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
This script demonstrates how to use KVM CPU without perf.
This simulation boots Ubuntu 18.04 using 2 KVM CPUs without using perf.
Usage
-----
```
scons build/X86/gem5.opt -j`nproc`
./build/X86/gem5.opt configs/example/gem5_library/x86-ubuntu-run-with-kvm-no-perf.py
```
"""
from gem5.utils.requires import requires
from gem5.components.boards.x86_board import X86Board
from gem5.components.cachehierarchies.ruby.mesi_two_level_cache_hierarchy import (
MESITwoLevelCacheHierarchy,
)
from gem5.components.memory.single_channel import SingleChannelDDR4_2400
from gem5.components.processors.simple_switchable_processor import (
SimpleSwitchableProcessor,
)
from gem5.components.processors.cpu_types import CPUTypes
from gem5.isas import ISA
from gem5.coherence_protocol import CoherenceProtocol
from gem5.simulate.simulator import Simulator
from gem5.simulate.exit_event import ExitEvent
from gem5.resources.workload import Workload
# This simulation requires using KVM with gem5 compiled for X86 simulation
# and with MESI_Two_Level cache coherence protocol.
requires(
isa_required=ISA.X86,
coherence_protocol_required=CoherenceProtocol.MESI_TWO_LEVEL,
kvm_required=True,
)
from gem5.components.cachehierarchies.ruby.mesi_two_level_cache_hierarchy import (
MESITwoLevelCacheHierarchy,
)
cache_hierarchy = MESITwoLevelCacheHierarchy(
l1d_size="32KiB",
l1d_assoc=8,
l1i_size="32KiB",
l1i_assoc=8,
l2_size="512KiB",
l2_assoc=16,
num_l2_banks=1,
)
# Main memory
memory = SingleChannelDDR4_2400(size="3GiB")
# This is a switchable CPU. We first boot Ubuntu using KVM, then the guest
# will exit the simulation by calling "m5 exit" (see the `command` variable
# below, which contains the command to be run in the guest after booting).
# Upon exiting from the simulation, the Exit Event handler will switch the
# CPU type (see the ExitEvent.EXIT line below, which contains a map to
# a function to be called when an exit event happens).
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
isa=ISA.X86,
num_cores=2,
)
# Here we tell the KVM CPU (the starting CPU) not to use perf.
for proc in processor.start:
proc.core.usePerf = False
# Here we setup the board. The X86Board allows for Full-System X86 simulations.
board = X86Board(
clk_freq="3GHz",
processor=processor,
memory=memory,
cache_hierarchy=cache_hierarchy,
)
# Here we set the Full System workload.
# The `set_kernel_disk_workload` function for the X86Board takes a kernel, a
# disk image, and, optionally, a command to run.
# This is the command to run after the system has booted. The first `m5 exit`
# will stop the simulation so we can switch the CPU cores from KVM to timing
# and continue the simulation to run the echo command, sleep for a second,
# then, again, call `m5 exit` to terminate the simulation. After simulation
# has ended you may inspect `m5out/system.pc.com_1.device` to see the echo
# output.
command = (
"m5 exit;"
+ "echo 'This is running on Timing CPU cores.';"
+ "sleep 1;"
+ "m5 exit;"
)
workload = Workload("x86-ubuntu-18.04-boot")
workload.set_parameter("readfile_contents", command)
board.set_workload(workload)
simulator = Simulator(
board=board,
on_exit_event={
# Here we want override the default behavior for the first m5 exit
# exit event. Instead of exiting the simulator, we just want to
# switch the processor. The 2nd m5 exit after will revert to using
# default behavior where the simulator run will exit.
ExitEvent.EXIT: (func() for func in [processor.switch])
},
)
simulator.run()

5
configs/topologies/CustomMesh.py
View File

@@ -171,7 +171,9 @@ class CustomMesh(SimpleTopology):
def _createRNFRouter(self, mesh_router):
# Create a zero-latency router bridging node controllers
# and the mesh router
node_router = self._Router(router_id=len(self._routers), latency=0)
node_router = self._Router(
router_id=len(self._routers), latency=self.node_router_latency
)
self._routers.append(node_router)
# connect node_router <-> mesh router
@@ -270,6 +272,7 @@ class CustomMesh(SimpleTopology):
self._ExtLink = ExtLink
self._Router = Router
self.node_router_latency = 1 if options.network == "garnet" else 0
if hasattr(options, "router_link_latency"):
self._router_link_latency = options.router_link_latency
self._node_link_latency = options.node_link_latency

2
ext/sst/README.md
View File

@@ -78,7 +78,7 @@ the `bbl-busybox-boot-exit` resource, which contains an m5 binary, and
`m5 exit` will be called upon the booting process reaching the early userspace.
More information about building a bootloader containing a Linux Kernel and a
customized workload is available at
[https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/riscv-boot-exit-nodisk/].
[https://github.com/gem5/gem5-resources/tree/stable/src/riscv-boot-exit-nodisk].
## Running an example simulation (Arm)

2
src/Doxyfile
View File

@@ -31,7 +31,7 @@ PROJECT_NAME = gem5
# This could be handy for archiving the generated documentation or
# if some version control system is used.
PROJECT_NUMBER = v23.0.0.1
PROJECT_NUMBER = v23.0.1.0
# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute)
# base path where the generated documentation will be put.

10
src/SConscript
View File

@@ -536,6 +536,14 @@ Export('DebugFormatFlag')
# the corresponding build directory to pick up generated include
# files.
env.Append(CPPPATH=Dir('.'))
parent_dir_set = set()
for extra_dir in extras_dir_list:
parent_dir_set.add(str(Dir(extra_dir).Dir('..').abspath))
if not GetOption('duplicate_sources'):
for parent_dir in parent_dir_set:
env.Append(CPPPATH=Dir(parent_dir))
for extra_dir in extras_dir_list:
env.Append(CPPPATH=Dir(extra_dir))
@@ -615,7 +623,7 @@ PySource('m5', 'python/m5/defines.py')
# Generate a file that wraps the basic top level files
gem5py_env.Command('python/m5/info.py',
[ File('#/COPYING'), File('#/LICENSE'), File('#/README'),
[ File('#/COPYING'), File('#/LICENSE'), File('#/README.md'),
"${GEM5PY}", "${INFOPY_PY}" ],
MakeAction('"${GEM5PY}" "${INFOPY_PY}" "${TARGET}" '
'${SOURCES[:-2]}',

2
src/arch/arm/fastmodel/CortexA76/x1/x1.sgproj
View File

@@ -16,8 +16,6 @@ config "gcc"
SIMGEN_COMMAND_LINE = "--num-comps-file 50";
TARGET_SYSTEMC = "1";
TARGET_SYSTEMC_AUTO = "1";
INCLUDE_DIRS="../../../../../";
}
files
{

2
src/arch/arm/fastmodel/CortexA76/x2/x2.sgproj
View File

@@ -16,8 +16,6 @@ config "gcc"
SIMGEN_COMMAND_LINE = "--num-comps-file 50";
TARGET_SYSTEMC = "1";
TARGET_SYSTEMC_AUTO = "1";
INCLUDE_DIRS="../../../../../";
}
files
{

2
src/arch/arm/fastmodel/CortexA76/x3/x3.sgproj
View File

@@ -16,8 +16,6 @@ config "gcc"
SIMGEN_COMMAND_LINE = "--num-comps-file 50";
TARGET_SYSTEMC = "1";
TARGET_SYSTEMC_AUTO = "1";
INCLUDE_DIRS="../../../../../";
}
files
{

2
src/arch/arm/fastmodel/CortexA76/x4/x4.sgproj
View File

@@ -16,8 +16,6 @@ config "gcc"
SIMGEN_COMMAND_LINE = "--num-comps-file 50";
TARGET_SYSTEMC = "1";
TARGET_SYSTEMC_AUTO = "1";
INCLUDE_DIRS="../../../../../";
}
files
{

2
src/arch/arm/fastmodel/CortexR52/x1/x1.sgproj
View File

@@ -16,8 +16,6 @@ config "gcc"
SIMGEN_COMMAND_LINE = "--num-comps-file 50";
TARGET_SYSTEMC = "1";
TARGET_SYSTEMC_AUTO = "1";
INCLUDE_DIRS="../../../../../";
}
files
{

2
src/arch/arm/fastmodel/CortexR52/x2/x2.sgproj
View File

@@ -16,8 +16,6 @@ config "gcc"
SIMGEN_COMMAND_LINE = "--num-comps-file 50";
TARGET_SYSTEMC = "1";
TARGET_SYSTEMC_AUTO = "1";
INCLUDE_DIRS="../../../../../";
}
files
{

2
src/arch/arm/fastmodel/CortexR52/x3/x3.sgproj
View File

@@ -16,8 +16,6 @@ config "gcc"
SIMGEN_COMMAND_LINE = "--num-comps-file 50";
TARGET_SYSTEMC = "1";
TARGET_SYSTEMC_AUTO = "1";
INCLUDE_DIRS="../../../../../";
}
files
{

2
src/arch/arm/fastmodel/CortexR52/x4/x4.sgproj
View File

@@ -16,8 +16,6 @@ config "gcc"
SIMGEN_COMMAND_LINE = "--num-comps-file 50";
TARGET_SYSTEMC = "1";
TARGET_SYSTEMC_AUTO = "1";
INCLUDE_DIRS="../../../../../";
}
files
{

2
src/arch/arm/fastmodel/GIC/GIC.sgproj
View File

@@ -5,7 +5,7 @@ ACTIVE_CONFIG_LINUX = "gcc";
ACTIVE_CONFIG_WINDOWS = "Win64-Release-VC2015";
config "gcc"
{
ADDITIONAL_COMPILER_SETTINGS = "-O3 -Wall -std=c++14 -Wno-deprecated -Wno-unused-function -I../../../../../";
ADDITIONAL_COMPILER_SETTINGS = "-O3 -Wall -std=c++14 -Wno-deprecated -Wno-unused-function";
ADDITIONAL_LINKER_SETTINGS = "-Wl,--no-undefined";
BUILD_DIR = "./gcc";
COMPILER = "gcc-7.3";

2
src/arch/arm/fastmodel/PL330_DMAC/PL330.sgproj
View File

@@ -5,7 +5,7 @@ ACTIVE_CONFIG_LINUX = "gcc";
ACTIVE_CONFIG_WINDOWS = "Win64-Release-VC2015";
config "gcc"
{
ADDITIONAL_COMPILER_SETTINGS = "-O3 -Wall -std=c++14 -Wno-deprecated -Wno-unused-function -I../../../../../";
ADDITIONAL_COMPILER_SETTINGS = "-O3 -Wall -std=c++14 -Wno-deprecated -Wno-unused-function";
ADDITIONAL_LINKER_SETTINGS = "-Wl,--no-undefined";
BUILD_DIR = "./gcc";
COMPILER = "gcc-7.3";

6
src/arch/arm/fastmodel/SConscript
View File

@@ -218,10 +218,9 @@ class ProjectFileParser(Grammar):
t_ID = r'[A-Za-z_]\w*'
def t_STRLIT(self, t):
r'(?m)"([^"])*"'
r'"([^"])*"'
# strip off quotes
t.value = t.value[1:-1]
t.lexer.lineno += t.value.count('\n')
return t
t_EQUALS = r'='
@@ -377,11 +376,12 @@ class ArmFastModelComponent(object):
self.rpaths = [simgen_dir, project_file_dir]
self.log = gen_dir.File('build_%s.log' % tlc)
self.simgen_cmd = env.subst('${CONF["SIMGEN"]} -p %s '
'--configuration %s -b --verbose off --num-build-cpus %d %s '
'--configuration %s -b --verbose off --num-build-cpus %d -I %s %s '
'--build-dir %s >%s') % \
(shlex.quote(project_file.srcnode().abspath),
shlex.quote(config_name),
GetOption('num_jobs'),
shlex.quote(Dir('#/src').srcnode().abspath),
simgen_command_line,
shlex.quote(simgen_dir.abspath),
shlex.quote(self.log.abspath))

16
src/arch/arm/isa/formats/aarch64.isa
View File

@@ -545,21 +545,37 @@ namespace Aarch64
return new Tlbi64LocalHub(
machInst, miscReg, rt);
case MISCREG_TLBI_ALLE3IS:
case MISCREG_TLBI_ALLE3OS:
case MISCREG_TLBI_ALLE2IS:
case MISCREG_TLBI_ALLE2OS:
case MISCREG_TLBI_ALLE1IS:
case MISCREG_TLBI_ALLE1OS:
case MISCREG_TLBI_VMALLS12E1IS:
case MISCREG_TLBI_VMALLS12E1OS:
case MISCREG_TLBI_VMALLE1IS:
case MISCREG_TLBI_VMALLE1OS:
case MISCREG_TLBI_VAE3IS_Xt:
case MISCREG_TLBI_VAE3OS_Xt:
case MISCREG_TLBI_VALE3IS_Xt:
case MISCREG_TLBI_VALE3OS_Xt:
case MISCREG_TLBI_VAE2IS_Xt:
case MISCREG_TLBI_VAE2OS_Xt:
case MISCREG_TLBI_VALE2IS_Xt:
case MISCREG_TLBI_VALE2OS_Xt:
case MISCREG_TLBI_VAE1IS_Xt:
case MISCREG_TLBI_VAE1OS_Xt:
case MISCREG_TLBI_VALE1IS_Xt:
case MISCREG_TLBI_VALE1OS_Xt:
case MISCREG_TLBI_ASIDE1IS_Xt:
case MISCREG_TLBI_ASIDE1OS_Xt:
case MISCREG_TLBI_VAAE1IS_Xt:
case MISCREG_TLBI_VAAE1OS_Xt:
case MISCREG_TLBI_VAALE1IS_Xt:
case MISCREG_TLBI_VAALE1OS_Xt:
case MISCREG_TLBI_IPAS2E1IS_Xt:
case MISCREG_TLBI_IPAS2E1OS_Xt:
case MISCREG_TLBI_IPAS2LE1IS_Xt:
case MISCREG_TLBI_IPAS2LE1OS_Xt:
return new Tlbi64ShareableHub(
machInst, miscReg, rt, dec.dvmEnabled);
default:

4
src/arch/arm/isa/insts/neon64.isa
View File

@@ -3403,7 +3403,7 @@ let {{
destElem = (srcElem1 >> shiftAmt);
}
destElem += rBit;
} else {
} else if (shiftAmt > 0) {
if (shiftAmt >= sizeof(Element) * 8) {
if (srcElem1 != 0) {
destElem = mask(sizeof(Element) * 8);
@@ -3421,6 +3421,8 @@ let {{
destElem = srcElem1 << shiftAmt;
}
}
} else {
destElem = srcElem1;
}
FpscrQc = fpscr;
'''

11
src/arch/isa_parser/isa_parser.py
View File

@@ -514,6 +514,7 @@ class InstObjParams(object):
class ISAParser(Grammar):
def __init__(self, output_dir):
super().__init__()
self.lex_kwargs["reflags"] = int(re.MULTILINE)
self.output_dir = output_dir
self.filename = None # for output file watermarking/scaremongering
@@ -851,7 +852,7 @@ class ISAParser(Grammar):
# String literal. Note that these use only single quotes, and
# can span multiple lines.
def t_STRLIT(self, t):
r"(?m)'([^'])+'"
r"'([^'])+'"
# strip off quotes
t.value = t.value[1:-1]
t.lexer.lineno += t.value.count("\n")
@@ -860,19 +861,19 @@ class ISAParser(Grammar):
# "Code literal"... like a string literal, but delimiters are
# '{{' and '}}' so they get formatted nicely under emacs c-mode
def t_CODELIT(self, t):
r"(?m)\{\{([^\}]|}(?!\}))+\}\}"
r"\{\{([^\}]|}(?!\}))+\}\}"
# strip off {{ & }}
t.value = t.value[2:-2]
t.lexer.lineno += t.value.count("\n")
return t
def t_CPPDIRECTIVE(self, t):
r"^\#[^\#].*\n"
r"^\#[^\#][^\n]*\n"
t.lexer.lineno += t.value.count("\n")
return t
def t_NEWFILE(self, t):
r'^\#\#newfile\s+"[^"]*"\n'
r'^\#\#newfile\s+"[^"\n]*"\n'
self.fileNameStack.push(t.lexer.lineno)
t.lexer.lineno = LineTracker(t.value[11:-2])
@@ -892,7 +893,7 @@ class ISAParser(Grammar):
# Comments
def t_comment(self, t):
r"//.*"
r"//[^\n]*\n"
# Completely ignored characters
t_ignore = " \t\x0c"

20
src/arch/riscv/isa/decoder.isa
View File

@@ -307,6 +307,11 @@ decode QUADRANT default Unknown::unknown() {
CIMM1 << 5 |
CIMM5<2:0> << 6;
}}, {{
STATUS status = xc->readMiscReg(MISCREG_STATUS);
if (status.fs == FPUStatus::OFF)
return std::make_shared<IllegalInstFault>("FPU is off",
machInst);
Fc1_bits = Mem;
}}, {{
EA = rvZext(sp + offset);
@@ -330,6 +335,11 @@ decode QUADRANT default Unknown::unknown() {
CIMM1 << 5 |
CIMM5<1:0> << 6;
}}, {{
STATUS status = xc->readMiscReg(MISCREG_STATUS);
if (status.fs == FPUStatus::OFF)
return std::make_shared<IllegalInstFault>("FPU is off",
machInst);
freg_t fd;
fd = freg(f32(Mem_uw));
Fd_bits = fd.v;
@@ -387,6 +397,11 @@ decode QUADRANT default Unknown::unknown() {
offset = CIMM6<5:3> << 3 |
CIMM6<2:0> << 6;
}}, {{
STATUS status = xc->readMiscReg(MISCREG_STATUS);
if (status.fs == FPUStatus::OFF)
return std::make_shared<IllegalInstFault>("FPU is off",
machInst);
Mem_ud = Fc2_bits;
}}, {{
EA = rvZext(sp + offset);
@@ -404,6 +419,11 @@ decode QUADRANT default Unknown::unknown() {
offset = CIMM6<5:2> << 2 |
CIMM6<1:0> << 6;
}}, {{
STATUS status = xc->readMiscReg(MISCREG_STATUS);
if (status.fs == FPUStatus::OFF)
return std::make_shared<IllegalInstFault>("FPU is off",
machInst);
Mem_uw = unboxF32(boxF32(Fs2_bits));
}}, {{
EA = (uint32_t)(sp_uw + offset);

2
src/base/version.cc
View File

@@ -32,6 +32,6 @@ namespace gem5
/**
* @ingroup api_base_utils
*/
const char *gem5Version = "23.0.0.1";
const char *gem5Version = "23.0.1.0";
} // namespace gem5

5
src/cpu/kvm/BaseKvmCPU.py
View File

@@ -64,6 +64,11 @@ class BaseKvmCPU(BaseCPU):
def support_take_over(cls):
return True
usePerf = Param.Bool(
True,
"Use perf for gathering statistics from the guest and providing "
"statistic-related functionalities",
)
useCoalescedMMIO = Param.Bool(False, "Use coalesced MMIO (EXPERIMENTAL)")
usePerfOverflow = Param.Bool(
False, "Use perf event overflow counters (EXPERIMENTAL)"

84
src/cpu/kvm/base.cc
View File

@@ -71,12 +71,15 @@ BaseKvmCPU::BaseKvmCPU(const BaseKvmCPUParams &params)
alwaysSyncTC(params.alwaysSyncTC),
threadContextDirty(true),
kvmStateDirty(false),
usePerf(params.usePerf),
vcpuID(-1), vcpuFD(-1), vcpuMMapSize(0),
_kvmRun(NULL), mmioRing(NULL),
pageSize(sysconf(_SC_PAGE_SIZE)),
tickEvent([this]{ tick(); }, "BaseKvmCPU tick",
false, Event::CPU_Tick_Pri),
activeInstPeriod(0),
hwCycles(nullptr),
hwInstructions(nullptr),
perfControlledByTimer(params.usePerfOverflow),
hostFactor(params.hostFactor), stats(this),
ctrInsts(0)
@@ -96,6 +99,22 @@ BaseKvmCPU::BaseKvmCPU(const BaseKvmCPUParams &params)
thread->setStatus(ThreadContext::Halted);
tc = thread->getTC();
threadContexts.push_back(tc);
if ((!usePerf) && perfControlledByTimer) {
panic("KVM: invalid combination of parameters: cannot use "
"perfControlledByTimer without usePerf\n");
}
// If we use perf, we create new PerfKVMCounters
if (usePerf) {
hwCycles = std::unique_ptr<PerfKvmCounter>(new PerfKvmCounter());
hwInstructions = std::unique_ptr<PerfKvmCounter>(new PerfKvmCounter());
} else {
inform("Using KVM CPU without perf. The stats related to the number "
"of cycles and instructions executed by the KVM CPU will not "
"be updated. The stats should not be used for performance "
"evaluation.");
}
}
BaseKvmCPU::~BaseKvmCPU()
@@ -248,7 +267,7 @@ BaseKvmCPU::restartEqThread()
setupCounters();
if (p.usePerfOverflow) {
runTimer.reset(new PerfKvmTimer(hwCycles,
runTimer.reset(new PerfKvmTimer(*hwCycles,
KVM_KICK_SIGNAL,
p.hostFactor,
p.hostFreq));
@@ -424,8 +443,10 @@ BaseKvmCPU::notifyFork()
vcpuFD = -1;
_kvmRun = NULL;
hwInstructions.detach();
hwCycles.detach();
if (usePerf) {
hwInstructions->detach();
hwCycles->detach();
}
}
}
@@ -690,7 +711,9 @@ BaseKvmCPU::kvmRunDrain()
uint64_t
BaseKvmCPU::getHostCycles() const
{
return hwCycles.read();
if (usePerf)
return hwCycles->read();
return 0;
}
Tick
@@ -746,21 +769,26 @@ BaseKvmCPU::kvmRun(Tick ticks)
// Get hardware statistics after synchronizing contexts. The KVM
// state update might affect guest cycle counters.
uint64_t baseCycles(getHostCycles());
uint64_t baseInstrs(hwInstructions.read());
uint64_t baseInstrs = 0;
if (usePerf) {
baseInstrs = hwInstructions->read();
}
// Arm the run timer and start the cycle timer if it isn't
// controlled by the overflow timer. Starting/stopping the cycle
// timer automatically starts the other perf timers as they are in
// the same counter group.
runTimer->arm(ticks);
if (!perfControlledByTimer)
hwCycles.start();
if (usePerf && (!perfControlledByTimer)) {
hwCycles->start();
}
ioctlRun();
runTimer->disarm();
if (!perfControlledByTimer)
hwCycles.stop();
if (usePerf && (!perfControlledByTimer)) {
hwCycles->stop();
}
// The control signal may have been delivered after we exited
// from KVM. It will be pending in that case since it is
@@ -771,7 +799,10 @@ BaseKvmCPU::kvmRun(Tick ticks)
const uint64_t hostCyclesExecuted(getHostCycles() - baseCycles);
const uint64_t simCyclesExecuted(hostCyclesExecuted * hostFactor);
const uint64_t instsExecuted(hwInstructions.read() - baseInstrs);
uint64_t instsExecuted = 0;
if (usePerf) {
instsExecuted = hwInstructions->read() - baseInstrs;
}
ticksExecuted = runTimer->ticksFromHostCycles(hostCyclesExecuted);
/* Update statistics */
@@ -1288,13 +1319,14 @@ BaseKvmCPU::setupCounters()
// We might be re-attaching counters due threads being
// re-initialised after fork.
if (hwCycles.attached())
hwCycles.detach();
if (usePerf) {
if (hwCycles->attached()) {
hwCycles->detach();
}
hwCycles.attach(cfgCycles,
0); // TID (0 => currentThread)
setupInstCounter();
hwCycles->attach(cfgCycles, 0); // TID (0 => currentThread)
setupInstCounter();
}
}
bool
@@ -1344,10 +1376,16 @@ BaseKvmCPU::setupInstStop()
void
BaseKvmCPU::setupInstCounter(uint64_t period)
{
// This function is for setting up instruction counter using perf
if (!usePerf) {
return;
}
// No need to do anything if we aren't attaching for the first
// time or the period isn't changing.
if (period == activeInstPeriod && hwInstructions.attached())
if (period == activeInstPeriod && hwInstructions->attached()) {
return;
}
PerfKvmCounterConfig cfgInstructions(PERF_TYPE_HARDWARE,
PERF_COUNT_HW_INSTRUCTIONS);
@@ -1366,15 +1404,15 @@ BaseKvmCPU::setupInstCounter(uint64_t period)
// We need to detach and re-attach the counter to reliably change
// sampling settings. See PerfKvmCounter::period() for details.
if (hwInstructions.attached())
hwInstructions.detach();
assert(hwCycles.attached());
hwInstructions.attach(cfgInstructions,
if (hwInstructions->attached())
hwInstructions->detach();
assert(hwCycles->attached());
hwInstructions->attach(cfgInstructions,
0, // TID (0 => currentThread)
hwCycles);
*hwCycles);
if (period)
hwInstructions.enableSignals(KVM_KICK_SIGNAL);
hwInstructions->enableSignals(KVM_KICK_SIGNAL);
activeInstPeriod = period;
}

7
src/cpu/kvm/base.hh
View File

@@ -653,6 +653,9 @@ class BaseKvmCPU : public BaseCPU
*/
bool kvmStateDirty;
/** True if using perf; False otherwise*/
bool usePerf;
/** KVM internal ID of the vCPU */
long vcpuID;
@@ -763,7 +766,7 @@ class BaseKvmCPU : public BaseCPU
* PerfKvmTimer (see perfControlledByTimer) to trigger exits from
* KVM.
*/
PerfKvmCounter hwCycles;
std::unique_ptr<PerfKvmCounter> hwCycles;
/**
* Guest instruction counter.
@@ -776,7 +779,7 @@ class BaseKvmCPU : public BaseCPU
* @see setupInstBreak
* @see scheduleInstStop
*/
PerfKvmCounter hwInstructions;
std::unique_ptr<PerfKvmCounter> hwInstructions;
/**
* Does the runTimer control the performance counters?

16
src/cpu/kvm/perfevent.cc
View File

@@ -173,12 +173,20 @@ PerfKvmCounter::attach(PerfKvmCounterConfig &config,
{
if (errno == EACCES)
{
panic("PerfKvmCounter::attach recieved error EACCESS\n"
panic("PerfKvmCounter::attach received error EACCESS.\n"
" This error may be caused by a too restrictive setting\n"
" in the file '/proc/sys/kernel/perf_event_paranoid'\n"
" The default value was changed to 2 in kernel 4.6\n"
" in the file '/proc/sys/kernel/perf_event_paranoid'.\n"
" The default value was changed to 2 in kernel 4.6.\n"
" A value greater than 1 prevents gem5 from making\n"
" the syscall to perf_event_open");
" the syscall to perf_event_open.\n"
" Alternatively, you can set the usePerf flag of the KVM\n"
" CPU to False. Setting this flag to False will limit some\n"
" functionalities of KVM CPU, such as counting the number of\n"
" cycles and the number of instructions, as well as the\n"
" ability of exiting to gem5 after a certain amount of cycles\n"
" or instructions when using KVM CPU. An example can be found\n"
" here, configs/example/gem5_library/"
"x86-ubuntu-run-with-kvm-no-perf.py.");
}
panic("PerfKvmCounter::attach failed (%i)\n", errno);
}

9
src/cpu/minor/dyn_inst.cc
View File

@@ -112,6 +112,11 @@ MinorDynInst::reportData(std::ostream &os) const
std::ostream &
operator <<(std::ostream &os, const MinorDynInst &inst)
{
if (!inst.pc) {
os << inst.id << " pc: 0x???????? (bubble)";
return os;
}
os << inst.id << " pc: 0x"
<< std::hex << inst.pc->instAddr() << std::dec << " (";
@@ -169,7 +174,7 @@ MinorDynInst::minorTraceInst(const Named &named_object) const
{
if (isFault()) {
minorInst(named_object, "id=F;%s addr=0x%x fault=\"%s\"\n",
id, pc->instAddr(), fault->name());
id, pc ? pc->instAddr() : 0, fault->name());
} else {
unsigned int num_src_regs = staticInst->numSrcRegs();
unsigned int num_dest_regs = staticInst->numDestRegs();
@@ -209,7 +214,7 @@ MinorDynInst::minorTraceInst(const Named &named_object) const
minorInst(named_object, "id=%s addr=0x%x inst=\"%s\" class=%s"
" flags=\"%s\"%s%s\n",
id, pc->instAddr(),
id, pc ? pc->instAddr() : 0,
(staticInst->opClass() == No_OpClass ?
"(invalid)" : staticInst->disassemble(0,NULL)),
enums::OpClassStrings[staticInst->opClass()],

1
src/dev/dma_device.cc
View File

@@ -68,6 +68,7 @@ DmaPort::handleRespPacket(PacketPtr pkt, Tick delay)
{
// Should always see a response with a sender state.
assert(pkt->isResponse());
warn_if(pkt->isError(), "Response pkt error.");
// Get the DMA sender state.
auto *state = dynamic_cast<DmaReqState*>(pkt->senderState);

2
src/learning_gem5/part2/hello_object.hh
View File

@@ -69,7 +69,7 @@ class HelloObject : public SimObject
* SimObjects have been constructed. It is called after the user calls
* simulate() for the first time.
*/
void startup();
void startup() override;
};
} // namespace gem5

1
src/mem/SConscript
View File

@@ -151,6 +151,7 @@ DebugFlag('MemCtrl')
DebugFlag('MMU')
DebugFlag('MemoryAccess')
DebugFlag('PacketQueue')
DebugFlag("PortTrace")
DebugFlag('ResponsePort')
DebugFlag('StackDist')
DebugFlag("DRAMSim2")

24
src/mem/port.cc
View File

@@ -45,6 +45,7 @@
#include "mem/port.hh"
#include "base/trace.hh"
#include "debug/PortTrace.hh"
#include "debug/ResponsePort.hh"
#include "sim/sim_object.hh"
@@ -186,6 +187,29 @@ RequestPort::printAddr(Addr a)
sendFunctional(&pkt);
}
void
RequestPort::addTrace(PacketPtr pkt) const
{
if (!gem5::debug::PortTrace || !pkt)
return;
auto ext = pkt->getExtension<TracingExtension>();
if (!ext) {
ext = std::make_shared<TracingExtension>();
pkt->setExtension(ext);
}
ext->add(name(), _responsePort->name());
}
void
RequestPort::removeTrace(PacketPtr pkt) const
{
if (!gem5::debug::PortTrace || !pkt)
return;
auto ext = pkt->getExtension<TracingExtension>();
panic_if(!ext, "There is no TracingExtension in the packet.");
ext->remove();
}
/**
* Response port
*/

73
src/mem/port.hh
View File

@@ -46,6 +46,10 @@
#ifndef __MEM_PORT_HH__
#define __MEM_PORT_HH__
#include <memory>
#include <stack>
#include <string>
#include "base/addr_range.hh"
#include "mem/packet.hh"
#include "mem/protocol/atomic.hh"
@@ -64,6 +68,43 @@ class SlavePort;
class ResponsePort;
/**
* TracingExtension is an Extension of the Packet for recording the trace
* of the Packet. The stack in the TracingExtension holds the name of the
* ports that the Packet has passed through.
*/
class TracingExtension : public gem5::Extension<Packet, TracingExtension>
{
public:
TracingExtension() = default;
TracingExtension(const std::stack<std::string>& q) { trace_ = q; }
std::unique_ptr<ExtensionBase> clone() const override
{
return std::make_unique<TracingExtension>(trace_);
}
void
add(std::string request_port, std::string response_port)
{
trace_.push(request_port);
trace_.push(response_port);
}
void
remove()
{
trace_.pop(); // Remove the response port name.
trace_.pop(); // Remove the request port name.
}
bool empty() { return trace_.empty(); }
std::stack<std::string>& getTrace() { return trace_; }
private:
std::stack<std::string> trace_;
};
/**
* A RequestPort is a specialisation of a Port, which
* implements the default protocol for the three different level of
@@ -266,6 +307,10 @@ class RequestPort: public Port, public AtomicRequestProtocol,
{
panic("%s was not expecting a snoop retry.\n", name());
}
private:
void addTrace(PacketPtr pkt) const;
void removeTrace(PacketPtr pkt) const;
};
class [[deprecated]] MasterPort : public RequestPort
@@ -393,7 +438,11 @@ class ResponsePort : public Port, public AtomicResponseProtocol,
sendTimingResp(PacketPtr pkt)
{
try {
return TimingResponseProtocol::sendResp(_requestPort, pkt);
_requestPort->removeTrace(pkt);
bool succ = TimingResponseProtocol::sendResp(_requestPort, pkt);
if (!succ)
_requestPort->addTrace(pkt);
return succ;
} catch (UnboundPortException) {
reportUnbound();
}
@@ -487,7 +536,10 @@ inline Tick
RequestPort::sendAtomic(PacketPtr pkt)
{
try {
return AtomicRequestProtocol::send(_responsePort, pkt);
addTrace(pkt);
Tick tick = AtomicRequestProtocol::send(_responsePort, pkt);
removeTrace(pkt);
return tick;
} catch (UnboundPortException) {
reportUnbound();
}
@@ -497,8 +549,11 @@ inline Tick
RequestPort::sendAtomicBackdoor(PacketPtr pkt, MemBackdoorPtr &backdoor)
{
try {
return AtomicRequestProtocol::sendBackdoor(_responsePort,
pkt, backdoor);
addTrace(pkt);
Tick tick = AtomicRequestProtocol::sendBackdoor(_responsePort,
pkt, backdoor);
removeTrace(pkt);
return tick;
} catch (UnboundPortException) {
reportUnbound();
}
@@ -508,7 +563,9 @@ inline void
RequestPort::sendFunctional(PacketPtr pkt) const
{
try {
return FunctionalRequestProtocol::send(_responsePort, pkt);
addTrace(pkt);
FunctionalRequestProtocol::send(_responsePort, pkt);
removeTrace(pkt);
} catch (UnboundPortException) {
reportUnbound();
}
@@ -530,7 +587,11 @@ inline bool
RequestPort::sendTimingReq(PacketPtr pkt)
{
try {
return TimingRequestProtocol::sendReq(_responsePort, pkt);
addTrace(pkt);
bool succ = TimingRequestProtocol::sendReq(_responsePort, pkt);
if (!succ)
removeTrace(pkt);
return succ;
} catch (UnboundPortException) {
reportUnbound();
}

2
src/mem/ruby/network/garnet/flit.cc
View File

@@ -51,7 +51,7 @@ flit::flit(int packet_id, int id, int vc, int vnet, RouteInfo route, int size,
m_enqueue_time = curTime;
m_dequeue_time = curTime;
m_time = curTime;
m_packet_id = id;
m_packet_id = packet_id;
m_id = id;
m_vnet = vnet;
m_vc = vc;

12
src/python/SConscript
View File

@@ -154,6 +154,18 @@ PySource('gem5.components.cachehierarchies.ruby.caches.mesi_three_level',
PySource('gem5.components.cachehierarchies.ruby.caches.mesi_three_level',
'gem5/components/cachehierarchies/ruby/caches/mesi_three_level/'
'l3_cache.py')
PySource('gem5.components.cachehierarchies.ruby.caches.mesi_three_level',
'gem5/components/cachehierarchies/ruby/caches/prebuilt/octopi_cache/'
'octopi.py')
PySource('gem5.components.cachehierarchies.ruby.caches.mesi_three_level',
'gem5/components/cachehierarchies/ruby/caches/prebuilt/octopi_cache/'
'core_complex.py')
PySource('gem5.components.cachehierarchies.ruby.caches.mesi_three_level',
'gem5/components/cachehierarchies/ruby/caches/prebuilt/octopi_cache/'
'octopi_network.py')
PySource('gem5.components.cachehierarchies.ruby.caches.mesi_three_level',
'gem5/components/cachehierarchies/ruby/caches/prebuilt/octopi_cache/'
'ruby_network_components.py')
PySource('gem5.components.cachehierarchies.ruby.caches.mi_example',
'gem5/components/cachehierarchies/ruby/caches/mi_example/__init__.py')
PySource('gem5.components.cachehierarchies.ruby.caches.mi_example',

25
src/python/gem5/components/cachehierarchies/ruby/caches/prebuilt/octopi_cache/__init__.py Normal file
View File

@@ -0,0 +1,25 @@
# Copyright (c) 2022-2023 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

245
src/python/gem5/components/cachehierarchies/ruby/caches/prebuilt/octopi_cache/core_complex.py Normal file
View File

@@ -0,0 +1,245 @@
# Copyright (c) 2022-2023 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
from typing import List, Tuple
from gem5.isas import ISA
from gem5.components.boards.abstract_board import AbstractBoard
from gem5.components.processors.abstract_core import AbstractCore
from gem5.components.cachehierarchies.ruby.caches.mesi_three_level.l1_cache import (
L1Cache,
)
from gem5.components.cachehierarchies.ruby.caches.mesi_three_level.l2_cache import (
L2Cache,
)
from gem5.components.cachehierarchies.ruby.caches.mesi_three_level.l3_cache import (
L3Cache,
)
from m5.objects import SubSystem, RubySequencer
from .ruby_network_components import (
RubyRouter,
RubyExtLink,
RubyIntLink,
RubyNetworkComponent,
)
class CoreComplex(SubSystem, RubyNetworkComponent):
_core_id = 0
_core_complex_id = 0
@classmethod
def _get_core_id(cls):
cls._core_id += 1
return cls._core_id - 1
@classmethod
def _get_core_complex_id(cls):
cls._core_complex_id += 1
return cls._core_complex_id - 1
def __init__(
self,
board: AbstractBoard,
cores: List[AbstractCore],
ruby_system,
l1i_size: str,
l1i_assoc: int,
l1d_size: str,
l1d_assoc: int,
l2_size: str,
l2_assoc: int,
l3_size: str,
l3_assoc: int,
):
SubSystem.__init__(self=self)
RubyNetworkComponent.__init__(self=self)
self._l1i_size = l1i_size
self._l1i_assoc = l1i_assoc
self._l1d_size = l1d_size
self._l1d_assoc = l1d_assoc
self._l2_size = l2_size
self._l2_assoc = l2_assoc
self._l3_size = l3_size
self._l3_assoc = l3_assoc
self._board = board
self._cores = cores
self._ruby_system = ruby_system
self._cache_line_size = 64
self._directory_controllers = []
self._core_complex_id = self._get_core_complex_id()
self.main_router = RubyRouter(
self._ruby_system
) # this will be connect to component outside the core complex
self._add_router(self.main_router)
self._create_core_complex()
def get_main_router(self):
return self.main_router
def _create_core_complex(self):
# Create L1 caches, L2 cache, and corresponding controllers per core
self.core_clusters = [
self._create_core_cluster(core) for core in self._cores
]
# Create L3 cache and its corresponding controller
self._create_shared_cache()
# Setting up one router and one external link per controller
self._create_external_links()
# Setting up L1/L2 links, L2/main links, L3/main link
self._create_internal_links()
def _create_core_cluster(self, core: AbstractCore):
cluster = SubSystem()
core_id = self._get_core_id()
cluster.l1_cache = L1Cache(
l1i_size=self._l1i_size,
l1i_assoc=self._l1i_assoc,
l1d_size=self._l1d_size,
l1d_assoc=self._l1d_assoc,
network=self._ruby_system.network,
core=core,
cache_line_size=self._cache_line_size,
target_isa=self._board.processor.get_isa(),
clk_domain=self._board.get_clock_domain(),
)
cluster.l1_cache.sequencer = RubySequencer(
version=core_id,
dcache=cluster.l1_cache.Dcache,
clk_domain=cluster.l1_cache.clk_domain,
)
if self._board.has_io_bus():
cluster.l1_cache.sequencer.connectIOPorts(self._board.get_io_bus())
cluster.l1_cache.ruby_system = self._ruby_system
core.connect_icache(cluster.l1_cache.sequencer.in_ports)
core.connect_dcache(cluster.l1_cache.sequencer.in_ports)
core.connect_walker_ports(
cluster.l1_cache.sequencer.in_ports,
cluster.l1_cache.sequencer.in_ports,
)
if self._board.get_processor().get_isa() == ISA.X86:
core.connect_interrupt(
cluster.l1_cache.sequencer.interrupt_out_port,
cluster.l1_cache.sequencer.in_ports,
)
else:
core.connect_interrupt()
cluster.l2_cache = L2Cache(
l2_size=self._l2_size,
l2_assoc=self._l2_assoc,
network=self._ruby_system.network,
core=core,
num_l3Caches=1, # each core complex has 1 slice of L3 Cache
cache_line_size=self._cache_line_size,
cluster_id=self._core_complex_id,
target_isa=self._board.processor.get_isa(),
clk_domain=self._board.get_clock_domain(),
)
cluster.l2_cache.ruby_system = self._ruby_system
# L0Cache in the ruby backend is l1 cache in stdlib
# L1Cache in the ruby backend is l2 cache in stdlib
cluster.l2_cache.bufferFromL0 = cluster.l1_cache.bufferToL1
cluster.l2_cache.bufferToL0 = cluster.l1_cache.bufferFromL1
return cluster
def _create_shared_cache(self):
self.l3_cache = L3Cache(
l3_size=self._l3_size,
l3_assoc=self._l3_assoc,
network=self._ruby_system.network,
num_l3Caches=1,
cache_line_size=self._cache_line_size,
cluster_id=self._core_complex_id,
)
self.l3_cache.ruby_system = self._ruby_system
# This is where all routers and links are created
def _create_external_links(self):
# create a router per cache controller
# - there is one L3 per ccd
self.l3_router = RubyRouter(self._ruby_system)
self._add_router(self.l3_router)
# - there is one L1 and one L2 per cluster
for cluster in self.core_clusters:
cluster.l1_router = RubyRouter(self._ruby_system)
self._add_router(cluster.l1_router)
cluster.l2_router = RubyRouter(self._ruby_system)
self._add_router(cluster.l2_router)
# create an ext link from a controller to a router
self.l3_router_link = RubyExtLink(
ext_node=self.l3_cache,
int_node=self.l3_router,
bandwidth_factor=64,
)
self._add_ext_link(self.l3_router_link)
for cluster in self.core_clusters:
cluster.l1_router_link = RubyExtLink(
ext_node=cluster.l1_cache, int_node=cluster.l1_router
)
self._add_ext_link(cluster.l1_router_link)
cluster.l2_router_link = RubyExtLink(
ext_node=cluster.l2_cache, int_node=cluster.l2_router
)
self._add_ext_link(cluster.l2_router_link)
def _create_internal_links(self):
# create L1/L2 links
for cluster in self.core_clusters:
l1_to_l2, l2_to_l1 = RubyIntLink.create_bidirectional_links(
cluster.l1_router, cluster.l2_router
)
cluster.l1_to_l2_link = l1_to_l2
cluster.l2_to_l1_link = l2_to_l1
self._add_int_link(l1_to_l2)
self._add_int_link(l2_to_l1)
# create L2/main_router links
for cluster in self.core_clusters:
l2_to_main, main_to_l2 = RubyIntLink.create_bidirectional_links(
cluster.l2_router, self.main_router
)
cluster.l2_to_main_link = l2_to_main
cluster.main_to_l2_link = main_to_l2
self._add_int_link(l2_to_main)
self._add_int_link(main_to_l2)
# create L3/main_router link
l3_to_main, main_to_l3 = RubyIntLink.create_bidirectional_links(
self.l3_router, self.main_router, bandwidth_factor=64
)
self.l3_to_main_link = l3_to_main
self.main_to_l3_link = main_to_l3
self._add_int_link(l3_to_main)
self._add_int_link(main_to_l3)

257
src/python/gem5/components/cachehierarchies/ruby/caches/prebuilt/octopi_cache/octopi.py Normal file
View File

@@ -0,0 +1,257 @@
# Copyright (c) 2022-2023 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
from ...abstract_ruby_cache_hierarchy import AbstractRubyCacheHierarchy
from ....abstract_three_level_cache_hierarchy import (
AbstractThreeLevelCacheHierarchy,
)
from ......coherence_protocol import CoherenceProtocol
from ......components.boards.abstract_board import AbstractBoard
from ......utils.requires import requires
from ......components.cachehierarchies.ruby.caches.mesi_three_level.directory import (
Directory,
)
from ......components.cachehierarchies.ruby.caches.mesi_three_level.dma_controller import (
DMAController,
)
from m5.objects import RubySystem, DMASequencer, RubyPortProxy
from .core_complex import CoreComplex
from .octopi_network import OctopiNetwork
from .ruby_network_components import RubyRouter, RubyExtLink, RubyIntLink
# CoreComplex sub-systems own the L1, L2, L3 controllers
# OctopiCache owns the directory controllers
# RubySystem owns the DMA Controllers
class OctopiCache(
AbstractRubyCacheHierarchy, AbstractThreeLevelCacheHierarchy
):
def __init__(
self,
l1i_size: str,
l1i_assoc: int,
l1d_size: str,
l1d_assoc: int,
l2_size: str,
l2_assoc: int,
l3_size: str,
l3_assoc: int,
num_core_complexes: int,
is_fullsystem: bool,
):
AbstractRubyCacheHierarchy.__init__(self=self)
AbstractThreeLevelCacheHierarchy.__init__(
self=self,
l1i_size=l1i_size,
l1i_assoc=l1i_assoc,
l1d_size=l1d_size,
l1d_assoc=l1d_assoc,
l2_size=l2_size,
l2_assoc=l2_assoc,
l3_size=l3_size,
l3_assoc=l3_assoc,
)
self._directory_controllers = []
self._dma_controllers = []
self._io_controllers = []
self._core_complexes = []
self._num_core_complexes = num_core_complexes
self._is_fullsystem = is_fullsystem
def incorporate_cache(self, board: AbstractBoard) -> None:
requires(
coherence_protocol_required=CoherenceProtocol.MESI_THREE_LEVEL
)
cache_line_size = board.get_cache_line_size()
self.ruby_system = RubySystem()
# MESI_Three_Level needs 3 virtual networks
self.ruby_system.number_of_virtual_networks = 3
self.ruby_system.network = OctopiNetwork(self.ruby_system)
# Setting up the core complex
all_cores = board.get_processor().get_cores()
num_cores_per_core_complex = len(all_cores) // self._num_core_complexes
self.core_complexes = [
CoreComplex(
board=board,
cores=all_cores[
core_complex_idx
* num_cores_per_core_complex : (core_complex_idx + 1)
* num_cores_per_core_complex
],
ruby_system=self.ruby_system,
l1i_size=self._l1i_size,
l1i_assoc=self._l1i_assoc,
l1d_size=self._l1d_size,
l1d_assoc=self._l1d_assoc,
l2_size=self._l2_size,
l2_assoc=self._l2_assoc,
l3_size=self._l3_size,
l3_assoc=self._l3_assoc,
)
for core_complex_idx in range(self._num_core_complexes)
]
self.ruby_system.network.incorporate_ccds(self.core_complexes)
self._create_directory_controllers(board)
self._create_dma_controllers(board, self.ruby_system)
self.ruby_system.num_of_sequencers = (
len(all_cores)
+ len(self._dma_controllers)
+ len(self._io_controllers)
)
# SimpleNetwork requires .int_links and .routers to exist
# if we want to call SimpleNetwork.setup_buffers()
self.ruby_system.network.int_links = (
self.ruby_system.network._int_links
)
self.ruby_system.network.ext_links = (
self.ruby_system.network._ext_links
)
self.ruby_system.network.routers = self.ruby_system.network._routers
self.ruby_system.network.setup_buffers()
# Set up a proxy port for the system_port. Used for load binaries and
# other functional-only things.
self.ruby_system.sys_port_proxy = RubyPortProxy()
board.connect_system_port(self.ruby_system.sys_port_proxy.in_ports)
def _create_directory_controllers(self, board):
# Adding controllers
self.directory_controllers = [
Directory(
self.ruby_system.network,
board.get_cache_line_size(),
addr_range,
mem_port,
)
for addr_range, mem_port in board.get_mem_ports()
]
for ctrl in self.directory_controllers:
ctrl.ruby_system = self.ruby_system
# Adding controller routers
self.directory_controller_routers = [
RubyRouter(self.ruby_system.network)
for _ in range(len(self.directory_controllers))
]
for router in self.directory_controller_routers:
self.ruby_system.network._add_router(router)
# Adding an external link for each controller and its router
self.directory_controller_ext_links = [
RubyExtLink(ext_node=dir_ctrl, int_node=dir_router)
for dir_ctrl, dir_router in zip(
self.directory_controllers, self.directory_controller_routers
)
]
for ext_link in self.directory_controller_ext_links:
self.ruby_system.network._add_ext_link(ext_link)
_directory_controller_int_links = []
for router in self.directory_controller_routers:
int_link_1, int_link_2 = RubyIntLink.create_bidirectional_links(
router, self.ruby_system.network.cross_ccd_router
)
_directory_controller_int_links.extend([int_link_1, int_link_2])
self.ruby_system.network._add_int_link(int_link_1)
self.ruby_system.network._add_int_link(int_link_2)
self.directory_controller_int_links = _directory_controller_int_links
def _create_dma_controllers(self, board, ruby_system):
# IOController for full system simulation
if self._is_fullsystem:
self.io_sequencer = DMASequencer(
version=0, ruby_system=self.ruby_system
)
self.io_sequencer.in_ports = board.get_mem_side_coherent_io_port()
self.ruby_system.io_controller = DMAController(
dma_sequencer=self.io_sequencer, ruby_system=self.ruby_system
)
self._io_controllers.append(self.ruby_system.io_controller)
self.io_controller_router = RubyRouter(self.ruby_system.network)
self.ruby_system.network._add_router(self.io_controller_router)
self.io_controller_ext_link = RubyExtLink(
ext_node=self._io_controllers[0],
int_node=self.io_controller_router,
)
self.ruby_system.network._add_ext_link(self.io_controller_ext_link)
self.io_controller_int_links = (
RubyIntLink.create_bidirectional_links(
self.io_controller_router,
self.ruby_system.network.cross_ccd_router,
)
)
self.ruby_system.network._add_int_link(
self.io_controller_int_links[0]
)
self.ruby_system.network._add_int_link(
self.io_controller_int_links[1]
)
self._dma_controllers = []
if board.has_dma_ports():
self.ruby_system.dma_controllers = [
DMAController(
dma_sequencer=DMASequencer(version=i + 1, in_ports=port),
ruby_system=self.ruby_system,
)
for i, port in enumerate(board.get_dma_ports())
]
self._dma_controllers = self.ruby_system.dma_controllers
self.dma_routers = [
RubyRouter(self.ruby_system.network)
for dma_controller in self._dma_controllers
]
for dma_router in self.dma_routers:
self.ruby_system.network._add_router(dma_router)
self.dma_ext_links = [
RubyExtLink(ext_node=dma_controller, int_node=dma_router)
for dma_controller, dma_router in zip(
self._dma_controllers, self.dma_routers
)
]
for link in self.dma_ext_links:
self.ruby_system.network._add_ext_link(link)
self.dma_int_links = [
RubyIntLink(
dma_router, self.ruby_system.network.cross_ccd_router
)
for dma_router in self.dma_routers
] + [
RubyIntLink(
self.ruby_system.network.cross_ccd_router, dma_router
)
for dma_router in self.dma_routers
]
for link in self.dma_int_links:
self.ruby_system.network._add_int_link(link)

67
src/python/gem5/components/cachehierarchies/ruby/caches/prebuilt/octopi_cache/octopi_network.py Normal file
View File

@@ -0,0 +1,67 @@
# Copyright (c) 2022-2023 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
from m5.objects import SimpleNetwork
from .ruby_network_components import (
RubyNetworkComponent,
RubyRouter,
RubyIntLink,
)
# . The Network owns all routers, all int links and all ext links that are not in CCD's.
# . The CCD subsystems are not of type RubyNetwork, so we need to copy the references of
# routers and links to OctopiNetwork._routers, ._int_links, and ._ext_links; which will
# be, in turns, copied to RubyNetwork.routers, .int_links, and .ext_links respectively.
class OctopiNetwork(SimpleNetwork, RubyNetworkComponent):
def __init__(self, ruby_system):
SimpleNetwork.__init__(self=self)
RubyNetworkComponent.__init__(self=self)
self.netifs = []
self.ruby_system = ruby_system
self.number_of_virtual_networks = (
ruby_system.number_of_virtual_networks
)
self.cross_ccd_router = RubyRouter(self)
self._add_router(self.cross_ccd_router)
def connect_ccd_routers_to_cross_ccd_router(self, ccds):
for ccd in ccds:
int_link_1, int_link_2 = RubyIntLink.create_bidirectional_links(
self.cross_ccd_router,
ccd.get_main_router(),
bandwidth_factor=64,
)
ccd.to_cross_ccd_router_link = int_link_1
ccd.from_cross_ccd_router_link = int_link_2
self._add_int_link(int_link_1)
self._add_int_link(int_link_2)
def incorporate_ccds(self, ccds):
for ccd in ccds:
self.incorporate_ruby_subsystem(ccd)
self.connect_ccd_routers_to_cross_ccd_router(ccds)

111
src/python/gem5/components/cachehierarchies/ruby/caches/prebuilt/octopi_cache/ruby_network_components.py Normal file
View File

@@ -0,0 +1,111 @@
# Copyright (c) 2022-2023 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
from m5.objects import Switch, SimpleIntLink, SimpleExtLink
class RubyNetworkComponent:
def __init__(self):
super().__init__()
self._routers = []
self._ext_links = []
self._int_links = []
def _add_router(self, router):
self._routers.append(router)
def _add_ext_link(self, link):
self._ext_links.append(link)
def _add_int_link(self, link):
self._int_links.append(link)
def get_routers(self):
return self._routers
def get_ext_links(self):
return self._ext_links
def get_int_links(self):
return self._int_links
def incorporate_ruby_subsystem(self, other_ruby_subsystem):
self._routers.extend(other_ruby_subsystem.get_routers())
self._ext_links.extend(other_ruby_subsystem.get_ext_links())
self._int_links.extend(other_ruby_subsystem.get_int_links())
class RubyRouter(Switch):
_router_id = 0
@classmethod
def _get_router_id(cls):
cls._router_id += 1
return cls._router_id - 1
def __init__(self, network):
super().__init__()
self.router_id = self._get_router_id()
self.virt_nets = network.number_of_virtual_networks
class RubyExtLink(SimpleExtLink):
_link_id = 0
@classmethod
def _get_link_id(cls):
cls._link_id += 1
return cls._link_id - 1
def __init__(self, ext_node, int_node, bandwidth_factor=16):
super().__init__()
self.link_id = self._get_link_id()
self.ext_node = ext_node
self.int_node = int_node
self.bandwidth_factor = bandwidth_factor
class RubyIntLink(SimpleIntLink):
_link_id = 0
@classmethod
def _get_link_id(cls):
cls._link_id += 1
return cls._link_id - 1
@classmethod
def create_bidirectional_links(cls, node_1, node_2, bandwidth_factor=16):
return [
RubyIntLink(node_1, node_2, bandwidth_factor),
RubyIntLink(node_2, node_1, bandwidth_factor),
]
def __init__(self, src_node, dst_node, bandwidth_factor=16):
super().__init__()
self.link_id = self._get_link_id()
self.src_node = src_node
self.dst_node = dst_node
self.bandwidth_factor = bandwidth_factor

18
src/python/gem5/resources/downloader.py
View File

@@ -139,14 +139,32 @@ def _download(url: str, download_to: str, max_attempts: int = 6) -> None:
time.sleep((2**attempt) + random.uniform(0, 1))
else:
raise e
except ConnectionResetError as e:
# This catches the ConnectionResetError we see occassionally see
# when accessing resources on GitHub Actions. It retries using a
# Truncated Exponential backoff algorithm, truncating after
# "max_attempts". If any other is retrieved we raise the error.
if e.errno == 104:
attempt += 1
if attempt >= max_attempts:
raise Exception(
f"After {attempt} attempts, the resource json could "
"not be retrieved. OS Error Code retrieved: "
f"{e.errno}"
)
time.sleep((2**attempt) + random.uniform(0, 1))
else:
raise e
except ValueError as e:
raise Exception(
f"ValueError: {e}\n"
"Environment variable GEM5_USE_PROXY is set to "
f"'{use_proxy}'. The expected form is "
"<host>:<port>'."
)
except ImportError as e:
raise Exception(
f"ImportError: {e}\n"
"An import error has occurred. This is likely due "
"the Python SOCKS client module not being "
"installed. It can be installed with "

7
src/python/m5/main.py
View File

@@ -348,13 +348,14 @@ def interact(scope):
def _check_tracing():
import m5
import _m5.core
from .util import fatal
if _m5.core.TRACING_ON:
return
m5.fatal("Tracing is not enabled. Compile with TRACING_ON")
fatal("Tracing is not enabled. Compile with TRACING_ON")
def main():
@@ -369,7 +370,7 @@ def main():
from . import stats
from . import trace
from .util import inform, fatal, panic, isInteractive
from .util import inform, panic, isInteractive
from m5.util.terminal_formatter import TerminalFormatter
options, arguments = parse_options()

4
src/python/m5/params.py
View File

@@ -1560,8 +1560,8 @@ class MetaEnum(MetaParamValue):
if cls.is_class:
cls.cxx_type = f"{name}"
else:
cls.cxx_type = f"enums::{name}"
scope = init_dict.get("wrapper_name", "enums")
cls.cxx_type = f"{scope}::{name}"
super().__init__(name, bases, init_dict)

2
tests/pyunit/stdlib/resources/pyunit_resource_specialization.py
View File

@@ -278,7 +278,7 @@ class ResourceSpecializationSuite(unittest.TestCase):
Looppoint JSON file."""
resource = obtain_resource(
resource_id="looppoint-json-restore-resource-region-1",
resource_id="looppoint-json-restore-resource-region-1-example",
resource_directory=self.get_resource_dir(),
resource_version="1.0.0",
gem5_version="develop",

2
tests/pyunit/stdlib/resources/pyunit_workload_checks.py
View File

@@ -65,7 +65,7 @@ class CustomWorkloadTestSuite(unittest.TestCase):
function="set_se_binary_workload",
parameters={
"binary": obtain_resource(
"x86-hello64-static", gem5_version="develop"
"x86-hello64-static-example", gem5_version="develop"
),
"arguments": ["hello", 6],
},

18
tests/pyunit/stdlib/resources/refs/resource-specialization.json
View File

@@ -19,9 +19,9 @@
"id": "disk-image-example",
"description": "disk-image documentation.",
"architecture": "X86",
"is_zipped": true,
"md5sum": "90e363abf0ddf22eefa2c7c5c9391c49",
"url": "http://dist.gem5.org/dist/develop/images/x86/ubuntu-18-04/x86-ubuntu.img.gz",
"is_zipped": false,
"md5sum": "71b2cb004fe2cda4556f0b1a38638af6",
"url": "http://dist.gem5.org/dist/develop/test-progs/hello/bin/arm/linux/hello64-static",
"source": "src/x86-ubuntu",
"root_partition": "1",
"resource_version": "1.0.0",
@@ -64,9 +64,9 @@
"description": "checkpoint-example documentation.",
"architecture": "RISCV",
"is_zipped": false,
"md5sum": "3a57c1bb1077176c4587b8a3bf4f8ace",
"source": null,
"is_tar_archive": true,
"md5sum": "3a57c1bb1077176c4587b8a3bf4f8ace",
"url": "http://dist.gem5.org/dist/develop/checkpoints/riscv-hello-example-checkpoint.tar",
"resource_version": "1.0.0",
"gem5_versions": [
@@ -93,8 +93,8 @@
"id": "file-example",
"description": null,
"is_zipped": false,
"md5sum": "2efd144c11829ab18d54eae6371e120a",
"url": "http://dist.gem5.org/dist/develop/checkpoints/riscv-hello-example-checkpoint.tar",
"md5sum": "71b2cb004fe2cda4556f0b1a38638af6",
"url": "http://dist.gem5.org/dist/develop/test-progs/hello/bin/arm/linux/hello64-static",
"source": null,
"resource_version": "1.0.0",
"gem5_versions": [
@@ -106,10 +106,10 @@
"category": "directory",
"id": "directory-example",
"description": "directory-example documentation.",
"is_zipped": false,
"md5sum": "3a57c1bb1077176c4587b8a3bf4f8ace",
"source": null,
"is_zipped": false,
"is_tar_archive": true,
"md5sum": "3a57c1bb1077176c4587b8a3bf4f8ace",
"url": "http://dist.gem5.org/dist/develop/checkpoints/riscv-hello-example-checkpoint.tar",
"resource_version": "1.0.0",
"gem5_versions": [
@@ -177,7 +177,7 @@
},
{
"category": "looppoint-json",
"id": "looppoint-json-restore-resource-region-1",
"id": "looppoint-json-restore-resource-region-1-example",
"description": "A looppoint json file resource.",
"is_zipped": false,
"region_id": "1",

16
tests/pyunit/stdlib/resources/refs/workload-checks.json
View File

@@ -1,7 +1,7 @@
[
{
"category": "kernel",
"id": "x86-linux-kernel-5.2.3",
"id": "x86-linux-kernel-5.2.3-example",
"description": "The linux kernel (v5.2.3), compiled to X86.",
"architecture": "X86",
"is_zipped": false,
@@ -15,12 +15,12 @@
},
{
"category": "disk-image",
"id": "x86-ubuntu-18.04-img",
"id": "x86-ubuntu-18.04-img-example",
"description": "A disk image containing Ubuntu 18.04 for x86..",
"architecture": "X86",
"is_zipped": true,
"md5sum": "90e363abf0ddf22eefa2c7c5c9391c49",
"url": "http://dist.gem5.org/dist/develop/images/x86/ubuntu-18-04/x86-ubuntu.img.gz",
"is_zipped": false,
"md5sum": "dbf120338b37153e3334603970cebd8c",
"url": "http://dist.gem5.org/dist/develop/test-progs/hello/bin/x86/linux/hello64-static",
"source": "src/x86-ubuntu",
"root_partition": "1",
"resource_version": "1.0.0",
@@ -34,8 +34,8 @@
"description": "Description of workload here",
"function": "set_kernel_disk_workload",
"resources": {
"kernel": "x86-linux-kernel-5.2.3",
"disk-image": "x86-ubuntu-18.04-img"
"kernel": "x86-linux-kernel-5.2.3-example",
"disk-image": "x86-ubuntu-18.04-img-example"
},
"additional_params": {
"readfile_contents": "echo 'Boot successful'; m5 exit"
@@ -47,7 +47,7 @@
},
{
"category": "binary",
"id": "x86-hello64-static",
"id": "x86-hello64-static-example",
"description": "A 'Hello World!' binary.",
"architecture": "X86",
"is_zipped": false,

2
tests/weekly.sh
View File

@@ -94,7 +94,7 @@ rm -rf ${gem5_root}/m5out coAuthorsDBLP.graph 1k_128k.gr result.out
# Moreover, DNNMark builds a library and thus doesn't have a binary, so we
# need to build it before we run it.
# Need to pull this first because HACC's docker requires this path to exist
git clone https://gem5.googlesource.com/public/gem5-resources \
git clone https://github.com/gem5/gem5-resources \
"${gem5_root}/gem5-resources"

2
util/dockerfiles/gem5-all-min-dependencies/Dockerfile
View File

@@ -26,7 +26,7 @@
FROM gcr.io/gem5-test/ubuntu-22.04_min-dependencies:latest as source
RUN apt -y update && apt -y install git
RUN git clone -b develop https://gem5.googlesource.com/public/gem5 /gem5
RUN git clone -b develop https://github.com/gem5/gem5/ /gem5
WORKDIR /gem5
RUN scons -j`nproc` build/ALL/gem5.fast

2
util/gem5art/artifact/README.md
View File

@@ -51,7 +51,7 @@ gem5_binary = Artifact.registerArtifact(
inputs = [gem5_repo,],
documentation = '''
Default gem5 binary compiled for the X86 ISA.
This was built from the main gem5 repo (gem5.googlesource.com) without
This was built from the main gem5 repo (https://github.com/gem5/gem5/) without
any modifications. We recently updated to the current gem5 master
which has a fix for memory channel address striping.
'''

4
util/gem5art/artifact/setup.py
View File

@@ -56,8 +56,8 @@ setup(
install_requires=["pymongo"],
python_requires=">=3.6",
project_urls={
"Bug Reports": "https://gem5.atlassian.net/",
"Source": "https://gem5.googlesource.com/",
"Bug Reports": "https://github.com/gem5/issues/",
"Source": "https://github.com/gem5/gem5/",
"Documentation": "https://www.gem5.org/documentation/gem5art",
},
)

2
util/gem5art/run/setup.py
View File

@@ -57,7 +57,7 @@ setup(
python_requires=">=3.6",
project_urls={
"Bug Reports": "https://gem5.atlassian.net/",
"Source": "https://gem5.googlesource.com/",
"Source": "https://github.com/gem5/gem5/",
"Documentation": "https://www.gem5.org/documentation/gem5art",
},
scripts=["bin/gem5art-getruns"],

2
util/gem5art/tasks/setup.py
View File

@@ -58,7 +58,7 @@ setup(
python_requires=">=3.6",
project_urls={
"Bug Reports": "https://gem5.atlassian.net/",
"Source": "https://gem5.googlesource.com/",
"Source": "https://github.com/gem5/gem5/",
"Documentation": "https://www.gem5.org/documentation/gem5art",
},
)

132
util/github-runners-vagrant/README.md Normal file
View File

@@ -0,0 +1,132 @@
# Setting up a Github Actions Runner with Vagrant
This directory provides a way to setup a Github Actions runner using Vagrant to host the runner in a Virtual machine.
This tutorial has been written with the assumption of running on a machine with Ubuntu 22.04.
Setting up a runner on a different OS may require some changes.
Before anything else, copy this directory, "util/github-runners-vagrant", to the root of the location on your host system you wish to setup the VMs from.
The CWD is assumed to be this directory.
## Install Dependencies
```sh
sudo apt install vagrant
sudo apt-get build-dep vagrant ruby-libvirt
sudo apt-get install qemu libvirt-daemon-system libvirt-clients ebtables dnsmasq-base libxslt-dev libxml2-dev libvirt-dev zlib1g-dev ruby-dev
# Note: The vagrant-libvirt APT package does not work as intended. We must
# remove it from the system otherwise errors will occur (we will install it
# later using the vagrant plugin command).
sudo apt purge vagrant-libvirt
```
## Set up the Vagrantfiles for the GitHub repository
First, generate a Personal Access Token, which you can create [here](https://github.com/settings/tokens)
Make sure to set admin permissions on this token, then replace instances of `<PERSONAL ACCESS TOKEN>` in the Vagrantfiles ("Vagrantfile-builder" and "Vagrant-runner") with your token.
Next, replace instances of `<GITHUB REPO>` with your GitHub account name and the repository name, separated by a forward slash.
For example, if your GitHub account name is `example` and your repository name is `example-repo`, you would replace `<GITHUB REPO>` with `example/example-repo`.
## Install Vagrant Plugins
Once everything is set properly, set the `VAGRANT_HOME` environment variable to the directory in which the Vagrant files and other scripts are stored (i.e., the CWD).
For example:
```sh
export VAGRANT_HOME=`pwd`
```
After this, install the relevant vagrant plugins:
``` sh
vagrant plugin install dotenv
vagrant plugin install vagrant-libvirt
vagrant plugin install vagrant-reload
```
## The "builder" and "runner" VMs
The number of CPUs and the memory size differs between the "Vagrantfile-builder" and "Vagrantfile-runner".
In our work we have two types of machines "runners" and "builders".
Runners are single core machines with 8GB of memory, and builders are 4 core machines with 16GB of memory.
The latter is used for building gem5 binaries while the former is used for running instances of gem5.
You can expect each machine to take up approximately 60GB of disk space though VMs will consume the disk space they require.
The "Vagrantfile-builder" file is set to create a runner machine and the "Vagrantfile-builder" file is set to create a builder machine.
Specifying which Vagrantfile to use is done by setting the `VAGRANT_VAGRANTFILE` environment variable.
## Creating the virtual machine
Each VM on your host system must have a unique name.
Give the VM to be created a unique name by setting the `<VM NAME>` variables in the Vagrantfile you wish to utilize.
Then run:
```sh
VAGRANT_VAGRANTFILE=<VAGRANTFILE> vagrant up --provider=libvirt
```
This should automatically create your machine, as well as configure and start up a Github Actions runner.
You can check the status of the runner here: https://github.com/<account>/<repo>/settings/actions/runners
If the runner ever shows as offline, you can rerun the `vagrant up --provider=libvirt` command to make sure everything is working properly.
If you wish to create more than one runner you must edit the `<VM NAME>` in the Vagrant file.
## Helper scripts
The "vm_manager" script can be used to set up multiple builder and runner VMs.
To use this script simply modify the `NUM_RUNNERS`, `NUM_BUILDERS`, `RUNNER_PREFIX`, and `BUILDER_PREFIX` variables to the desired values.
Then run the script with:
```sh
./vm_manager.sh
```
This script will create any VMs that don't already exist and ensure those that do exists are running.
If you wish to destroy all the VMs you can run:
```sh
./vm_manager.sh destroy
```
**Note:** This script assumes "VAGRANT_HOME" is set to the CWD.
## Improving stability
Occasionally GitHub runner services, or VMs, go down. This is often silent and
usually only noticable from going to the GitHub repo page "settings" -> "actions" -> "runners" and observing the status.
When the VMs or the service stop working they need restarted.
To do so you can sun `./vm_manager.sh`. This will cycle through the VMs and execute a `vagrant up` command.
This does one of three things depending on the state of the VM:
1. If the VM is down this will bring the VM back online and start the GitHub runner service.
2. If the VM is up but the GitHub runner service is down, this will start the GitHub runner service.
3. If the VM is up and the GitHub runner service is running (i.e., everything is fine) then this does nothing.
Given there is no harm in running this command frequently, we recommend setting up a cron job to automatically execute `./vm_manager.sh` every few hours.
## Troubleshooting
### The default libvirt disk image storage pool is on the wrong drive
By default libvirt will store disk images in "/var/lib/libvirt/images".
This is not ideal as it is on a small root partition.
A solution to this is to change the default storage location.
To do so, do the following:
```sh
virsh pool-list --all # Confirm here a "default" pool exist. We'll modify this.
virsh pool-dumpxml default >default-pool.xml # We take a dump of the default then removed it.
virsh pool-destroy default
virsh pool-undefine default
vim default-pool.xml # Change the image path to the desired path
virsh pool-define default-pool.xml # From here we re-add the default.
virsh pool-start default
virsh pool-autostart default
```

57
util/github-runners-vagrant/Vagrantfile-builder Normal file
View File

@@ -0,0 +1,57 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.box = "generic/ubuntu2204"
config.vm.box_check_update = true
config.vm.define "<VM NAME>"
config.vm.hostname = "<VM NAME>"
# allows us to ssh into the machine, addressing the problem below
# https://www.reddit.com/r/vagrant/comments/sb7hfl/new_to_vagrant_getting_efault_warning/
config.ssh.username = "vagrant"
config.ssh.password = "vagrant"
config.vm.provider "libvirt" do |vb|
# Customize the amount of cpus and memory on the VM:
vb.cpus = "4".to_i
vb.memory = "16384".to_i
end
# sets up vm
config.vm.provision :shell, path: "provision_root.sh"
config.vm.provision :shell, privileged: false, path: "provision_nonroot.sh"
# To ensure we don't run out of memory, we enable dynamic Swap Space. This is
# done via the "swapspace" daemon: https://pqxx.org/development/swapspace/
config.vm.provision :shell, inline: "sudo apt install swapspace -y"
# The provision_root.sh adds the vagrant user to the docker group, so we need to reload the VM.
config.vm.provision :reload
config.vm.provision :shell, run: 'always', inline: <<-SHELL
# When running gem5 in SE mode we must overcommit memory.
# This is run on every startup of the VM.
/sbin/sysctl vm.overcommit_memory=1
SHELL
config.vm.provision :shell, privileged: false, run: 'always', inline: <<-SHELL
if [ -d ~/actions-runner ]; then
# This will be run everytime the VM is run (once created).
cd actions-runner
nohup ./run.sh &
else
# This will be run the first time the VM is created.
mkdir ~/actions-runner && cd ~/actions-runner
curl -so actions-runner-linux-x64-2.304.0.tar.gz -L \
https://github.com/actions/runner/releases/download/v2.304.0/actions-runner-linux-x64-2.304.0.tar.gz
tar xzf ./actions-runner-linux-x64-2.304.0.tar.gz
# configure the runner
# echo automatically sets the name of the runner, and the tags
# create a personal access token with admin permission and copy it into the curl command
echo -ne '\n\nbuild\n\n' | ./config.sh --url https://github.com/<GITHUB REPO> --token $(curl -L \
-X POST -H "Accept: application/vnd.github+json" -H "Authorization: Bearer <PERSONAL ACCESS TOKEN>" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/<GITHUB REPO>/actions/runners/registration-token | jq -r '.token')
# start the runner
nohup ./run.sh &
fi
SHELL
end

56
util/github-runners-vagrant/Vagrantfile-runner Normal file
View File

@@ -0,0 +1,56 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.box = "generic/ubuntu2204"
config.vm.box_check_update = true
config.vm.define "<VM NAME>"
config.vm.hostname = "<VM NAME>"
# allows us to ssh into the machine, addressing the problem below
# https://www.reddit.com/r/vagrant/comments/sb7hfl/new_to_vagrant_getting_efault_warning/
config.ssh.username = "vagrant"
config.ssh.password = "vagrant"
config.vm.provider "libvirt" do |vb|
# Customize the amount of cpus and memory on the VM:
vb.cpus = "1".to_i
vb.memory = "8192".to_i
end
# sets up vm
config.vm.provision :shell, path: "provision_root.sh"
config.vm.provision :shell, privileged: false, path: "provision_nonroot.sh"
# To ensure we don't run out of memory, we enable dynamic Swap Space. This is
# done via the "swapspace" daemon: https://pqxx.org/development/swapspace/
config.vm.provision :shell, inline: "sudo apt install swapspace -y"
# The provision_root.sh adds the vagrant user to the docker group, so we need to reload the VM.
config.vm.provision :reload
config.vm.provision :shell, run: 'always', inline: <<-SHELL
# When running gem5 in SE mode we must overcommit memory.
# This is run on every startup of the VM.
/sbin/sysctl vm.overcommit_memory=1
SHELL
config.vm.provision :shell, privileged: false, run: 'always', inline: <<-SHELL
if [ -d ~/actions-runner ]; then
# This will be run everytime the VM is run (once created).
cd actions-runner
nohup ./run.sh &
else
# This will be run the first time the VM is created.
mkdir ~/actions-runner && cd ~/actions-runner
curl -so actions-runner-linux-x64-2.304.0.tar.gz -L \
https://github.com/actions/runner/releases/download/v2.304.0/actions-runner-linux-x64-2.304.0.tar.gz
tar xzf ./actions-runner-linux-x64-2.304.0.tar.gz
# configure the runner
# echo automatically sets the name of the runner, and the tags
# create a personal access token with admin permission and copy it into the curl command
echo -ne '\n\nrun\n\n' | ./config.sh --url https://github.com/<GITHUB REPO> --token $(curl -L \
-X POST -H "Accept: application/vnd.github+json" -H "Authorization: Bearer <PERSONAL ACCESS TOKEN>" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/<GITHUB REPO>/actions/runners/registration-token | jq -r '.token')
# start the runner
nohup ./run.sh &
fi
SHELL
end

15
util/github-runners-vagrant/provision_nonroot.sh Normal file
View File

@@ -0,0 +1,15 @@
#!/usr/bin/env bash
# fail on unset variables and command errors
set -eu -o pipefail # -x: is for debugging
# Install deno
curl -fsSL https://deno.land/x/install/install.sh | sh
echo "export PATH=\"\${HOME}/.deno/bin:\${PATH}\"" >> ~/.profile
echo "export PATH=\"\${HOME}/.deno/bin:\${PATH}\"" >> ~/.bash_profile
# Install docker compose
DOCKER_COMPOSE_VERSION=$(curl -s https://api.github.com/repos/docker/compose/releases/latest | jq -r '.tag_name')
mkdir -p "${HOME}/.docker/cli-plugins"
curl -sL "https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-$(uname -s)-$(uname -m)" -o "${HOME}/.docker/cli-plugins/docker-compose"
chmod +x "${HOME}/.docker/cli-plugins/docker-compose"

43
util/github-runners-vagrant/provision_root.sh Normal file
View File

@@ -0,0 +1,43 @@
#!/usr/bin/env bash
# fail on unset variables and command errors
set -eu -o pipefail # -x: is for debugging
apt-get update
apt-get upgrade -y
apt-get install -y software-properties-common
add-apt-repository --yes --update ppa:git-core/ppa
apt-get install -y \
bash \
build-essential \
clang-format \
git \
git-lfs \
jq \
libffi-dev \
libssl-dev \
nkf \
python3 \
python3-dev \
python3-pip \
python3-venv \
shellcheck \
tree \
wget \
yamllint \
zstd
snap install jq
# Install docker
apt-get install -y apt-transport-https ca-certificates curl gnupg lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update -y
apt-get install -y docker-ce docker-ce-cli containerd.io
groupadd docker || true
gpasswd -a vagrant docker
newgrp docker
systemctl restart docker
# Cleanup
apt-get autoremove -y

43
util/github-runners-vagrant/vm_manager.sh Executable file
View File

@@ -0,0 +1,43 @@
#!/bin/bash
NUM_RUNNERS=20
NUM_BUILDERS=3
RUNNER_PREFIX="$(hostname)-runner-"
BUILDER_PREFIX="$(hostname)-builder-"
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
export VAGRANT_HOME=${SCRIPT_DIR}
param="up"
if [[ $# -ge 1 ]]; then
param=$1
if [[ "${param}" != "destroy" ]] && [[ "${param}" != "shutdown" ]]; then
echo "Only valid parameters are 'destroy' and 'shutdown' to destroy all VMs or shutdown all VMs"
exit 1
fi
fi
for (( i=1; i<=NUM_RUNNERS; i++ )); do
sed -i "s/ config.vm.define.*/ config.vm.define \"${RUNNER_PREFIX}${i}\"/g" Vagrantfile-runner
sed -i "s/ config.vm.hostname.*/ config.vm.hostname = \"${RUNNER_PREFIX}${i}\"/g" Vagrantfile-runner
if [[ "${param}" == "destroy" ]]; then
VAGRANT_VAGRANTFILE=Vagrantfile-runner vagrant destroy -f
elif [[ "${param}" == "shutdown" ]]; then
VAGRANT_VAGRANTFILE=Vagrantfile-runner vagrant halt -f
else
VAGRANT_VAGRANTFILE=Vagrantfile-runner vagrant up --provider=libvirt
fi
done
for (( i=1; i<=NUM_BUILDERS; i++ )); do
sed -i "s/ config.vm.define.*/ config.vm.define \"${BUILDER_PREFIX}${i}\"/g" Vagrantfile-builder
sed -i "s/ config.vm.hostname.*/ config.vm.hostname = \"${BUILDER_PREFIX}${i}\"/g" Vagrantfile-builder
if [[ "${param}" == "destroy" ]]; then
VAGRANT_VAGRANTFILE=Vagrantfile-builder vagrant destroy -f
elif [[ "${param}" == "shutdown" ]]; then
VAGRANT_VAGRANTFILE=Vagrantfile-builder vagrant halt -f
else
VAGRANT_VAGRANTFILE=Vagrantfile-builder vagrant up --provider=libvirt
fi
done