Commit Graph

978 Commits

Author SHA1 Message Date
Bobby R. Bruce
4ad1150372 stdlib: Add the LooppointCsvResource resource
This resource wraps the LooppointCsvLoader class so it may be obtained
as a specialized resource via gem5 resources.

Relevant tests and config scripts have been updated.

Change-Id: Ib8e5ff5500fb1560951c9c0110e3c3aec8ca3c42
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67857
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2023-02-23 12:07:30 +00:00
Bobby R. Bruce
aae3430281 stdlib: Refactor Looppoint
This change refactors the Looppoint files. While functionally
equivalent, this classes have been moved and altered to be easier to
handle going forward. The following changes have been made:

- New classes have been added to represent the data structure of the
  Looppoint JSON. This simplifies the parsing of JSON files and makes it
  handle Looppoint data structures. Ultimately this is hidden from the
  user via the new 'gem5.resources.Looppoint' class which will be the
  front-facing class for Looppoint interactions.
- The `LooppointCheckpoint` class has been replaced with
  `LooppointCsvLoader`. This new class takes in a CSV pintpoints file
  to load necessary looppoint data.
- The `LoopPointRestore` class has been replaced by
  `LooppointJsonLoader`.
- All Looppoint classes have been moved to `gem5.resources`. This will
  make it easier when we add Looppoints as specific gem5 resources.

Change-Id: I11dd1fe8f76658db220320584270d57cb37a3c62
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67611
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2023-02-23 12:07:30 +00:00
Bobby R. Bruce
82aec1c19b stdlib: Update LoopPointRestore to take singular region_id
This change has been introduced as we only restore to one checkpoint at
a time. The change updates LoopPointRestore to take a sigular region_id
and use it to generate the LoopPpoint from the LoopPoint JSON file.

Change-Id: I0e88d5ba03b164bdd5da098397f44e16af591134
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67492
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2023-02-23 12:07:30 +00:00
Bobby R. Bruce
de5044cfe0 stdlib: Change the default Looppoint JSON output to m5out
Change-Id: I6ac9aa55dfd4a0250c487ae743a1f4ea0b1bc154
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67491
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2023-02-23 12:07:30 +00:00
Zhantong Qiu
afbca3b6e7 stdlib: Added LoopPoint checkpoint specific generator
Added looppoint_save_checkpoint_generator to take checkpoints for
LoopPoint methodology.
Users can decide to update the relative counts storing in the LoopPoint
module and exit when all the target PC-count pairs are encountered or
not.

Change-Id: Id1cf1516f4fa838e20a67530e94b361e42ca09f3
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67197
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2023-02-22 19:30:09 +00:00
Zhantong Qiu
82c587bd88 stdlib: Allow se_binary_workload to setup LoopPoints
Added a set_se_looppoint_workload function to take in information for
workload and a stdlib LoopPoint object that stores all the information
the workload needed to run the LoopPoint sampling method.
Added a get_looppoint function to return the stdlib LoopPoint object.

Change-Id: I7afc5c4c743256f7df97345f331b6f72b7a5fd07
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67196
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2023-02-22 19:30:09 +00:00
Zhantong Qiu
41816bf030 stdlib: Added stdlib LoopPoint classes
LoopPoint is a multithreaded workload sampling method that targets
PCs and PC execution counts.
The main idea for LoopPoint is to base the beginning and end of the
simjulation sample on the number of times a particular loop (PC) has
been executed globally across all threads in a region that partitioned
with a set length of instruction counts. This in some senses
generalizes SimPoint which use the instruction count of a single
thread.
The link to the paper: https://ieeexplore.ieee.org/document/9773236

The LoopPointCheckpoint is designed to take in LoopPoint data file
and generate the information needed to take checkpoints for LoopPoint
regions(warmup regions+simulation region)
The LoopPointRestore is designed to take in the LoopPOint data file
and generate information needed to to restore a checkpoint taken by
the LoopPOintCheckpoint.
The LoopPoint is the parent class for LoopPointCheckpoint and
LoopPointRestore.

Change-Id: I595b0ff9d350c7c496639748a9c63ecc61fbaec9
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67195
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2023-02-22 19:30:09 +00:00
Zhantong Qiu
717d3b239c base,python: Added PcCountPair type and parameter
This commit introduces a PcCountPair type that stores a Program Counter
address and an integer of counts for the Program Counter address.
The PcCountPair can be used in the same way and hashable in both C++
and Python.

Change-Id: I66d93e2c6a1d286cb9dd795ba97f8d887f67d503
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67193
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2023-02-22 19:30:09 +00:00
Bobby R. Bruce
e1601954f0 stdlib: Implement Simpoint Resources
This patches does the following:
- Adds 'SimpointResource' which encapsulates Simpoint data and
  functionality. It replaces the old 'gem5.util.simpoint.SimPoint'
  class. Simpoints can be loaded from gem5-resources using the
  `obtain_resource` function.
- Adds 'SimpointDirectoryResource'. This inherits form
  'SimpointResource'. While 'SimpointResource' takes raw Simpoint data
  via parameters, 'SimpointDirectoryResource' assumes the data exists
  in files, in a directory.
- Updates the
  "configs/example/gem5_library/checkpoints/simpoints-se-checkpoint.py"
  and
  "configs/example/gem5_library/checkpoints/simpoints-se-restory.py"
  example files to utilize this new Simpoint resource classes.

**Note**: While the old "SimPoint" class
("src/python/gem5/util/simpoint.py") is marked as deprecated, it may be
difficult to utilize given updates to the APIs in the gem5 stdlib Cores
and Simulator modules.

Change-Id: I9bed5c643ffc735838c9f22a58c53547941010e7
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67339
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2023-02-22 19:30:09 +00:00
Bobby R. Bruce
cc838d72a6 stdlib: Update resources to have downloads optional
With this patch, when a resource entry does not specify a "url" field,
there is no file downloaded. This is necessary infrastructure for
gem5-resources which do not have specific files/directories to be
downloaded but exist solely in the resources.json file.

Change-Id: I0d92e830bfcef750119078b8c226b0659ba7f6cb
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67338
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2023-02-22 19:30:09 +00:00
Bobby R. Bruce
a9b69ee055 stdlib: Add null/None versioning in resources.json
This patch allows for the "version" field in the resources.json file to
be `null` (translated to `None` in the Python JSON package) or not
declared. In this case the resources.json file will be used regardless
as to what version the gem5 binary is set. This is useful for testing
purposes.

Tests have been updated to utilize this where possible.

Change-Id: I9d8ae18cb3e61d58bc822bad30853fa3442cb33f
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67337
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
2023-02-22 19:30:09 +00:00
Bobby R. Bruce
4ee724e054 stdlib: Specialize the gem5-resources
This commit specializes the Resource class into specific sub-types.

The `Resource`, `CustomResource` and `CustomDiskImageResource` classes
have been deprecated in favor of the `AbstractResource` subclasses.
Custom Resources can be created via the resource specialization
constructor. Resources can be obtained via the gem5-resource
infrastructure with the `obtain_resource` function.

Fully implemented:

- DiskImageResource
- BinaryResource
- KernelResource
- BootloaderResource
- FileResource
- DirectoryResource

Partially implemented:

- SimpointResource
- CheckpointResource

While the schema of the resource.json file has changed, efforts have
been made to ensure backwards compatibility is maintained during this
transition.

Tests are included in this commit to verify this feature works as
expected.

**Note:** The Simpoint tests are disabled in this commit, to be
reenabled when Simpoint resource specialization is fully incorporated
here:
https://gem5-review.googlesource.com/c/public/gem5/+/67339

Change-Id: I77277ecaffc7abc86db08526aacc0b606ef04fe8
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67175
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2023-02-22 19:30:09 +00:00
Nikos Nikoleris
df0bed6858 python: Ensure that m5.internal.params is available
Add an import to m5.internal.params which became necessary after:

95f9017c2e configs,python: Clean some cruft out of m5.objects.

This import is necessary but also causes problems when scons calls
build_tools/sim_object_param_struct_hh.py to generate
params/SimObject.hh. m5.internal.params itself imports _m5 and _m5 is
unavalailable resulting in an ImportError. This is bening and we can
safely ignore it.

Change-Id: I3809e81284e730fb9c9e0e7e91bd61b801d73f90
Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67797
Maintainer: Giacomo Travaglini <giacomo.travaglini@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
2023-02-20 13:40:40 +00:00
Gabriel Busnot
8d0fde1961 python: Fix deprecated decorator
The deprecation message was firing during the decoration process instead of firing upon first call to deprecated function. The message now fires only if the deprected function is called.

Change-Id: I2d510eb24884fdba0123e71e8472db68ae9d2ce4
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67334
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Richard Cooper <richard.cooper@arm.com>
2023-01-19 23:03:05 +00:00
Daniel R. Carvalho
4f480fc6fc base: Remove the Stats namespace
This namespace has gone through the deprecation period
and can now be removed.

Change-Id: I80f25af68e03fff3df8316cb4d1d2669687d0fe4
Signed-off-by: Daniel R. Carvalho <odanrc@yahoo.com.br>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67359
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2023-01-17 09:16:20 +00:00
Hoa Nguyen
49ac00d060 stdlib: Fix errors in MESI_Three_Level_Cache_Hierarchy
Change-Id: I60ae47f4336cb1b54bcca3fce3bdd13858daa92a
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/66771
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2023-01-04 02:36:33 +00:00
Bobby R. Bruce
bbeec2d758 misc: Update version info for develop branch
Change-Id: Icd409acda0e88852938b2af9f170e2a410e91f8c
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67053
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2023-01-03 22:08:08 +00:00
Bobby R. Bruce
218b3925be misc: Merge branch stable into develop branch
This ensures both branches are in-sync and have not diverged.

Change-Id: Ib487d8596037017b9ec03d7e8a76229373c153db
2022-12-30 20:43:00 +00:00
Bobby R. Bruce
1e73beb620 python: Remove 'scheduleTickExit' in favor of 'exitSimLoop'
The commit https://gem5-review.googlesource.com/c/public/gem5/+/66231
added an API to m5 for scheduling to-tick exit events. This added the
function `schedule_tick_exit`. It was later pointed out that this
`schedule_tick_exit` event is redundant given the existance of
`exitSimLoop`. This patch therefore removes `schedule_tick_exit` in
favor of `exitSimLoop`.

Change-Id: Ibecf00b98256a5da2868427d766bdc93f03c3f97
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/66511
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2022-12-06 22:41:58 +00:00
Bobby R. Bruce
ae20719576 python: Remove 'scheduleTickExit' in favor of 'exitSimLoop'
The commit https://gem5-review.googlesource.com/c/public/gem5/+/66231
added an API to m5 for scheduling to-tick exit events. This added the
function `schedule_tick_exit`. It was later pointed out that this
`schedule_tick_exit` event is redundant given the existance of
`exitSimLoop`. This patch therefore removes `schedule_tick_exit` in
favor of `exitSimLoop`.

Change-Id: Ibecf00b98256a5da2868427d766bdc93f03c3f97
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/66451
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
2022-12-06 18:04:46 +00:00
Bobby R. Bruce
7185c9ea19 stdlib: Update the gem5 resources' version to "v22.1"
This is required to ensure the correct version of the resources are
obtained for this version of gem5.

Change-Id: If316081c776d510ec9a2f7ff9e077a1f837d2d2e
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65915
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2022-12-05 20:25:53 +00:00
Bobby R. Bruce
005049f548 stdlib,python: Allow setting of to tick exits via m5
This commit adds the following functions to the `m5` python module:

- setMaxTick(tick) -> None
- getMaxTick() -> int
- getTicksUntilMax() -> int
- scheduleTickExitFromCurrent(tick, exit_string) -> None
- scheduleTickExitAbsolute(tick, exit_string) -> None

Until this patch the only way to set an exit at a particular tick was
via `simulate.run` which would reschedule the maximum tick. This
functionality has been explicity exposed via the new `setMaxTick`
function. However, as this is only rescheduling the maximum tick, it
stops scheduling exits at multiple different ticks.

To get around this problem the `scheduleTickExit` functions have been
added. These allow a user to schedule multiple exit events. The
functions contain a `exit_string` parameter that provides the string
the simulator is to return when the specified tick is met. By default
this string is "Tick exit reached" which is used by the stdlib
Simulator module to declare a new `SCHEDULED_TICK` exit event (Note:
this has been deliberatly kept seperate from the `MAX_TICK` exit event.
This commit serves as an attempt to decouple these are two concepts).

Tests are provided in this patch to ensure these new functions work as
intended.

Additional notes:
- The `simulate` function has been fixed to match the documentation. If
  the `num_cycles` is -1 then the maximum ticks is set to MaxTicks.
  Otherwise the max ticks is set to `curTicks() + num_cycles`. The
  functionality of this function will remain unchanged to the end-user.
- Full integration into the Simulator module is not complete as of this
  patch. Users must us the m5 python module to set these exit events.

Change-Id: I6c92b31dd409dc866152224600ea8166cfcba38b
Issue-on: https://gem5.atlassian.net/browse/GEM5-1131
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/66231
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/66331
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2022-12-02 17:24:04 +00:00
Bobby R. Bruce
8479a691aa stdlib,python: Allow setting of to tick exits via m5
This commit adds the following functions to the `m5` python module:

- setMaxTick(tick) -> None
- getMaxTick() -> int
- getTicksUntilMax() -> int
- scheduleTickExitFromCurrent(tick, exit_string) -> None
- scheduleTickExitAbsolute(tick, exit_string) -> None

Until this patch the only way to set an exit at a particular tick was
via `simulate.run` which would reschedule the maximum tick. This
functionality has been explicity exposed via the new `setMaxTick`
function. However, as this is only rescheduling the maximum tick, it
stops scheduling exits at multiple different ticks.

To get around this problem the `scheduleTickExit` functions have been
added. These allow a user to schedule multiple exit events. The
functions contain a `exit_string` parameter that provides the string
the simulator is to return when the specified tick is met. By default
this string is "Tick exit reached" which is used by the stdlib
Simulator module to declare a new `SCHEDULED_TICK` exit event (Note:
this has been deliberatly kept seperate from the `MAX_TICK` exit event.
This commit serves as an attempt to decouple these are two concepts).

Tests are provided in this patch to ensure these new functions work as
intended.

Additional notes:
- The `simulate` function has been fixed to match the documentation. If
  the `num_cycles` is -1 then the maximum ticks is set to MaxTicks.
  Otherwise the max ticks is set to `curTicks() + num_cycles`. The
  functionality of this function will remain unchanged to the end-user.
- Full integration into the Simulator module is not complete as of this
  patch. Users must us the m5 python module to set these exit events.

Change-Id: I6c92b31dd409dc866152224600ea8166cfcba38b
Issue-on: https://gem5.atlassian.net/browse/GEM5-1131
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/66231
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
2022-12-02 06:04:53 +00:00
Hoa Nguyen
aeb617868f stdlib: Add MESI Three Level cache hierarchy
Change-Id: Ibea6b71d62b71f7817f6860bbceed9e1915bb002
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65591
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
2022-11-30 01:19:37 +00:00
Hoa Nguyen
eac06ad681 python: Fix multiline quotes in a single line
An example case,
```python
mem_side_port = RequestPort(
    "This port sends requests and " "receives responses"
)
```

This is the residue of running the python formatter.
This is done by finding all tokens matching the regex `"\s"(?![.;"])`
and manually replacing them by empty strings.

Change-Id: Icf223bbe889e5fa5749a81ef77aa6e721f38b549
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/66111
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-29 23:44:38 +00:00
Hoa Nguyen
f99947059d stdlib: Clean up Ruby cache directory
- Fix typos.
- Fix type inconsistencies.

Change-Id: I98d82ec7e62130abb09295c5ec6cde86b1f7fa27
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65571
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-29 23:36:26 +00:00
Hoa Nguyen
c8949f085f stdlib: Change #virtual_networks of mesi_two_level to 3
A grep of "virtual_network" in src/mem/ruby/protocol shows that
files in MESI_Two_Level.slicc utilizes at 3 virtual networks.

Change-Id: I3e8dd09dd82b9c802fdf91145c6d998bc6db541b
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65592
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
2022-11-29 19:51:16 +00:00
Hoa Nguyen
8391f47bc9 stdlib: More helpful message for the filelock error
Change-Id: Ib8e3bc9fc145a9604670e8288209ac62bfbd7932
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/66091
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-29 19:09:37 +00:00
Jasjeet Rangi
f172c41c68 stdlib: Fix get_isa_from_str() exception behavior in isas.py
When given an input string that does not match any valid ISA, the
get_isa_from_str() function should call get_isas_str_set() to to print
the valid ISA strings in the exception. The current behavior is to
recursively call get_isa_from_str() with no input, which prevents
the correct exception from being raised. This change causes the
correct exception to be raised for invalid inputs.

Change-Id: I92bfe862bbd99ce0b63bfc124e539fab3b175e0c
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65411
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-09 18:37:14 +00:00
Jasjeet Rangi
78b978686c stdlib: Fix get_isa_from_str() exception behavior in isas.py
When given an input string that does not match any valid ISA, the
get_isa_from_str() function should call get_isas_str_set() to to print
the valid ISA strings in the exception. The current behavior is to
recursively call get_isa_from_str() with no input, which prevents
the correct exception from being raised. This change causes the
correct exception to be raised for invalid inputs.

Change-Id: I92bfe862bbd99ce0b63bfc124e539fab3b175e0c
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65311
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-09 17:16:34 +00:00
Hoa Nguyen
553096ee53 stdlib: Make the Matched board a package
So that the board and its components can be reused.

Change-Id: Idae1a4493fbb4d826ac8da76532692a985f8025f
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65331
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
2022-11-08 05:32:32 +00:00
Hoa Nguyen
26c27b1bf8 stdlib: Change the default kernel boot param from "ro" to "rw"
Running workloads likely causes some content to be written to
the disk image, e.g., `m5 readfile`. However, on riscv boards,
the default kernel param specifies the disk image to be read-only.

This change changes this param so that the disk image is
read-write by default.

Change-Id: I414e483ad11d747f34433560e32a8f91a425ce7e
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65194
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-04 19:29:38 +00:00
Bobby R. Bruce
c871d25ede stdlib: Fixing erroneous typing in Simulator __init__
The key in the `on_exit_event` dictionary can only be an ExitEvent. The
typing information here suggested a string could be passed. This is not
the case. This patch fixes the issue.

Change-Id: Iaa4b66417c15e31f6aafe9e550a7552461a15be1
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64773
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
2022-11-04 18:04:54 +00:00
Bobby R. Bruce
353c20e8bf stdlib: Fix typos and remove unneeded import in Simulator
Change-Id: I6faa63959d09444b8e931e14846fb1af98866ade
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/62931
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
2022-11-04 18:04:54 +00:00
Bobby R. Bruce
c88b528738 stdlib: Update AbstractCore's 'set_inst_stop_any_thread'
This patch:
- Makes this function private.
- Updates the function's documentation.
- Changes the 'init' parameter to 'board_initialized'.

It doesn't make much sense for this function to be exposed directly to
the user as it requires knowing whether the board is initialized or not.

In addition to this I believe it makes more sense for the 'init' logic
to be flipped and renamed "board_initialized' so that this value is True
if the board has been initialized.

The documentation for this function has been updated.

Change-Id: I016c65bde88357111d3e648d7aa99aeb6e31f410
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64833
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-04 18:04:54 +00:00
Bobby R. Bruce
12d8d5ca26 stdlib: Update AbstractCore set_simpoint func
This change:
- Makes this function private.
- Adds better documentation describing the usage.
- Changes the 'init' param to 'board_initialized'

This function really doesn't make much sense to set directly by an
stdlib user. It requires knowing whether or not the the board has been
initialized which is an annoying detail and will cause error if set
incorrectly.

The logic of the `init` parameter has been flipped to be
`board_initialized`. This makes it clearer what the parameter is
doing and what it's for.

The documentation for this function has been updated to make it clearer
on how the `board_initialized` parameter should be used correctly.

Change-Id: I567a48df06e6327b38673a2c510065d4334657e2
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64832
Reviewed-by: Melissa Jost <mkjost@ucdavis.edu>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-04 18:04:54 +00:00
Bobby R. Bruce
905b71c375 python: Move find from group to AbstractStat
This expands the 'find' feature to be recursive and find all the
stats/groups of stats of that regex all the way down the SimStats tree.

Change-Id: Id888911a6189e0440d2537f9720aa594353e00c7
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63692
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-03 22:32:36 +00:00
Bobby R. Bruce
b65fa9e0d8 python: Add AbstractStat for PyStats
Previously all PyStats inheritted from JsonSerializable. The
AbstractStat class has been added to give a cleaner, clearer Base class
for PyStats.

Change-Id: I7e1808c4b4dcd6110fd524ad3553a9dc19f72e24
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63691
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-03 22:32:36 +00:00
Bobby R. Bruce
68f8c2946d stdlib: Add 'get_simstats' function to simulator
Change-Id: Iedf937a66f33c5a5feada4ffbf550540f65680d1
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63272
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
2022-11-03 22:32:36 +00:00
Bobby R. Bruce
4a06375212 stdlib: Add __repr__ to pystats
For Statistics the value is returned. E.g.:

```
print(simstats.board.core.some_integer)
> 5
```

For Groups the names of the stats in that group are listed.
E.g.:

```
print(stats.board.core)
> [Group: [some_integer, another_stat, another_group]]
```

Change-Id: I94cea907608fba622f4fc141d5b22ac95d8cde40
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63271
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
2022-11-03 22:32:36 +00:00
Bobby R. Bruce
be33be87da misc: Update black to process src/python/m5/ext/pystats
The exclusion in .pre-commit-config.yaml covered all files in
src/python/m5/ext. This excludes src/python/m5/exit/pystats, which we
want covered by black. This commit updates .pre-commit-config.yaml to
only exclude src/python/m5/ext/pyfdt.

This change also runs black on these files.

Change-Id: Iecff45ea2a27a37fc0d00b867d41300aad911c7a
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63711
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2022-11-03 22:32:36 +00:00
Bobby R. Bruce
5204b58e19 stdlib: Rename JsonSerializable to SerializableStat
As this abstract class now allows the output of text stats, it's more
appropriate to rename it. It no longer handles processing just for JSON
output

Change-Id: Ia9a1e3ef4029de45a11ac261fb14c9bdfa412cdd
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/59273
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2022-11-03 22:32:36 +00:00
Melissa Jost
23d405ea55 tests, resources: CVE-2007-4559 Patch
Hi, we are security researchers from the Advanced Research Center at Trellix.
We have began a campaign to patch a widespread bug named CVE-2007-4559.
CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using
extract() or extractall() on a tarfile object without sanitizing input,
a maliciously crafted .tar file could perform a directory path traversal
attack. We found at least one unsantized extractall() in your codebase
and are providing a patch for you via pull request. The patch essentially
checks to see if all tarfile members will be extracted safely and throws
an exception otherwise. We encourage you to use this patch or your own
solution to secure against CVE-2007-4559.

If you have further questions you may contact us through this
projects lead researcher Kasimir Schulz.

Change-Id: I891ac6652cfbd479aed51d64ef6d4e0fe740e06d
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65271
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
2022-11-03 22:17:50 +00:00
Earl Ou
5bf88bf7a1 sim: allow specifying remote gdb port for each workload
In a platform with multiple systems, we may want to specify the
remote gdb port for each system. This change makes it
possible to specify the port number at each Workload instance.

Change-Id: I755b3960ee920ae5289819aa05d98902614a5615
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65151
Maintainer: Earl Ou <shunhsingou@google.com>
Reviewed-by: Yu-hsin Wang <yuhsingw@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-03 05:34:09 +00:00
Jason Lowe-Power
c6918c8f74 python,stdlib: Add multiprocessing module
This changeset replicates some of the multiprocessing module
implementation from the python standard library in gem5. The goal of
this and following changesets is to enable users to use a *single* set
of python scripts to run and analyze a suite of gem5 simulations.

We must reimplement some of the multiprocessing module becaue it is not
flexible enough to allow for customized command line parameter to the
"python" executable (gem5 in our case). To get around this, I extended
the Process and context objects to be gem5 specific.

The next steps is to wrap the Process and Pool types with gem5-specific
versions that will improve their usability for our needs. With this
changeset, these objects are usable, but it will require significant
user effort to reach the goal of running/analyzing many different gem5
simulations.

There are some limitation:
- The pool will only work if the max tasks per child is 1
- The functions that are executed must come from another module

As an example, the following code should work after applying this
change.

test.py:
```python
from gem5.utils.multiprocessing import Process, Pool
from sim import info, run_sim
if __name__ == '__m5_main__' or __name__ == '__main__':
    info('main line')
    p1 = Process(target=run_sim, args=('bob',))
    p2 = Process(target=run_sim, args=('jane',))
    p1.start()
    p2.start()
    p2.join()
    p1.join()
    with Pool(processes=4, maxtasksperchild=1) as pool:
        pool.map(run_sim, range(10))
```

sim.py:
```
import os
def info(title):
    print(title)
    print('module name:', __name__)
    print('parent process:', os.getppid())
    print('process id:', os.getpid())
def run_sim(name):
    info('function g')
    from gem5.prebuilt.demo.x86_demo_board import X86DemoBoard
    from gem5.resources.resource import Resource
    from gem5.simulate.simulator import Simulator
    board = X86DemoBoard()
    board.set_kernel_disk_workload(
        kernel=Resource("x86-linux-kernel-5.4.49"),
        disk_image=Resource("x86-ubuntu-18.04-img"),
    )
    simulator = Simulator(board=board)
    simulator.run(max_ticks=10000000)
```

Change-Id: I4348ebaa75d006949ec96e732f5dc2a5173c6048
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63432
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Richard Cooper <richard.cooper@arm.com>
2022-11-02 20:05:49 +00:00
Bobby R. Bruce
26858db854 stdlib: Refactor the ArmBoard for _connect_things move
Since moving `_connect_things` to a pre-init step, the ArmBoard can now
be refactored to set up things in a more logical manner. In particular,
this patch moves activity out of the `_add_disk_to_board` function and
into the `_pre_initialization` function.

Change-Id: I5d40267f28ae87cd483a0396739c09b8b2b46383
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65052
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2022-11-01 18:57:36 +00:00
Bobby R. Bruce
25d4fb2d91 stdlib: Move _connect_things to run as pre_instantiation
Through working with the gem5 stdlib there have been instances where
connecting the memory, processor, and cache hierarchy to the board (via
the AbstractBoard's `_connect_things` function) at the point of the
AbstractBoard's construction is problematic as the memory, processor,
and cache hierarchy may require information to connect correctly that is
only known to the AbstractBoard after construction. In particular this
can occur when a Workload contains information needed to configure
correctly.

To resolve this problem the `_connect_things` function has been moved to
run as a pre-initialization step. That is, run immediately before
`m5.instantiate`. This is done in the Simulator module.

This will break cases where a user utilizes the stdlib AbstractBoard but
does not use the stdlib Simulator module. As such, an Exception is
raised in these cases explaining the fix to the user. This is done via a
hack where the boards' `createCCObject` function (inheritted
from SimObject) is overriden with a check to ensure `_connect_things`
has been run. To fix the `_pre_instantiate` function must be executed
prior to `m5.instantiate` in the Python configuration script. Test and
config scripts in the gem5 repo have been updated accordingly.

Change-Id: Ibaef36eb7433ce104b861b1da80fc600f08f715a
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65051
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-01 18:57:36 +00:00
Kaustav Goswami
75c1df0d06 stdlib,arch-arm: Add ruby cache support to the ArmBoard
This change adds ruby cache support to the ArmBoard. Previously
only classic caches were supported by the ArmBoard. The ArmBoard
was tested with CHI, MESI_Two_Level and MI_example caches from
the gem5's stdlib.

Change-Id: I480fe6ae13e3bd8438a425548ed113d443fcee40
Signed-off-by: Kaustav Goswami <kggoswami@ucdavis.edu>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64011
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
2022-11-01 18:57:36 +00:00
Jason Lowe-Power
04ac9d9f4f stdlib: Give board interface for mem ports
It is possible that the board has more than just a "main" memory. For
instance, the ArmBoard has a boot memory which is separate from the
`get_memory` function.

This moves the `get_mem_ports` function to the board so that the board
can optionally override it.

Change-Id: I05e388cc93e691e9a4fa674023f158af447349f9
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64631
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-11-01 18:57:36 +00:00
Bobby R. Bruce
3ab6a7496b stdlib: Move setting of checkpoints to set_workload funcs
It never made much sense to set checkpoint via the Simulator module as
Checkpoints are very tightly coupled with the Workload being run. This
change therefore moves the checkpoint to the set_workload functions.

Setting checkpoints via the Simulator is deprecated and will be removed
in a future release.

Change-Id: I24d2133b38a86423d3553ec888c917c5fe47b93d
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64571
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
2022-10-28 00:16:44 +00:00