This commit makes it possible to make invocations such as:
gem5.opt se.py --stats-root 'system.cpu[:].dtb' --stats-root 'system.membus'
When --stats-root is given, only stats that are under any of the root
SimObjects get dumped. E.g. the above invocation would dump stats such as:
system.cpu0.dtb.walker.pwrStateResidencyTicks::UNDEFINED
system.cpu1.dtb.walker.pwrStateResidencyTicks::UNDEFINED
system.membus.pwrStateResidencyTicks::UNDEFINED
system.membus.trans_dist::ReadReq
but not for example `system.clk_domain.clock`.
If the --stats-root is given, only new stats as defined at:
Idc8ff448b9f70a796427b4a5231e7371485130b4 get dumped, and old ones are
ignored. The commits following that one have done some initial conversion
work, but many stats are still in the old format.
Change-Id: Iadaef26edf9a678b39f774515600884fbaeec497
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/28628
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The alignment spaces in stats.txt takes up a lot of space and increases
simulation time, this commit adds the option to disable them with:
--stats-file stats.txt?spaces=False
Sample old lines with ?desc=False:
system.cpu.op_class::FloatMultAcc 0 0.00% 65.92%
system.cpu.op_class::FloatDiv 0 0.00% 65.92%
Sample new lines with ?desc=False;spaces=False:
system.cpu.op_class::FloatMultAcc 0 0.00% 65.92%
system.cpu.op_class::FloatDiv 0 0.00% 65.92%
On a 1000 dumpstats m5op loop spaces=False reduces:
* size: from 38MB to 20MB
* time: from 4.5s to 3.5s
Change-Id: Ib738b996b5646c329094cf61aaa1d977e844e759
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/28627
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This commit changes m5's tick rounding mechanism from python's round()
to python's ceil() function.
Currently, non intergral ticks are rounded by round() function in python.
In python2, this function rounds values >= 0.5 to 1. However, in python3,
0.5 is rounded to 0. This causes the function to return 0 ticks for
non-zero second values, which doesn't make sense, and also causes
several tests to fail.
ceil() function is now used to round up the tick values. This makes more
sense as non-zero second values won't be rounded to zero in any cases.
Signed-off-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Change-Id: I14c43e38e8c678f77baf13407f7eeff4b86f1014
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/29372
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
By default, DOT configs are always generated when pydot is present.
This change allows a user to pass an empty --dot-config='' to disable
generating the DOT configuration. This can be useful to save space, or
to reduce Gem5 startup time when running many small regression tests.
This brings the behavior in-line with providing an empty
--dump_config='' and/or --json_config='' which similarly disables
generation of those output files.
Change-Id: I5bf39fda0409b948a8d14f3afa95db8fc78de6ee
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/29232
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
When the internal utility function make_version_list sees a string, it
tries to convert it into a list using the map() function. In python 3,
that returns an iterator. The following call to zip() will consume those
iterators, and then the following calls to len() will die because they
don't work on map iterators.
This is only a problem if all the common components of the version lists
are equal, and the comparison needs to then check if one of the lists
was equal to the other but with more components. When versions are
equal, for instance when compiling with the oldest supported version of
gcc (4.8.0) this error surfaces and breaks our scons build.
A simple fix is to just wrap the call to map() with list() to convert
the iterator to a flat list, making the other logic work as before.
Change-Id: If9dc5cd7fff70c21229ac3dd9a017edeccd26148
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/28309
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Maintainer: Giacomo Travaglini <giacomo.travaglini@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This change adds a member function to the Group class that returns a
stat given its name. The function will go through all stats in the
group and its subgroups and will return the stat that matches the
name. For example, if g is the Group system.bigCluster.cpus then a
call to
p = g.resolveStat("ipc")
will return a pointer to the stat system.bigCluster.cpus.ipc.
Change-Id: I5af8401b38b41aee611728f6d1a595f99d22d9de
Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/27890
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The components in base/loader were moved into a namespace called
Loader. This will make it easier to add loader components with fairly
short natural names which don't invite name collisions.
gem5 should use namespaces more in general for that reason and to make
it easier to write independent components without having to worry about
name collisions being added in the future.
Unfortunately this namespace has the same name as a class used to load
an object file into a process object. These names can be disambiguated
because the Process loader is inside the Process scope and the Loader
namespace is at global scope, but it's still confusing to read.
Fortunately, this shouldn't last for very long since the responsibility
for loading Processes is going to move to a fake OS object which will
expect to load a particular type of Process, for instance, fake 64 bit
x86 linux will load either 32 or 64 bit x86 processes.
That means that the capability to feed any binary that matches the
current build into gem5 and have gem5 figure out what to do with it
will likely be going away in the future. That's likely for the best,
since it will force users to be more explicit about what they're trying
to do, ie what OS they want to try to load a given binary, and also
will prevent loading two or more Processes which are for different OSes
to the same system, something that's possible today as far as I know
since there are no consistency checks.
Change-Id: Iea0012e98f39f5e20a7c351b78cdff9401f5e326
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/24783
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Python 3 uses the concepts of two different types:
text and binary strings.
Those cannot be implicilty combined (as it was happening in python2) and
in order to be used together one of them must be converted to the other
type:
* Text can be encoded into Bytes via the encode() method
* Bytes can be decoded to Text using the decode() method
By default encode/decode will assume UTF-8 format
JIRA: https://gem5.atlassian.net/browse/GEM5-345
Change-Id: I1bdf7db17b49cc109239fd5f44791769370853f8
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/26250
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
* dict methods dict.keys(), dict.items() and dict.values()
return "views" instead of lists
* The dict.iterkeys(), dict.iteritems() and dict.itervalues()
methods are no longer supported.
* map() and filter() return iterators.
* range() now behaves like xrange() used to behave, except it works with
values of arbitrary size. The latter no longer exists.
* zip() now returns an iterator.
Change-Id: Id480018239db88d7f5d60588c93719056de4a0c0
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/26248
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Python 2.x and Python 3 use different meta class syntax. Fix this by
implementing meta classes using the add_metaclass decorator in the six
Python library.
Due to the way meta classes are implemented in six,
MetaParamValue.__new__ seems to be called twice for some classes. This
triggers an assertion which when param that checks that Param types
have only been registered once. I have turned this assertion into a
warning.
The assertion was triggered in params.CheckedInt and params.Enum. It
seems like the cause of the issue is that these classes have their own
meta classes (CheckedIntType and MetaEnum) that inherit from
MetaParamValue and a base class (ParamValue) that also inherits from
MetaParamValue.
Change-Id: I5dea08bf0558cfca57897a124cb131c78114e59e
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/26083
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
In python 3, the curses escape sequences are bytes objects and not
strings, making them unsuitable to concatenate to strings which are
being print()-ed. This uses the decode() method to turn them from bytes
objects into string objects, assuming they represent UTF-8. In python
2, bytes objects and strings are treated interchangeably, and so this
isn't necessary.
Change-Id: Ifc5d788e1c62751090a350d3a064e89f434559e8
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/23265
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This is the second step towards being able to run dynamically linked
applications when the guest ISA != than host ISA.
Once the guest interpreter is loaded to memory, we are able to redirect
shared object loads through the redirectPath interface.
How do we load the guest interpreter?
The elf file is for example asking for the /lib/ld-linux-aarch64.so
interpreter.
That would point to a valid dynamic linker/loader if guest ISA == host
ISA, but if we are running on X86 we should point to the guest
(aarch64 in the example) toolchain wherever it is installed.
This patch is adding the --interp-dir option to point to the parent
folder of the guest /lib in the host fs.
Change-Id: Id27b97c060008d2e847776a49323d45c8809a27f
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/23066
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>
These tests assume the "end address" is not included in the range. This
exposed some bugs in addr_range.hh which have been fixed. Where
appropriate code comments in addr_range.hh have been extended to improve
understanding of the class's behavior.
Hard-coded AddrRange values in the project have been updated to take
into account that end address is now exclusive. The python params.py
interface has been updated to conform to this new standard.
Change-Id: Idd1e75d5771d198c4b8142b28de0f3a6e9007a52
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/22427
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Descriptions were previously printed on one line, unless explicitly broken
when writing the description of the Sim-Object. In this commit, line
wrapping is enabled when printing these descriptions. Developers, when
writing the Sim-Object descriptions, may now over multiple lines with
triple double-quotes and still have the description output correctly when
viewing the Sim-Objects within the CLI.
E.g.: X86System previously had the following load_addr_mask component which
was output as:
load_addr_mask
default: 18446744073709551615
desc: Address to mask loading binaries with, if 0, system \
auto-calculates the mask to be the most restrictive, otherwise it obeys a \
custom mask.
This was defined by the developer via:
load_addr_mask = Param.UInt64(0xffffffffffffffff,
"Address to mask loading binaries with, if 0, system "
"auto-calculates the mask to be the most restrictive, "
"otherwise it obeys a custom mask.")
This is now displayed as:
load_addr_mask
default: 18446744073709551615
desc: Address to mask loading binaries with, if 0,
system auto-calculates the mask to be the most
restrictive, otherwise it obeys a custom mask.
JiraID: Gem5-57
Built: Linux (GCC)
Tested: Ran quick tests for X86, ARM, and RISC-V
Change-Id: If012304e50af60f6ba10c1fa2b44da8bac1c09cf
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/21179
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Some objects need to know that we are about to dump stats to perform
prepare statistics. This is currently done by registering a callback
with the stat system. Expose this callback as a virtual method
in Stats::Group to make this pattern more convenient.
Change-Id: I5aa475b7d04c288e45f5f413ab7a1907b971dae5
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/21139
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
New-world stats didn't implement the old gem5 quirk where the index of
an element in a vector of size 1 is ignored. This affects the object
path when printed in a stat file. Implement the same quirk as the
original code to make sure names of old- and new-world stats are
compatible.
Change-Id: I12696d40baaed2d0bb1ef061e87c3b1d0f4bb4b1
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/21163
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
This changeset add support for stat dumps in the HDF5 file
format. HDF5 is a binary data format that represents data in a
file-system-like balanced tree. It has native support for
N-dimensional arrays and binary data (e.g., frame buffers).
It has the following benefits over traditional text stat files:
* Efficient storage of time series (multiple stat dumps)
* Fast lookup of stats
* Plenty of existing tooling (e.g., Python libraries and graphical
viewers)
* File format can be used to store frame buffers together with
normal stats.
Drawbacks:
* Large startup cost (single stat dump larger than text equivalent)
* Stat dumps are slower than text
Known limitations:
* Distributions and histograms aren't supported.
HDF5 stat output can be enabled using the 'h5' URL scheme when
overriding the stat file name on gem5's command line. The following
parameters are supported:
* chunking (unsigned): Number of time steps to pre-allocate
(default: 10)
* desc (bool): Output stat descriptions (default: True)
* formulas (bool): Output derived stats (default: True)
Example gem5 command line:
./build/ARM/gem5.opt \
--stats-file="h5://stats.h5?desc=False;formulas=False" \
configs/example/fs.py
Example Python stat consumer that computes IPC:
import h5py
f = h5py.File('stats.h5', 'r')
group = f['/system/cpu']
for i, c in zip(group['committedInsts'], group['numCycles']):
print i, c, i / c
Change-Id: I351c6cbff2fb7bef9012f47876ba227ed288975b
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/8121
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Ciro Santilli <ciro.santilli@arm.com>
Add support for partial stat dumps by passing an optional 'root'
keyword argument to m5.stats.dump(). Specifying root slightly changes
the semantics of the dump command. For legacy reasons, gem5 only
allows one stat dump per tick. This is likely a limitation introduced
as a hack to prevent automatic dumping at the end of simulation from
interfering with explicit dumping from a simulation script. This
restriction does not apply when specifying a root. However, these stat
dumps will still prevent an additional stat dump in the same tick with
an unspecified root.
N.B.: This new API /only/ works for new-style stats that have an
explicit hierarchy. Legacy stats will not be dumped if a root is
specified.
Change-Id: Idc8ff448b9f70a796427b4a5231e7371485130b4
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19369
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This change makes the stat system aware of the hierarchical nature of
stats. The aim is to achieve the following goals:
* Make the SimObject hierarchy explicit in the stat system (i.e.,
get rid of name() + ".foo"). This makes stat naming less fragile
and makes it possible to implement hierarchical formats like
XML/HDF5/JSON in a clean way.
* Make it more convenient to split stats into a separate
struct/class that can be bound to a SimObject. This makes the
namespace cleaner and makes stat accesses a bit more obvious.
* Make it possible to build groups of stats in C++ that can be used
in subcomponents in a SimObject (similar to what we do for
checkpoint sections). This makes it easier to structure large
components.
* Enable partial stat dumps. Some of our internal users have been
asking for this since a full stat dump can be large.
* Enable better stat access from Python.
This changeset implements solves the first three points by introducing
a class (Stats::Group) that owns statistics belonging to the same
object. SimObjects inherit from Stats::Group since they typically have
statistics.
New-style statistics need to be associated with a parent group at
instantiation time. Instantiation typically sets the name and the
description, other parameters need to be set by overriding
Group::regStats() just like with legacy stats. Simple objects with
scalar stats can typically avoid implementing regStats() altogether
since the stat name and description are both specified in the
constructor.
For convenience reasons, statistics groups can be merged into other
groups. This means that a SimObject can create a stat struct that
inherits from Stats::Group and merge it into the parent group
(SimObject). This can make the code cleaner since statistics tracking
gets grouped into a single object.
Stat visitors have a new API to expose the group structure. The
Output::beginGroup(name) method is called at the beginning of a group
and the Output::endGroup() method is called when all stats, and
sub-groups, have been visited. Flat formats (e.g., the text format)
typically need to maintain a stack to track the full path to a stat.
Legacy, flat, statistics are still supported after applying this
change. These stats don't belong to any group and stat visitors will
not see a Output::beginGroup(name) call before their corresponding
Output::visit() methods are called.
Change-Id: I9025d61dfadeabcc8ecf30813ab2060def455648
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19368
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
In large configs the tooltip may be greater then the maximum line
size graphviz supports when parsing the dot file (typically 16k).
Adding '/' causes graphviz to break the string in multiple lines
while parsing and works around this limitation.
Change-Id: I16a0030127de4165080de97f5213309eed9fdeca
Signed-off-by: Tiago Mück <tiago.muck@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19208
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Previously an AddrRange could express interleaving using a number of
consecutive bits and in additional optionally a second number of
consecutive bits. The two sets of consecutive bits would be xored and
matched against a value to determine if an address is in the
AddrRange. For example:
sel[0] = a[8] ^ a[12]
sel[1] = a[9] ^ a[13]
where sel == intlvMatch
This change extends AddrRange to allow more flexible interleavings
with an abritary number of set of bits which do not need be
consecutive. For example:
sel[0] = a[8] ^ a[11] ^ a[13]
sel[1] = a[15] ^ a[17] ^ a[19]
where sel == intlvMatch
Change-Id: I42220a6d5011a31f0560535762a25bfc823c3ebb
Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19130
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
MemObject doesn't provide anything beyond its base ClockedObject any
more, so this change removes it from most inheritance hierarchies.
Occasionally MemObject is replaced with SimObject when I was fairly
confident that the extra functionality of ClockedObject wasn't needed.
Change-Id: Ic014ab61e56402e62548e8c831eb16e26523fdce
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/18289
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Anthony Gutierrez <anthony.gutierrez@amd.com>
Maintainer: Gabe Black <gabeblack@google.com>
A recent change got rid of the strict Master => Slave port relationship
which used to be checked in python and instead left the checking up to
C++. One major downside to this approach is that it was no longer
obvious in the configuration what was supposed to be connected to what,
and it still left the arbitrary and misleading MasterPort and SlavePort
types in the Ethernet devices which could now connect with each other
symmetrically but couldn't actually connect to an arbitrary
MasterPort/SlavePort.
This change exposes the base Port and VectorPort types, and makes them
accept a "role" parameter in __init__ which used to be set directly by
their subclasses. This role can be any string, and will be used later
to check for compatiblity and to give a hint as to what can be
connected to what in the SimObject definitions.
To make the checks work with arbitrary compatible pairs, the base Port
type now has a class method called compat() which accepts a pair of
roles which will become mutually compatible, ie any port with the first
role will be allowed to connect to any port with the second role, and
vice versa. To be self compatible, the same role should be passed in
for both parameters.
To maintain compatibility, the MasterPort and SlavePort types are
retained, but now they're nothing special and could have been set up
in any arbitrary SimObject .py file. The same is true for
MasterVectorPort and SlaveVectorPort.
Also, since we can no longer assume that all edges in the dot graph of
the config should start with a port with the MASTER role and end with
a port with the SLAVE role, Ports now track an is_source property which
says whether the arrow head should be surpressed at that end of the
edge representing the connection.
Change-Id: Ifcc6faab05e437ad87cd21f0ba613b09cf21c321
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/18168
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Gabe Black <gabeblack@google.com>
