Some objects need to know that we are about to dump stats to perform
prepare statistics. This is currently done by registering a callback
with the stat system. Expose this callback as a virtual method
in Stats::Group to make this pattern more convenient.
Change-Id: I5aa475b7d04c288e45f5f413ab7a1907b971dae5
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/21139
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Info::setName() performs a sanity check to ensure that the same stat
name isn't used twice. This doesn't work for new-style stats with a
parent group since the name is only unique within the group. Disable
the check for new-style stats since these usually use names generated
from member variable names.
Change-Id: I590abe6040407c6a4fe582c0782a418165ff5588
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/20760
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>
When using the HDF5 stats backend, we currently pass the filename from
the command line straight to the HDF5 library. This behaviour is
different from the expected behaviour when using plain text stat files
where relative paths are resolved relative to simout. This change adds
support for resolving hdf5 paths relative to simout.
Change-Id: I09cb0b7468735e697125eda7f04c5a85725ca8d0
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/20508
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
This changeset add support for stat dumps in the HDF5 file
format. HDF5 is a binary data format that represents data in a
file-system-like balanced tree. It has native support for
N-dimensional arrays and binary data (e.g., frame buffers).
It has the following benefits over traditional text stat files:
* Efficient storage of time series (multiple stat dumps)
* Fast lookup of stats
* Plenty of existing tooling (e.g., Python libraries and graphical
viewers)
* File format can be used to store frame buffers together with
normal stats.
Drawbacks:
* Large startup cost (single stat dump larger than text equivalent)
* Stat dumps are slower than text
Known limitations:
* Distributions and histograms aren't supported.
HDF5 stat output can be enabled using the 'h5' URL scheme when
overriding the stat file name on gem5's command line. The following
parameters are supported:
* chunking (unsigned): Number of time steps to pre-allocate
(default: 10)
* desc (bool): Output stat descriptions (default: True)
* formulas (bool): Output derived stats (default: True)
Example gem5 command line:
./build/ARM/gem5.opt \
--stats-file="h5://stats.h5?desc=False;formulas=False" \
configs/example/fs.py
Example Python stat consumer that computes IPC:
import h5py
f = h5py.File('stats.h5', 'r')
group = f['/system/cpu']
for i, c in zip(group['committedInsts'], group['numCycles']):
print i, c, i / c
Change-Id: I351c6cbff2fb7bef9012f47876ba227ed288975b
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/8121
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Ciro Santilli <ciro.santilli@arm.com>
This change makes the stat system aware of the hierarchical nature of
stats. The aim is to achieve the following goals:
* Make the SimObject hierarchy explicit in the stat system (i.e.,
get rid of name() + ".foo"). This makes stat naming less fragile
and makes it possible to implement hierarchical formats like
XML/HDF5/JSON in a clean way.
* Make it more convenient to split stats into a separate
struct/class that can be bound to a SimObject. This makes the
namespace cleaner and makes stat accesses a bit more obvious.
* Make it possible to build groups of stats in C++ that can be used
in subcomponents in a SimObject (similar to what we do for
checkpoint sections). This makes it easier to structure large
components.
* Enable partial stat dumps. Some of our internal users have been
asking for this since a full stat dump can be large.
* Enable better stat access from Python.
This changeset implements solves the first three points by introducing
a class (Stats::Group) that owns statistics belonging to the same
object. SimObjects inherit from Stats::Group since they typically have
statistics.
New-style statistics need to be associated with a parent group at
instantiation time. Instantiation typically sets the name and the
description, other parameters need to be set by overriding
Group::regStats() just like with legacy stats. Simple objects with
scalar stats can typically avoid implementing regStats() altogether
since the stat name and description are both specified in the
constructor.
For convenience reasons, statistics groups can be merged into other
groups. This means that a SimObject can create a stat struct that
inherits from Stats::Group and merge it into the parent group
(SimObject). This can make the code cleaner since statistics tracking
gets grouped into a single object.
Stat visitors have a new API to expose the group structure. The
Output::beginGroup(name) method is called at the beginning of a group
and the Output::endGroup() method is called when all stats, and
sub-groups, have been visited. Flat formats (e.g., the text format)
typically need to maintain a stack to track the full path to a stat.
Legacy, flat, statistics are still supported after applying this
change. These stats don't belong to any group and stat visitors will
not see a Output::beginGroup(name) call before their corresponding
Output::visit() methods are called.
Change-Id: I9025d61dfadeabcc8ecf30813ab2060def455648
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19368
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
These files aren't a collection of miscellaneous stuff, they're the
definition of the Logger interface, and a few utility macros for
calling into that interface (panic, warn, etc.).
Change-Id: I84267ac3f45896a83c0ef027f8f19c5e9a5667d1
Reviewed-on: https://gem5-review.googlesource.com/6226
Reviewed-by: Brandon Potter <Brandon.Potter@amd.com>
Maintainer: Gabe Black <gabeblack@google.com>
This patch adds a total() function to the Vector2D
stat type. Similar to other stats such as Scalar or
Vector it is useful to be able to read the total for
a given stat.
This changeset adds support for changing the simulator output
directory. This can be useful when the simulation goes through several
stages (e.g., a warming phase, a simulation phase, and a verification
phase) since it allows the output from each stage to be located in a
different directory. Relocation is done by calling core.setOutputDir()
from Python or simout.setOutputDirectory() from C++.
This change affects several parts of the design of the gem5's output
subsystem. First, files returned by an OutputDirectory instance (e.g.,
simout) are of the type OutputStream instead of a std::ostream. This
allows us to do some more book keeping and control re-opening of files
when the output directory is changed. Second, new subdirectories are
OutputDirectory instances, which should be used to create files in
that sub-directory.
Signed-off-by: Andreas Sandberg <andreas@sandberg.pp.se>
[sascha.bischoff@arm.com: Rebased patches onto a newer gem5 version]
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
This patch does a bit of housekeeping on the string helper functions
and relies on the C++11 standard library where possible. It also does
away with our custom string hash as an implementation is already part
of the standard library.
This patch adds a function to the HistStor class for adding two histograms.
This functionality is required for Ruby. It also adds support for printing
histograms in a single line.
This patch addresses an issue with the text-based stats output which
resulted in Vector2D stats being printed without subnames in the event
that one of the dimensions was of length 1.
This patch also fixes the total printing for the 2D vector. Previously
totals were printed without explicitly stating that a total was being
printed. This has been rectified in this patch.
This patch removes the printing of the SparseHist total in the
stats.txt output file. This has been removed as a sparse histogram has
no total, and therefore this was printing out the value of a
non-local, unrelated variable.
This patch adds a new flag to specify if the data values for a given vector
should be printed in one line in the stats.txt file. The default behavior
will be to print the data in multiple lines. It makes changes to print
functions to enforce this behavior.
This patch checks that the compiler in use is either gcc >= 4.4 or
clang >= 2.9. and enables building with --std=c++0x in all cases. As a
consequence, we can tidy up the hashmap and always have static_assert
available. If anyone wants to use alternative compilers, icc for
example supports c++0x to a similar level and could be added if
needed.
This patch opens up for a more elaborate use of c++0x features that
are present in gcc 4.4 and clang 2.9, e.g. auto typed variables,
variadic templates, rvalues and move semantics, and strongly typed
enums. There will be no going back on this one...
This patch addresses a number of minor issues that cause problems when
compiling with clang >= 3.0 and gcc >= 4.6. Most importantly, it
avoids using the deprecated ext/hash_map and instead uses
unordered_map (and similarly so for the hash_set). To make use of the
new STL containers, g++ and clang has to be invoked with "-std=c++0x",
and this is now added for all gcc versions >= 4.6, and for clang >=
3.0. For gcc >= 4.3 and <= 4.5 and clang <= 3.0 we use the tr1
unordered_map to avoid the deprecation warning.
The addition of c++0x in turn causes a few problems, as the
compiler is more stringent and adds a number of new warnings. Below,
the most important issues are enumerated:
1) the use of namespaces is more strict, e.g. for isnan, and all
headers opening the entire namespace std are now fixed.
2) another other issue caused by the more stringent compiler is the
narrowing of the embedded python, which used to be a char array,
and is now unsigned char since there were values larger than 128.
3) a particularly odd issue that arose with the new c++0x behaviour is
found in range.hh, where the operator< causes gcc to complain about
the template type parsing (the "<" is interpreted as the beginning
of a template argument), and the problem seems to be related to the
begin/end members introduced for the range-type iteration, which is
a new feature in c++11.
As a minor update, this patch also fixes the build flags for the clang
debug target that used to be shared with gcc and incorrectly use
"-ggdb".
This patch adds a mechanism to collect run time samples for specific portions
of a benchmark, using work_begin and work_end pseudo instructions.It also enhances
the histogram stat to report geometric mean.
Due to certain changes made via changeset 8229, the compilation was failing
in certain cases. The compiler pointed to base/stats/mysql.hh for not naming
a certain types like uint64_t. To rectify this, base/types.hh is being
included in base/stats/mysql.hh.
Ran all the source files through 'perl -pi' with this script:
s|\s*(};?\s*)?/\*\s*(end\s*)?namespace\s*(\S+)\s*\*/(\s*})?|} // namespace $3|;
s|\s*};?\s*//\s*(end\s*)?namespace\s*(\S+)\s*|} // namespace $2\n|;
s|\s*};?\s*//\s*(\S+)\s*namespace\s*|} // namespace $1\n|;
Also did a little manual editing on some of the arch/*/isa_traits.hh files
and src/SConscript.
Time from base/time.hh has a name clash with Time from Ruby's
TypeDefines.hh. Eventually Ruby's Time should go away, so instead of
fixing this properly just try to avoid the clash.
