This has two purposes. First, SCons assumes that once you call
Configure, you won't set up the environment the Configure is based on
until after you get the environment back from it again with
conf.Finish(). We get away with this when the cache mode for config
tests is not "force", since Configure just reuses the environment we
pass in, and any changes we make are immediately communicated between
the two.
If the cache mode *is* "force" though, SCons modifies the decider so
that everything the conf environment goes to build looks like it's out
of date. It does that by cloning the original environment, and then
using that clone to do its tests. That causes a problem because we have
a long lived "conf" object and make further changes to main, and since
the two environments are now separate the one in conf doesn't see those
updates.
Second, and more subtly, we export our "main" and "env" environments so
that other SConsopts and SConscript files can use them and define things
in them. The way Configure is designed, if the config caching mode is
"force", then it will create a new environment, and then that
environment will replace what the, for instance, "main" variable points
to when "main = conf.Finish()" is executed.
Unfortunately, if we've already Export()-ed main, we've exported what
the "main" variable pointed to at that time. Our view of "main" will
track with the value that conf.Finish() returned, but since that
construction environment is mearly derived from the main we Exported and
not actually the same thing, they have diverged at that point and will
behave independently.
To solve both of these problems, this change modifies the
gem5_scons.Configure() method so that it's a context manager instead of
a regular function. As before, it will call Configure for us and create
a configuration context, which it will yield as the "with" value. When
the context exits, all the variables in the context Finish() returns
will be shoved back into the original context with Replace(). This isn't
perfect since variables which were deleted in the environment (probably
very rare in practice) will not exist and so will not overwrite the
still existent variable in the original dict.
This has several advantages. The environment never splits into two
copies which continue on independently. It makes the lifetime of a
configuration context short, which is good because behavior during that
time is tricky and unintuitive. It also makes the scope of the context
very clear, so that you won't miss the fact that you're in a special
setting and need to pay attention to what environment you're modifying.
Also, this keeps the conceptual overhead of configuration localized to
where the configuration is happening. In parts of the SConscripts which
are not doing anything with conf, etc, they don't have to modify their
behavior since no configuration context is active.
This change is based on this change from Hanhwi Jang who identified this
problem and proposed an initial solution:
https://gem5-review.googlesource.com/c/public/gem5/+/44265
Change-Id: Iae0a292d6b375c5da98619f31392ca1de6216fcd
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/44389
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Hanhwi Jang <jang.hanhwi@gmail.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Use SConsopts files local to individual domains to pull
non-foundational build code out of SConstruct. This greatly simplifies
SConstruct, and also makes it easier to find build configuration having
to do with particular pieces of gem5.
This change also converts some python level variables, all_protocols,
protocol_dirs, and slicc_includes, into the environment where the timing
of their initialization is more flexible.
Change-Id: Ie61ceb75ae9e5557cc400603c972a9582e99c1ea
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40872
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gabe Black <gabe.black@gmail.com>
The systemc dir was not included in this fix.
First it was identified that there were only occurrences
at 0, 1, and 2 levels of indentation (and 2 of 2 spaces,
1 of 3 spaces and 2 of 12 spaces), using:
grep -nrE --exclude-dir=systemc \
"^ *enum [A-Za-z].* {$" src/
Then the following commands were run to replace:
<indent level>enum X ... {
by:
<indent level>enum X ...
<indent level>{
Level 0:
grep -nrl --exclude-dir=systemc \
"^enum [A-Za-z].* {$" src/ | \
xargs sed -Ei \
's/^enum ([A-Za-z].*) \{$/enum \1\n\{/g'
Level 1:
grep -nrl --exclude-dir=systemc \
"^ enum [A-Za-z].* {$" src/ | \
xargs sed -Ei \
's/^ enum ([A-Za-z].*) \{$/ enum \1\n \{/g'
and so on.
Change-Id: Ib186cf379049098ceaec20dfe4d1edcedd5f940d
Signed-off-by: Daniel R. Carvalho <odanrc@yahoo.com.br>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/43326
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Maintainer: Giacomo Travaglini <giacomo.travaglini@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The instruction representation is already encoded in the trace
protobuf, so there's no reason to encode a part of it again. This is
especially true since this supposedly generic code is extracting the
first 8 bits of the machInst, a totally arbitrary set of bits for most
ISAs. If certain bits within a machine instruction are actually
relevant, the consumer of the trace should be able to interpret the
instruction bytes which are already there and extract the same bits
within the context of whatever ISA they're appropriate for.
Change-Id: Idaebe6a110d7d4812c3d7c434582d5a9470bcec1
Reviewed-on: https://gem5-review.googlesource.com/9401
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Gabe Black <gabeblack@google.com>
This change adds an inst_bytes field which is of type bytes, and puts
it in a oneof with the previously required inst field. If an
instruction's encoding happens to be 4 bytes long, the original inst
field will be used. Otherwise, the new variably sized inst_bytes field
will be used.
Because this tracer doesn't have visibility into how the data in
inst_bytes is structured, it can't do any endian conversion itself.
To maintain compatibility between producers and consumers who may have
different endiannesses, all data should be manually converted to
little endian before being stored in this field.
inst will be converted into little endian by protobuf, and so
compatibility doesn't have to be handled manually.
Change-Id: I290713f70e7124d8aa9550c022c71334939d84a6
Reviewed-on: https://gem5-review.googlesource.com/7561
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Gabe Black <gabeblack@google.com>
These files aren't a collection of miscellaneous stuff, they're the
definition of the Logger interface, and a few utility macros for
calling into that interface (panic, warn, etc.).
Change-Id: I84267ac3f45896a83c0ef027f8f19c5e9a5667d1
Reviewed-on: https://gem5-review.googlesource.com/6226
Reviewed-by: Brandon Potter <Brandon.Potter@amd.com>
Maintainer: Gabe Black <gabeblack@google.com>
This map is intended to map from request MasterIDs to master names. It could
be used to map to arbitrary strings in other situations, however.
The original idea to store this information was to add a new message type
which would store one ID and the string associated with it. This change stores
the IDs in the header instead so that they'll be easy to find and all
available before the packet data.
One downside of this approach is that it won't be possible to add new master
ID strings as they come up during a trace. If that becomes an issue, the two
approaches could be combined and messages could be added which would augment
the map in the header.
Also worth mentioning is that the proto2 version of the protobuf description
language does not support the "map" field type, and the protoc compiler on my
workstation doesn't support proto3. Because that's such an appropriate
representation for this data, the map is represented in an equivalent format
described in the proto3 documentation.
Change-Id: I137c8611c33d9ce6589e196d50c8638c1d88750c
Reviewed-on: https://gem5-review.googlesource.com/4782
Reviewed-by: Rahul Thakur <rjthakur@google.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
protoc v3 introduces a new syntax for proto files and warns when the
syntax is not explicitly stated.
protoc relies on the fact that undefined preprocessor symbols are
explanded to 0 but since we use -Wundef they end up generating
warnings.
Change-Id: If07abeb54e932469c8f2c4d38634a97fdae40f77
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
This patch adds support to optionally capture the virtual address and asid
for load/store instructions in the elastic traces. If they are present in
the traces, Trace CPU will set those fields of the request during replay.
This patch replaces the booleans that specified the elastic trace record
type with an enum type. The source of change is the proto message for
elastic trace where the enum is introduced. The struct definitions in the
elastic trace probe listener as well as the Trace CPU replace the boleans
with the proto message enum.
The patch does not impact functionality, but traces are not compatible with
previous version. This is preparation for adding new types of records in
subsequent patches.
The elastic trace is a type of probe listener and listens to probe points
in multiple stages of the O3CPU. The notify method is called on a probe
point typically when an instruction successfully progresses through that
stage.
As different listener methods mapped to the different probe points execute,
relevant information about the instruction, e.g. timestamps and register
accesses, are captured and stored in temporary InstExecInfo class objects.
When the instruction progresses through the commit stage, the timing and the
dependency information about the instruction is finalised and encapsulated in
a struct called TraceInfo. TraceInfo objects are collected in a list instead
of writing them out to the trace file one a time. This is required as the
trace is processed in chunks to evaluate order dependencies and computational
delay in case an instruction does not have any register dependencies. By this
we achieve a simpler algorithm during replay because every record in the
trace can be hooked onto a record in its past. The instruction dependency
trace is written out as a protobuf format file. A second trace containing
fetch requests at absolute timestamps is written to a separate protobuf
format file.
If the instruction is not executed then it is not added to the trace.
The code checks if the instruction had a fault, if it predicated
false and thus previous register values were restored or if it was a
load/store that did not have a request (e.g. when the size of the
request is zero). In all these cases the instruction is set as
executed by the Execute stage and is picked up by the commit probe
listener. But a request is not issued and registers are not written.
So practically, skipping these should not hurt the dependency modelling.
If squashing results in squashing younger instructions, it may happen that
the squash probe discards the inst and removes it from the temporary
store but execute stage deals with the instruction in the next cycle which
results in the execute probe seeing this inst as 'new' inst. A sequence
number of the last processed trace record is used to trap these cases and
not add to the temporary store.
The elastic instruction trace and fetch request trace can be read in and
played back by the TraceCPU.
The namespace Message conflicts with the Message data type used extensively
in Ruby. Since Ruby is being moved to the same Master/Slave ports based
configuration style as the rest of gem5, this conflict needs to be resolved.
Hence, the namespace is being renamed to ProtoMessage.
This patch changes the decode script to output the optional fields of
the proto message Packet, namely id and flags. The flags field is set
by the communication monitor.
The id field is useful for CPU trace experiments, e.g. linking the
fetch side to decode side. It had to be renamed because it clashes
with a built in python function id() for getting the "identity" of an
object.
This patch also takes a few common function definitions out from the
multiple scripts and adds them to a protolib python module.
This patch changes how the streams are created to avoid the size
limitation on the coded streams. As we only read/write a single
message at a time, there is never any message larger than a few
bytes. However, the coded stream eventually complains that its
internal counter reaches 64+ MByte if the total file size exceeds this
value.
Based on suggestions in the protobuf discussion forums, the coded
stream is now created for every message that is read/written. The
result is that the internal byte count never goes about tens of bytes,
and we can read/write any size file that the underlying file I/O can
handle.
This patch adds an optional generic 64-bit identifier field to the
packet trace. This can be used to store the sequential number of the
instruction that gave rise to the packet, thread id, master id,
"sub"-master within a larger module etc. As the field is optional it
has a marginal cost if not used.
This patch adds an optional flags field to the packet trace to encode
the request flags that contain information about whether the request
is (un)cacheable, instruction fetch, preftech etc.
This patch adds support for inputting protobuf messages through a
ProtoInputStream which hides the internal streams used by the
library. The stream is created based on the name of an input file and
optionally includes decompression using gzip.
The input stream will start by getting a magic number from the file,
and also verify that it matches with the expected value. Once opened,
messages can be read incrementally from the stream, returning
true/false until an error occurs or the end of the file is reached.
This patch adds packet tracing to the communication monitor using a
protobuf as the mechanism for creating the trace.
If no file is specified, then the tracing is disabled. If a file is
specified, then for every packet that is successfully sent, a protobuf
message is serialized to the file.
This patch adds support for outputting protobuf messages through a
ProtoOutputStream which hides the internal streams used by the
library. The stream is created based on the name of an output file and
optionally includes compression using gzip.
The output stream will start by putting a magic number in the file,
and then for every message that is serialized prepend the size such
that the stream can be written and read incrementally. At this point
this merely serves as a proof of concept.
