While there was code present in "serialize.cc" to create the checkpoint
directory, it did not do recursively. This patch ensures all the
directories are created in a path to the checkpoint directory.
Change-Id: Ibcf7f800358fd89946f550b8cfb0cef8b51fceac
This change adds a new file to m5out which is citations.bib.
This file will contain the citations to the papers which describe the
aspects of the gem5 simulator that the simulation uses. In other words,
each simulation configuration could generate a different bib file
referencing different works.
Each SimObject can now have a set of citations associated with it. After
the system is built (in `instantiate`), the citations.bib file is
created by parsing all SimObjects that have been instantiated and taking
the union of their associated citations.
This commit is not meant to add all citations, but to act as an example
for others to add more citations to gem5.
Change-Id: Icd5c46fd9ee44adbeec1fea162657f5716f7e5ef
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
This commit adds the following functions to the `m5` python module:
- setMaxTick(tick) -> None
- getMaxTick() -> int
- getTicksUntilMax() -> int
- scheduleTickExitFromCurrent(tick, exit_string) -> None
- scheduleTickExitAbsolute(tick, exit_string) -> None
Until this patch the only way to set an exit at a particular tick was
via `simulate.run` which would reschedule the maximum tick. This
functionality has been explicity exposed via the new `setMaxTick`
function. However, as this is only rescheduling the maximum tick, it
stops scheduling exits at multiple different ticks.
To get around this problem the `scheduleTickExit` functions have been
added. These allow a user to schedule multiple exit events. The
functions contain a `exit_string` parameter that provides the string
the simulator is to return when the specified tick is met. By default
this string is "Tick exit reached" which is used by the stdlib
Simulator module to declare a new `SCHEDULED_TICK` exit event (Note:
this has been deliberatly kept seperate from the `MAX_TICK` exit event.
This commit serves as an attempt to decouple these are two concepts).
Tests are provided in this patch to ensure these new functions work as
intended.
Additional notes:
- The `simulate` function has been fixed to match the documentation. If
the `num_cycles` is -1 then the maximum ticks is set to MaxTicks.
Otherwise the max ticks is set to `curTicks() + num_cycles`. The
functionality of this function will remain unchanged to the end-user.
- Full integration into the Simulator module is not complete as of this
patch. Users must us the m5 python module to set these exit events.
Change-Id: I6c92b31dd409dc866152224600ea8166cfcba38b
Issue-on: https://gem5.atlassian.net/browse/GEM5-1131
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/66231
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
It is currently not possible to call m5.fork when the simulator is
running in with multiple parallel event queues. The POSIX standard
have very weak guarantees when forking a process with multiple
threads. In order to use fork correctly, we need to ensure that all
helper threads servicing event queues have terminated before the fork
system call is invoked.
There are two ways this could be implemented: 1) Always terminate
helper threads when taking a global simulator exit event, or 2)
terminate helper threads just before fork is called from Python.
This change implements the second strategy since the KVM-based CPUs
currently assume that TIDs don't change unless there is a fork event.
Change-Id: I22feaecd49f7f81689b43185d63a8f14428bed63
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/50408
Reviewed-by: Austin Harris <mail@austin-harris.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
With this change serialize.hh is no longer responsible
for the (un)serialization of events. As a general rule,
rules to (un)serialize non-basic types should be defined
at the file that introduces that type. Therefore,
(UN)SERIALIZE_EVENT have been moved to eventq.hh.
Globals has a single instance which must be serialized
and unserialized. Instead of having a stray global
variable handled by Serialization, we pass its management
to Root. As a side effect, Globals is assigned its own
files: sim/globals.(cc/hh).
Finally, 'unserializeGlobals()' is removed, so that
Root can fully handle Globals' serialization. This
breaks checkpoint compatibility, so a checkpoint
upgrader is added.
Change-Id: I9c8e57306f83f9cc30ab2b745a4972755191bec4
Signed-off-by: Daniel R. Carvalho <odanrc@yahoo.com.br>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/43586
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Occasionally gem5's stdout/stderr, when run within the TestLib
framework, will be shuffled. This is resolved by flushing the
stdout/stderr buffer before and after simulation.
In addition to this, the verifier.py has been improved to remove
boilerplate gem5 code from the stdout comparison.
Change-Id: I04c8f9cee4475b8eab2f1ba9bb76bfa3cfcca6ec
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/34995
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
By default, DOT configs are always generated when pydot is present.
This change allows a user to pass an empty --dot-config='' to disable
generating the DOT configuration. This can be useful to save space, or
to reduce Gem5 startup time when running many small regression tests.
This brings the behavior in-line with providing an empty
--dump_config='' and/or --json_config='' which similarly disables
generation of those output files.
Change-Id: I5bf39fda0409b948a8d14f3afa95db8fc78de6ee
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/29232
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This change makes the stat system aware of the hierarchical nature of
stats. The aim is to achieve the following goals:
* Make the SimObject hierarchy explicit in the stat system (i.e.,
get rid of name() + ".foo"). This makes stat naming less fragile
and makes it possible to implement hierarchical formats like
XML/HDF5/JSON in a clean way.
* Make it more convenient to split stats into a separate
struct/class that can be bound to a SimObject. This makes the
namespace cleaner and makes stat accesses a bit more obvious.
* Make it possible to build groups of stats in C++ that can be used
in subcomponents in a SimObject (similar to what we do for
checkpoint sections). This makes it easier to structure large
components.
* Enable partial stat dumps. Some of our internal users have been
asking for this since a full stat dump can be large.
* Enable better stat access from Python.
This changeset implements solves the first three points by introducing
a class (Stats::Group) that owns statistics belonging to the same
object. SimObjects inherit from Stats::Group since they typically have
statistics.
New-style statistics need to be associated with a parent group at
instantiation time. Instantiation typically sets the name and the
description, other parameters need to be set by overriding
Group::regStats() just like with legacy stats. Simple objects with
scalar stats can typically avoid implementing regStats() altogether
since the stat name and description are both specified in the
constructor.
For convenience reasons, statistics groups can be merged into other
groups. This means that a SimObject can create a stat struct that
inherits from Stats::Group and merge it into the parent group
(SimObject). This can make the code cleaner since statistics tracking
gets grouped into a single object.
Stat visitors have a new API to expose the group structure. The
Output::beginGroup(name) method is called at the beginning of a group
and the Output::endGroup() method is called when all stats, and
sub-groups, have been visited. Flat formats (e.g., the text format)
typically need to maintain a stack to track the full path to a stat.
Legacy, flat, statistics are still supported after applying this
change. These stats don't belong to any group and stat visitors will
not see a Output::beginGroup(name) call before their corresponding
Output::visit() methods are called.
Change-Id: I9025d61dfadeabcc8ecf30813ab2060def455648
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19368
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Starting with version 3, scons imposes using the print function instead
of the print statement in code it processes. To get things building
again, this change moves all python code within gem5 to use the
function version. Another change by another author separately made this
same change to the site_tools and site_init.py files.
Change-Id: I2de7dc3b1be756baad6f60574c47c8b7e80ea3b0
Reviewed-on: https://gem5-review.googlesource.com/8761
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Gabe Black <gabeblack@google.com>
Swig wrappers for native objects currently share the _m5.internal name
space with Python code. This is undesirable if we ever want to switch
from Swig to some other framework for native binding (e.g., PyBind11
or Boost::Python). This changeset moves all of such wrappers to the
_m5 namespace, which is now reserved for native code.
Change-Id: I2d2bc12dbc05b57b7c5a75f072e08124413d77f3
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
This changeset updates the dot output to bail out if it is unable to
resolve the voltage or clock domains (which will cause it to raise an
AttributeError). Additionally, the DVFS dot output is disabled by
default for speed purposes.
Minor fixup for 0aeca8f.
This patch adds a secondary dot output file which shows the DVFS domains. This
has been done separately for now to avoid cluttering the already existing
diagram. Due to the way that the clock domains are assigned to components in
gem5, this output must be generated after the C++ objects have been
instantiated. This further motivates the need to generate this file separately
to the current dot output, and not to replace it entirely.
This changeset adds forking capabilities to the gem5 python scripts. A fork
method is added to simulate.py. This method is responsible for forking the
simulator itself, and will direct all output files to a new output directory
based on the fork sequence number. The default name of the output directory is
the same as the parent with the suffix ".fN" added where N is the fork sequence
number. The fork method provides the option to specify if the system should be
drained prior to forking, or not. By default the system is drained to ensure
that there are no in-flight transactions.
When forking the simulator, the fork method returns the PID of the child
process, or returns 0 if running in the child. This is in line with the standard
Python forking interface.
Signed-off-by: Andreas Sandberg <andreas@sandberg.pp.se>
[sascha.bischoff@arm.com: Rebased patches onto a newer gem5 version]
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
[andreas.sandberg@arm.com: Updated to comply with modern draining semantics ]
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
When forking a gem5 process, some objects need to clean up resources
(mainly file descriptions) shared between the child and the parent of
the fork. This changeset adds the notifyFork() method to Drainable,
which is called in the child process.
Signed-off-by: Andreas Sandberg <andreas@sandberg.pp.se>
[sascha.bischoff@arm.com: Rebased patches onto a newer gem5 version]
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Draining is currently done by traversing the SimObject graph and
calling drain()/drainResume() on the SimObjects. This is not ideal
when non-SimObjects (e.g., ports) need draining since this means that
SimObjects owning those objects need to be aware of this.
This changeset moves the responsibility for finding objects that need
draining from SimObjects and the Python-side of the simulator to the
DrainManager. The DrainManager now maintains a set of all objects that
need draining. To reduce the overhead in classes owning non-SimObjects
that need draining, objects inheriting from Drainable now
automatically register with the DrainManager. If such an object is
destroyed, it is automatically unregistered. This means that drain()
and drainResume() should never be called directly on a Drainable
object.
While implementing the new functionality, the DrainManager has now
been made thread safe. In practice, this means that it takes a lock
whenever it manipulates the set of Drainable objects since SimObjects
in different threads may create Drainable objects
dynamically. Similarly, the drain counter is now an atomic_uint, which
ensures that it is manipulated correctly when objects signal that they
are done draining.
A nice side effect of these changes is that it makes the drain state
changes stricter, which the simulation scripts can exploit to avoid
redundant drains.
When the Python helper code switches CPU models, it sometimes also
needs to change the memory mode of the simulator. When this happens,
it accidentally tried to drain the simulator despite having done so
already. This changeset removes the redundant drain.
The probe patch is motivated by the desire to move analytical and trace code
away from functional code. This is achieved by the probe interface which is
essentially a glorified observer model.
What this means to users:
* add a probe point and a "notify" call at the source of an "event"
* add an isolated module, that is being used to carry out *your* analysis (e.g. generate a trace)
* register that module as a probe listener
Note: an example is given for reference in src/cpu/o3/simple_trace.[hh|cc] and src/cpu/SimpleTrace.py
What is happening under the hood:
* every SimObject maintains has a ProbeManager.
* during initialization (src/python/m5/simulate.py) first regProbePoints and
the regProbeListeners is called on each SimObject. this hooks up the probe
point notify calls with the listeners.
FAQs:
Why did you develop probe points:
* to remove trace, stats gathering, analytical code out of the functional code.
* the belief that probes could be generically useful.
What is a probe point:
* a probe point is used to notify upon a given event (e.g. cpu commits an instruction)
What is a probe listener:
* a class that handles whatever the user wishes to do when they are notified
about an event.
What can be passed on notify:
* probe points are templates, and so the user can generate probes that pass any
type of argument (by const reference) to a listener.
What relationships can be generated (1:1, 1:N, N:M etc):
* there isn't a restriction. You can hook probe points and listeners up in a
1:1, 1:N, N:M relationship. They become useful when a number of modules
listen to the same probe points. The idea being that you can add a small
number of probes into the source code and develop a larger number of useful
analysis modules that use information passed by the probes.
Can you give examples:
* adding a probe point to the cpu's commit method allows you to build a trace
module (outputting assembler), you could re-use this to gather instruction
distribution (arithmetic, load/store, conditional, control flow) stats.
Why is the probe interface currently restricted to passing a const reference:
* the desire, initially at least, is to allow an interface to observe
functionality, but not to change functionality.
* of course this can be subverted by const-casting.
What is the performance impact of adding probes:
* when nothing is actively listening to the probes they should have a
relatively minor impact. Profiling has suggested even with a large number of
probes (60) the impact of them (when not active) is very minimal (<1%).
Currently statistics are reset after the initial / checkpoint state
has been loaded. But ruby does some checkpoint processing in its
startup() function. So the stats need to be reset after the startup()
function has been called. This patch moves the class to stats.reset()
to achieve this change in functionality.
2013-12-03 10:51:40 -06:00
Steve Reinhardt ext:(%2C%20Nilay%20Vaish%20%3Cnilay%40cs.wisc.edu%3E%2C%20Ali%20Saidi%20%3CAli.Saidi%40ARM.com%3E)
This patch adds support for simulating with multiple threads, each of
which operates on an event queue. Each sim object specifies which eventq
is would like to be on. A custom barrier implementation is being added
using which eventqs synchronize.
The patch was tested in two different configurations:
1. ruby_network_test.py: in this simulation L1 cache controllers receive
requests from the cpu. The requests are replied to immediately without
any communication taking place with any other level.
2. twosys-tsunami-simple-atomic: this configuration simulates a client-server
system which are connected by an ethernet link.
We still lack the ability to communicate using message buffers or ports. But
other things like simulation start and end, synchronizing after every quantum
are working.
Committed by: Nilay Vaish
The output from the switcheroo tests is voluminous and
(because it includes timestamps) highly sensitive to
minor changes, leading to extremely large updates to the
reference outputs. This patch addresses this problem
by suppressing output from the tests. An internal
parameter can be set to enable the output. Wiring that
up to a command-line flag (perhaps even the rudimantary
-v/-q options in m5/main.py) is left for future work.
Virtualized CPUs and the fastmem mode of the atomic CPU require direct
access to physical memory. We currently require caches to be disabled
when using them to prevent chaos. This is not ideal when switching
between hardware virutalized CPUs and other CPU models as it would
require a configuration change on each switch. This changeset
introduces a new version of the atomic memory mode,
'atomic_noncaching', where memory accesses are inserted into the
memory system as atomic accesses, but bypass caches.
To make memory mode tests cleaner, the following methods are added to
the System class:
* isAtomicMode() -- True if the memory mode is 'atomic' or 'direct'.
* isTimingMode() -- True if the memory mode is 'timing'.
* bypassCaches() -- True if caches should be bypassed.
The old getMemoryMode() and setMemoryMode() methods should never be
used from the C++ world anymore.
CPU switching consists of the following steps:
1. Drain the system
2. Switch out old CPUs (cpu.switchOut())
3. Change the system timing mode to the mode the new CPUs require
4. Flush caches if switching to hardware virtualization
5. Inform new CPUs of the handover (cpu.takeOverFrom())
6. Resume the system
m5.switchCpus() previously only did step 2 & 5. Since information
about the new processors' memory system requirements is now exposed,
do all of the steps above.
This patch adds automatic memory system switching and flush (if
needed) to switchCpus(). Additionally, it adds optional draining to
switchCpus(). This has the following implications:
* changeToTiming and changeToAtomic are no longer needed, so they have
been removed.
* changeMemoryMode is only used internally, so it is has been renamed
to be private.
* switchCpus requires a reference to the system containing the CPUs as
its first parameter.
WARNING: This changeset breaks compatibility with existing
configuration scripts since it changes the signature of
m5.switchCpus().
This patch introduces the following sanity checks when switching
between CPUs:
* Check that the set of new and old CPUs do not overlap. Having an
overlap between the set of new CPUs and the set of old CPUs is
currently not supported. Doing such a switch used to result in the
following assertion error:
BaseCPU::takeOverFrom(BaseCPU*): \
Assertion `!new_itb_port->isConnected()' failed.
* Check that all new CPUs are in the switched out state.
* Check that all old CPUs are in the switched in state.
This patch adds the following two methods to the Drainable base class:
memWriteback() - Write back all dirty cache lines to memory using
functional accesses.
memInvalidate() - Invalidate memory system buffers. Dirty data
won't be written back.
Specifying calling memWriteback() after draining will allow us to
checkpoint systems with caches. memInvalidate() can be used to drop
memory system buffers in preparation for switching to an accelerated
CPU model that bypasses the gem5 memory system (e.g., hardware
virtualized CPUs).
Note: This patch only adds the methods to Drainable, the code for
flushing the TLB and the cache is committed separately.
There is no point in exporting the old drain() method in
Simulate.py. It should only be used internally by doDrain(). This
patch moves the old drain() method into doDrain() and renames
doDrain() to drain().
changeToAtomic and changeToTiming both do essentially the same thing,
they check the type of their input argument, drain the system, and
switch to the desired memory mode. This patch moves all of that code
to a separate method (changeMemoryMode) and calls that from both
changeToAtomic and changeToTiming.
This patch moves the draining interface from SimObject to a separate
class that can be used by any object needing draining. However,
objects not visible to the Python code (i.e., objects not deriving
from SimObject) still depend on their parents informing them when to
drain. This patch also gets rid of the CountedDrainEvent (which isn't
really an event) and replaces it with a DrainManager.
Changeset 4f54b0f229b5 removed the call to doDrain in changeToTiming
based on the assumption that the system does not need draining when
running in atomic mode. This is a false assumption since at least the
System class requires the system to be drained before it allows
switching of memory modes. This patch reverts that part of the
changeset.
This patch adds a function, periodicStatDump(long long period), which will dump
and reset the statistics every period. This function is designed to be called
from the python configuration scripts. This allows the periodic stats dumping to
be configured more easilly at run time.
The period is currently specified as a long long as there are issues passing
Tick into the C++ from the python as they have conflicting definitions. If the
period is less than curTick, the first occurance occurs at curTick. If the
period is set to 0, then the event is descheduled and the stats are not
periodically dumped.
Due to issues when resumung from a checkpoint, the StatDump event must be moved
forward such that it occues AFTER the current tick. As the function is called
from the python, the event is scheduled before the system resumes from the
checkpoint. Therefore, the event is moved using the updateEvents() function.
This is called from simulate.py once the system has resumed from the checkpoint.
NOTE: It should be noted that this is a fairly temporary patch which re-adds the
capability to extract temporal information from the communication monitors. It
should not be used at the same time as anything that relies on dumping the
statistics based on in simulation events i.e. a context switch.
Remove SimObject::setMemoryMode from the main SimObject class since it
is only valid for the System class. In addition to removing the method
from the C++ sources, this patch also removes getMemoryMode and
changeTiming from SimObject.py and updates the simulation code to call
the (get|set)MemoryMode method on the System object instead.
When switching from an atomic CPU to any of the timing CPUs, a drain is
unnecessary since no events are scheduled in atomic mode. However, when
trying to switch CPUs starting with a timing CPU, there may be events
scheduled. This change ensures that all events are drained from the system
by calling m5.drain before switching CPUs.
Simulation objects normally register derived statistics, presumably
what regFormulas originally was meant for, in regStats(). This patch
removes regRegformulas since there is no need to have a separate
method call to register formulas.
Revised system visualization to reflect structure and memory hierarchy.
Improved visualization: less congested and cluttered; more colorful.
Nodes reflect components; directed edges reflect dirctional relation, from
a master port to a slave port. Requires pydot.
