Many functions that used to return lists (e.g., dict.items()) now
return iterators and their iterator counterparts (e.g.,
dict.iteritems()) have been removed. Switch calls to the Python 2.7
iterator methods to use the Python 3 equivalent and add explicit list
conversions where necessary.
Change-Id: I0c18114955af8f4932d81fb689a0adb939dafaba
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/15992
Reviewed-by: Juha Jäykkä <juha.jaykka@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Add a utility method, SimObject.apply_config that can be used to
implement SimObject param overrides from the command line. This
function provides safe and convenient semantics for CLI assignment:
* The override expression is evaluated in a restricted environment. The
only global variables are the child objects and params from the root
object.
* Only params can be overridden. For example, calling methods or setting
attributes on SimObjects isn't possible.
* Vectors use non-standard list semantics which enable something similar
to glob expansion on the shell. For example, setting:
root.system.cpu[0:2].numThreads = 2
will override numThreads for cpu 0 and 1 and:
root.system.cpus[0,2].numThreads = 2
sets it for cpus 0 and 2.
The intention is that the helper method is called to override default
values before calling m5.instantiate.
Change-Id: I73f99da21d6d8ce1ff2ec8db2bb34338456f6799
Reviewed-on: https://gem5-review.googlesource.com/c/12984
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
The base for the c++ version of python SimObject classes is normally
inferred from the c++ version of the python base. There are some
specific cases where that isn't desired. This change makes it possible
to override the default behavior.
Change-Id: I2438dad767e2f56823bad42b3e6c7714ce97ef79
Reviewed-on: https://gem5-review.googlesource.com/10662
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
cxx_bases adds in additional c++ base classes beyond those implied by
the python SimObject inheritance hierarchy. To imply the fact that
these are additional bases, and to disambiguate a future mechanism
which changes the implied bases, this flag/field is being renamed from
cxx_bases to cxx_extra_bases.
As far as I can tell, this field was only used internally in
SimObject.py.
Change-Id: Ie7cc3d0107ff71cc31424d6e20c9a2f430022ab9
Reviewed-on: https://gem5-review.googlesource.com/10661
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Starting with version 3, scons imposes using the print function instead
of the print statement in code it processes. To get things building
again, this change moves all python code within gem5 to use the
function version. Another change by another author separately made this
same change to the site_tools and site_init.py files.
Change-Id: I2de7dc3b1be756baad6f60574c47c8b7e80ea3b0
Reviewed-on: https://gem5-review.googlesource.com/8761
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Gabe Black <gabeblack@google.com>
This patch adds an extra layer to the pyfdt library such that usage
gets easier and device tree nodes can be specified in less code,
without limiting original usage. Note to not import both the pyfdt
and fdthelper in the same namespace (but generally fdthelper is all
you need, because it supplies the same classes even when they are not
extended in any way)
Also, this patch lays out the primary functionality for generating a
device tree, where every SimObject gets an empty generateDeviceTree
method and ArmSystems loop over their children in an effort to merge
all the nodes. Devices are implemented in other patches.
Change-Id: I4d0a0666827287fe42e18447f19acab4dc80cc49
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/5962
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
The PyBind wrappers could potentially delete SimObjects if they don't
have any references. This is not desirable since there could be
pointers to such objects within the C++ world. This problem doesn't
normally occur since Python typically holds a pointer to the root node
as long as the simulator is running.
Prevent SimObject and Param deletion by using a PyBind-prescribed
unique_ptr with a dummy deleter as the pointer wrapper for the Python
world.
Change-Id: Ied14602c9ee69a083a69c5dae1b5fcf8efb4548a
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/3224
Reviewed-by: Gabe Black <gabeblack@google.com>
There is a weird issue with the PyBind wrapper of
vector<AddrRange>. Assigning new values to a param that is a vector of
AddrRange sometimes results in an out-of-bounds memory access.
We work around this issue by treating AddrRange vectors as opaque
types. This slightly changes the semantics of the wrapper since Python
now manipulates the real object rather than a copy that has been
converted to a list.
Change-Id: Ie027c06e7a7262214b43b19a76b24fe4b20426c5
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Timothy Hayes <timothy.hayes@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/3223
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
The style checker complains about line length and ordering for these
files. This fix should make these two files kosher.
Change-Id: I822a0518a98d9e379a543d2017e90c4e9666a58d
Reviewed-on: https://gem5-review.googlesource.com/3380
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Brandon Potter <Brandon.Potter@amd.com>
Use the PyBind11 wrapping infrastructure instead of SWIG to generate
wrappers for functionality that needs to be exported to Python. This
has several benefits:
* PyBind11 can be redistributed with gem5, which means that we have
full control of the version used. This avoid a large number of
hard-to-debug SWIG issues we have seen in the past.
* PyBind11 doesn't rely on a custom C++ parser, instead it relies on
wrappers being explicitly declared in C++. The leads to slightly
more boiler-plate code in manually created wrappers, but doesn't
doesn't increase the overall code size. A big benefit is that this
avoids strange compilation errors when SWIG doesn't understand
modern language features.
* Unlike SWIG, there is no risk that the wrapper code incorporates
incorrect type casts (this has happened on numerous occasions in
the past) since these will result in compile-time errors.
As a part of this change, the mechanism to define exported methods has
been redesigned slightly. New methods can be exported either by
declaring them in the SimObject declaration and decorating them with
the cxxMethod decorator or by adding an instance of
PyBindMethod/PyBindProperty to the cxx_exports class variable. The
decorator has the added benefit of making it possible to add a
docstring and naming the method's parameters.
The new wrappers have the following known issues:
* Global events can't be memory managed correctly. This was the
case in SWIG as well.
Change-Id: I88c5a95b6cf6c32fa9e1ad31dfc08b2e8199a763
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Reviewed-by: Andrew Bardsley <andrew.bardsley@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/2231
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>
Reviewed-by: Pierre-Yves Péneau <pierre-yves.peneau@lirmm.fr>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Swig wrappers for native objects currently share the _m5.internal name
space with Python code. This is undesirable if we ever want to switch
from Swig to some other framework for native binding (e.g., PyBind11
or Boost::Python). This changeset moves all of such wrappers to the
_m5 namespace, which is now reserved for native code.
Change-Id: I2d2bc12dbc05b57b7c5a75f072e08124413d77f3
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
The headers declared in export_method_cxx_predecls are redundant since a
SimObject's main header is automatically included.
Change-Id: Ied9e84630b36960e54efe91d16f8c66fba7e0da0
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Joe Gross <joseph.gross@amd.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
The gem5 option '--list-sim-objects' is supposed to list all available
SimObjects and their parameters. It currently chokes on SimObjects
with parameters that have an object instance as their default
value. This is caused by __str__ in SimObject trying to resolve its
complete path. When the path resolution method reaches the parent
object (a MetaSimObject since it hasn't been instantiated), it dies
with a Python exception.
This changeset adds a guard to stop path resolution if the parent
object is a MetaSimObject.
The memWriteback() and memInvalidate() calls used to live in the
Serializable interface. In this series of patches, the Serializable
interface will be redesigned to make serialization independent of the
object graph and always work on the entire simulator. This means that
the Serialization interface won't be useful to perform maintenance of
the caches in a sub-graph of the entire SimObject graph. This
changeset moves these memory maintenance methods to the SimObject
interface instead.
Objects that are can be serialized are supposed to inherit from the
Serializable class. This class is meant to provide a unified API for
such objects. However, so far it has mainly been used by SimObjects
due to some fundamental design limitations. This changeset redesigns
to the serialization interface to make it more generic and hide the
underlying checkpoint storage. Specifically:
* Add a set of APIs to serialize into a subsection of the current
object. Previously, objects that needed this functionality would
use ad-hoc solutions using nameOut() and section name
generation. In the new world, an object that implements the
interface has the methods serializeSection() and
unserializeSection() that serialize into a named /subsection/ of
the current object. Calling serialize() serializes an object into
the current section.
* Move the name() method from Serializable to SimObject as it is no
longer needed for serialization. The fully qualified section name
is generated by the main serialization code on the fly as objects
serialize sub-objects.
* Add a scoped ScopedCheckpointSection helper class. Some objects
need to serialize data structures, that are not deriving from
Serializable, into subsections. Previously, this was done using
nameOut() and manual section name generation. To simplify this,
this changeset introduces a ScopedCheckpointSection() helper
class. When this class is instantiated, it adds a new /subsection/
and subsequent serialization calls during the lifetime of this
helper class happen inside this section (or a subsection in case
of nested sections).
* The serialize() call is now const which prevents accidental state
manipulation during serialization. Objects that rely on modifying
state can use the serializeOld() call instead. The default
implementation simply calls serialize(). Note: The old-style calls
need to be explicitly called using the
serializeOld()/serializeSectionOld() style APIs. These are used by
default when serializing SimObjects.
* Both the input and output checkpoints now use their own named
types. This hides underlying checkpoint implementation from
objects that need checkpointing and makes it easier to change the
underlying checkpoint storage code.
This patch adds sorting based on the SimObject name or parameter name
for all situations where we iterate over dictionaries. This should
ensure a deterministic and consistent order across the host systems
and hopefully avoid regression results differing across python
versions.
This patch adds the ability to load in config.ini files generated from
gem5 into another instance of gem5 built without Python configuration
support. The intended use case is for configuring gem5 when it is a
library embedded in another simulation system.
A parallel config file reader is also provided purely in Python to
demonstrate the approach taken and to provided similar functionality
for as-yet-unknown use models. The Python configuration file reader
can read both .ini and .json files.
C++ configuration file reading:
A command line option has been added for scons to enable C++ configuration
file reading: --with-cxx-config
There is an example in util/cxx_config that shows C++ configuration in action.
util/cxx_config/README explains how to build the example.
Configuration is achieved by the object CxxConfigManager. It handles
reading object descriptions from a CxxConfigFileBase object which
wraps a config file reader. The wrapper class CxxIniFile is provided
which wraps an IniFile for reading .ini files. Reading .json files
from C++ would be possible with a similar wrapper and a JSON parser.
After reading object descriptions, CxxConfigManager creates
SimObjectParam-derived objects from the classes in the (generated with this
patch) directory build/ARCH/cxx_config
CxxConfigManager can then build SimObjects from those SimObjectParams (in an
order dictated by the SimObject-value parameters on other objects) and bind
ports of the produced SimObjects.
A minimal set of instantiate-replacing member functions are provided by
CxxConfigManager and few of the member functions of SimObject (such as drain)
are extended onto CxxConfigManager.
Python configuration file reading (configs/example/read_config.py):
A Python version of the reader is also supplied with a similar interface to
CxxConfigFileBase (In Python: ConfigFile) to config file readers.
The Python config file reading will handle both .ini and .json files.
The object construction strategy is slightly different in Python from the C++
reader as you need to avoid objects prematurely becoming the children of other
objects when setting parameters.
Port binding also needs to be strictly in the same port-index order as the
original instantiation.
This patch 'completes' .json config files generation by adding in the
SimObject references and String-valued parameters not currently
printed.
TickParamValues are also changed to print in the same tick-value
format as in .ini files.
This allows .json files to describe a system as fully as the .ini files
currently do.
This patch adds a new function config_value (which mirrors ini_str) to
each ParamValue and to SimObject. This function can then be explicitly
changed to give different .json and .ini printing behaviour rather than
being written in terms of ini_str.
This patch adds helper functions to SimObject.py, params.py and
simulate.py to enable the new configuration system. Functions like
enumerateParams() in SimObject lets the config system auto-generate
command line options for simobjects to be modified on the command
line.
Params in params.py have __call__() added
to their definition to allow the argparse module to use them
as a type to check command input is in the proper format.
The unproxy code for Parent.any can generate a circular reference
in certain situations with classes hierarchies like those in ClockDomain.py.
This patch solves this by marking ouself as visited to make sure the
search does not resolve to a self-reference.
The probe patch is motivated by the desire to move analytical and trace code
away from functional code. This is achieved by the probe interface which is
essentially a glorified observer model.
What this means to users:
* add a probe point and a "notify" call at the source of an "event"
* add an isolated module, that is being used to carry out *your* analysis (e.g. generate a trace)
* register that module as a probe listener
Note: an example is given for reference in src/cpu/o3/simple_trace.[hh|cc] and src/cpu/SimpleTrace.py
What is happening under the hood:
* every SimObject maintains has a ProbeManager.
* during initialization (src/python/m5/simulate.py) first regProbePoints and
the regProbeListeners is called on each SimObject. this hooks up the probe
point notify calls with the listeners.
FAQs:
Why did you develop probe points:
* to remove trace, stats gathering, analytical code out of the functional code.
* the belief that probes could be generically useful.
What is a probe point:
* a probe point is used to notify upon a given event (e.g. cpu commits an instruction)
What is a probe listener:
* a class that handles whatever the user wishes to do when they are notified
about an event.
What can be passed on notify:
* probe points are templates, and so the user can generate probes that pass any
type of argument (by const reference) to a listener.
What relationships can be generated (1:1, 1:N, N:M etc):
* there isn't a restriction. You can hook probe points and listeners up in a
1:1, 1:N, N:M relationship. They become useful when a number of modules
listen to the same probe points. The idea being that you can add a small
number of probes into the source code and develop a larger number of useful
analysis modules that use information passed by the probes.
Can you give examples:
* adding a probe point to the cpu's commit method allows you to build a trace
module (outputting assembler), you could re-use this to gather instruction
distribution (arithmetic, load/store, conditional, control flow) stats.
Why is the probe interface currently restricted to passing a const reference:
* the desire, initially at least, is to allow an interface to observe
functionality, but not to change functionality.
* of course this can be subverted by const-casting.
What is the performance impact of adding probes:
* when nothing is actively listening to the probes they should have a
relatively minor impact. Profiling has suggested even with a large number of
probes (60) the impact of them (when not active) is very minimal (<1%).
If you successfully export a C++ SimObject method, but try to
invoke it from Python before the C++ object is created, you
get a confusing error that says the attribute does not exist,
making you question whether you successfully exported the
method at all. In reality, your only problem is that you're
calling the method too soon. This patch enhances the error
message to give you a better clue.
Updating the SimObject topology of a cloned hierarchy is a little
dangerous, in that cloning is a "deep copy" and the clone does not
inherit SimObject updates the same way it would inherit scalar
variable assignments.
However, because of various SimObject-valued proxy parameters,
like 'memories', 'clk_domain', and 'system', it turns out that
there are a number of implicit topology changes that happen at
instantiation, which means that these changes are impossible to
avoid. So in order to make cloning systems useful, this error
has to go. Changing it to a warning produces a lot of noise,
so it seems best just to delete it.
2014-01-03 17:08:42 -08:00
Steve Reinhardt ext:(%2C%20Nilay%20Vaish%20%3Cnilay%40cs.wisc.edu%3E%2C%20Ali%20Saidi%20%3CAli.Saidi%40ARM.com%3E)
This patch adds support for simulating with multiple threads, each of
which operates on an event queue. Each sim object specifies which eventq
is would like to be on. A custom barrier implementation is being added
using which eventqs synchronize.
The patch was tested in two different configurations:
1. ruby_network_test.py: in this simulation L1 cache controllers receive
requests from the cpu. The requests are replied to immediately without
any communication taking place with any other level.
2. twosys-tsunami-simple-atomic: this configuration simulates a client-server
system which are connected by an ethernet link.
We still lack the ability to communicate using message buffers or ports. But
other things like simulation start and end, synchronizing after every quantum
are working.
Committed by: Nilay Vaish
SimObjectVector objects did not provide the same interface to
the _parent attribute through get_parent() like a normal
SimObject. It also handled assigning a _parent incorrectly
if objects in a SimObjectVector were changed post-creation,
leading to errors later when the simulator tried to execute.
This patch fixes these two omissions.
SimObjects are expected to only generate one port reference per
port belonging to them. There is a subtle bug with using "not"
here as a VectorPort is seen as not having a reference if it is
either None or empty as per Python docs sec 9.9 for Standard operators.
Intended behavior is to only check if we have not created the reference.
This patch adds two fuctions to m5.util, warn and inform, which mirror those
found in the C++ side of gem5. These are added in addition to the already
existing m5.util.panic and m5.util.fatal which already mirror the C++
functionality. This ensures that warning and information messages generated
by python are in the same format as those generated by C++.
Occurrences of
print "Warning: %s..." % name
have been replaced with
warn("%s...", name)
This patch makes the all proxy traverse any potential list that is
encountered in the object hierarchy instead of only looking at
children that are SimObjects. An example of where this is useful is
when creating a multi-channel memory system as a list of controllers,
whilst ensuring that the memories are still visible in the system.
This changeset adds a SWIG interface for the Serializable class, which
fixes a warning when compiling the SWIG interface for the event
queue. Currently, the only method exported is the name() method.
This patch moves the draining interface from SimObject to a separate
class that can be used by any object needing draining. However,
objects not visible to the Python code (i.e., objects not deriving
from SimObject) still depend on their parents informing them when to
drain. This patch also gets rid of the CountedDrainEvent (which isn't
really an event) and replaces it with a DrainManager.
When casting objects in the generated SWIG interfaces, SWIG uses
classical C-style casts ( (Foo *)bar; ). In some cases, this can
degenerate into the equivalent of a reinterpret_cast (mainly if only a
forward declaration of the type is available). This usually works for
most compilers, but it is known to break if multiple inheritance is
used anywhere in the object hierarchy.
This patch introduces the cxx_header attribute to Python SimObject
definitions, which should be used to specify a header to include in
the SWIG interface. The header should include the declaration of the
wrapped object. We currently don't enforce header the use of the
header attribute, but a warning will be generated for objects that do
not use it.
Remove SimObject::setMemoryMode from the main SimObject class since it
is only valid for the System class. In addition to removing the method
from the C++ sources, this patch also removes getMemoryMode and
changeTiming from SimObject.py and updates the simulation code to call
the (get|set)MemoryMode method on the System object instead.
Simulation objects normally register derived statistics, presumably
what regFormulas originally was meant for, in regStats(). This patch
removes regRegformulas since there is no need to have a separate
method call to register formulas.
Instead of just passing a list of controllers to the makeTopology function
in src/mem/ruby/network/topologies/<Topo>.py we pass in a function pointer
which knows how to make the topology, possibly with some extra state set
in the configs/ruby/<protocol>.py file. Thus, we can move all of the files
from network/topologies to configs/topologies. A new class BaseTopology
is added which all topologies in configs/topologies must inheirit from and
follow its API.
--HG--
rename : src/mem/ruby/network/topologies/Crossbar.py => configs/topologies/Crossbar.py
rename : src/mem/ruby/network/topologies/Mesh.py => configs/topologies/Mesh.py
rename : src/mem/ruby/network/topologies/MeshDirCorners.py => configs/topologies/MeshDirCorners.py
rename : src/mem/ruby/network/topologies/Pt2Pt.py => configs/topologies/Pt2Pt.py
rename : src/mem/ruby/network/topologies/Torus.py => configs/topologies/Torus.py
This patch changes the organisation of the JSON output slightly to
make it easier to traverse and use the files. Most importantly, the
hierarchical dictionaries now use keys that correspond to the
attribute names also in the case of VectorParams (used to be
e.f. "cpu0 cpu1"). It also adds the name and the path to each
SimObject directory entry. Before this patch, to get cpu0, you would
have to query dict['system']['cpu0 cpu1'][0] and this could be a dict
with 'cpu0' : { cpu parameters }. Now you use dict['system']['cpu'][0]
and get { cpu parameters } (where one is "name" : "cpu0").
Additionally this patch includes more verbose information about the
ports, specifying their role, and using a JSON array rather than a
concatenated string for the peer.
Revised system visualization to reflect structure and memory hierarchy.
Improved visualization: less congested and cluttered; more colorful.
Nodes reflect components; directed edges reflect dirctional relation, from
a master port to a slave port. Requires pydot.
Fixed broken code which visualizes the system configuration by generating a
tree from each component's children, starting from root.
Requires DOT (hence pydot).
This patch changes the behaviour of the All proxy parameter to not
only consider the direct children, but also do a pre-order depth-first
traversal of the object tree and append all results from the
children.
This is used in a later patch to find all the memories in the system,
independent of where they are located in the hierarchy.
