When a gem5 op is triggered using a KVM MMIO exit event, the PC has
already been advanced beyond the offending instruction. Normally when
a system call or gem5 op is triggered, the PC has not advanced because
the instruction hasn't actually finished executing. This means that if
a gem5 op, and by extension a system call in SE mode, want to advance
the PC to the instruction after the gem5 op, they have to check whether
they were triggered from KVM.
To avoid having to special case these sorts of situations (currently
only in the clone system call), we can have the code which dispatches to
gem5 ops from KVM adjust the next PC so that it points to what the
current PC is. That way the PC can be advanced unconditionally, and will
point to the instruction after the one that triggered the call.
To be fully consistent, we would also need to adjust the current PC.
That would be non-trivial since we'd have to figure out where the
current instruction started, and that may not even be possible to
unambiguously determine given x86's instruction structure. Then we would
also need to restore the original PC to avoid confusing KVM.
Change-Id: I9ef90b2df8e27334dedc25c59eb45757f7220eea
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/38486
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The create() method on Params structs usually instantiate SimObjects
using a constructor which takes the Params struct as a parameter
somehow. There has been a lot of needless variation in how that was
done, making it annoying to pass Params down to base classes. Some of
the different forms were:
const Params &
Params &
Params *
const Params *
Params const*
This change goes through and fixes up every constructor and every
create() method to use the const Params & form. We use a reference
because the Params struct should never be null. We use const because
neither the create method nor the consuming object should modify the
record of the parameters as they came in from the config. That would
make consuming them not idempotent, and make it impossible to tell what
the actual simulation configuration was since it would change from any
user visible form (config script, config.ini, dot pdf output).
Change-Id: I77453cba52fdcfd5f4eec92dfb0bddb5a9945f31
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/35938
Reviewed-by: Gabe Black <gabeblack@google.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The memory-mapped timer emulated by gem5 is driven by the underlying
gem5 tick, which means that we must align the tick with the host time
to make the timer interrupt fire at a nearly native rate.
In each KVM execution round, the number of ticks incremented is
directly calculated from the number of instructions executed. However,
when a guest CPU switches to idle state, KVM seems to stay in kernel-
space until the POSIX timer set up in user-space raises an expiration
signal, instead of trapping to user-space immediately; and somehow the
instruction count is just too low to match the elapsed host time. This
makes the gem5 tick increment very slowly when the guest is idle and
drastically slow down workloads being sensitive to the guest time which
is driven by timer interrupt.
Before switching to KVM to execute the guest code, gem5 programs the
POSIX timer to expire according to the remaining ticks before the next
event in the event queue. Based on this, we can come up with the
following solution: If KVM returns to user-space due to POSIX timer
expiration, it must be time to process the next gem5 event, so we just
fast-forward the tick (by scheduling the next CPU tick event) to that
event directly without calculating from the instruction count.
There is one more related issue needed to be solved. The KVM exit
reason, KVM_EXIT_INTR, was treated as the case where the KVM execution
was disturbed by POSIX timer expiration. However, there exists a case
where the exit reason is KVM_EXIT_INTR but the POSIX timer has not
expired. Its cause is still unknown, but it can be observed via the
"old_value" argument returned by timer_settime() when disarming the
POSIX timer. In addition, it seems to happen often when a guest CPU is
not in idle state. When this happens, the above tick event scheduling
incorrectly treats KVM_EXIT_INTR as POSIX timer expiration and fast-
forwards the tick to process the next event too early. This makes the
guest feel external events come too fast, and will sometimes cause
trouble. One example is the VSYNC interrupt from HDLCD. The guest seems
to get stuck in VSYNC handling if the KVM CPU is not given enough time
between each VSYNC interrupt to complete a service. (Honestly I did not
dig in to see how the guest handled the VSYNC interrupt and how the
above situation became trouble. I just observed from the debug trace of
GIC & HDLCD & timer, and made this conclusion.) This change also uses
a workaround to detect POSIX timer expiration correctly to make the
guest work with HDLCD.
JIRA: https://gem5.atlassian.net/browse/GEM5-663
Change-Id: I6159238a36fc18c0c881d177a742d8a7745a23ca
Signed-off-by: Hsuan Hsu <hsuan.hsu@mediatek.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/30919
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The new local access mechanism installs a callback in the request which
implements what the mmapped IPR was doing. That avoids having to have
stubs in ISAs that don't have mmapped IPRs, avoids having to encode
what to do to communicate from the TLB and the mmapped IPR functions,
and gets rid of another global ISA interface function and header files.
Jira Issue: https://gem5.atlassian.net/browse/GEM5-187
Change-Id: I772c2ae2ca3830a4486919ce9804560c0f2d596a
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/23188
Reviewed-by: Matthew Poremba <matthew.poremba@amd.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
TheISA::initCPU is basically an ISA specific implementation of reset
logic on architectural state. As such, it only needs to be called if
we're not going to load a checkpoint, ie in initState.
Also, since the implementation was the same across all CPUs, this
change collapses all the individual implementations down into the base
CPU class.
Change-Id: Id68133fd7f31619c90bf7b3aad35ae20871acaa4
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/24189
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
This was useful when transitioning away from the CPU based
comInstEventQueue, but now that objects backing the ThreadContexts have
access to the underlying comInstEventQueue and can manipulate it
directly, they don't need to do so through a generic interface.
Getting rid of this function narrows and simplifies the interface.
Change-Id: I202d466d266551675ef6792d38c658d8a8f1cb8b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/22113
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This queue was set up to allow triggering events based on the total
number of instructions executed at the system level, and was added in
a change which added a number of things to support McPAT. No code
checked into gem5 actually schedules an event on that queue, and no
code in McPAT (which seems to have gone dormant) either downloadable
from github or found in ext modify gem5 in a way that makes it use
the instEventQueue.
Also, the KVM CPU does not interact with the instEventQueue correctly.
While it does check the per-thread instruction event queue when
deciding how long to run, it does not check the instEventQueue. It will
poke it to run events when it stops for other reasons, but it may (and
likely will) have run beyond the point where it was supposed to stop.
Since this queue doesn't seem to actually be used for anything, isn't
being used properly in all cases anyway, and adds overhead to all the
CPU models, this change eliminates it.
Change-Id: I0e126df14788c37a6d58ca9e1bb2686b70e60d88
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/21783
Maintainer: Gabe Black <gabeblack@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Tiago Mück <tiago.muck@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This patch is changing the underlying type for RequestPtr from Request*
to shared_ptr<Request>. Having memory requests being managed by smart
pointers will simplify the code; it will also prevent memory leakage and
dangling pointers.
Change-Id: I7749af38a11ac8eb4d53d8df1252951e0890fde3
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/10996
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
GCC 7.2 is much stricter than previous GCC versions. The following changes
are needed:
* There is now a warning if there is an implicit fallthrough between two
case statments. C++17 adds the [[fallthrough]]; declaration. However,
to support non C++17 standards (i.e., C++11), we use M5_FALLTHROUGH.
M5_FALLTHROUGH checks for [[fallthrough]] compliant C++17 compiler and
if that doesn't exist, it defaults to nothing (no older compilers
generate warnings).
* The above resulted in a couple of bugs that were found. This is noted
in the review request on gerrit.
* throw() for dynamic exception specification is deprecated
* There were a couple of new uninitialized variable warnings
* Can no longer perform bitwise operations on a bool.
* Must now include <functional> for std::function
* Compiler bug for void* lambda. Changed to auto as work around. See
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82878
Change-Id: I5d4c782a4e133fa4cdb119e35d9aff68c6e2958e
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/5802
Reviewed-by: Gabe Black <gabeblack@google.com>
Move the code responsible for performing the actual probe point notify
into BaseCPU. Use BaseCPU activateContext and suspendContext to keep
track of sleep cycles. Create a probe point (ppActiveCycles) that does
not count cycles where the processor was asleep. Rename ppCycles
to ppAllCycles to reflect its nature.
Change-Id: I1907ddd07d0ff9f2ef22cc9f61f5f46c630c9d66
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/5762
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
The VM's event queue is normally used for devices in multi-core KVM
mode. Add a helper method, BaseKvmCPU::deviceEventQueue(), to access
this queue. This makes the intention of code migrating to device event
queues clearer.
Change-Id: Ifb10f553a6d7445c8d562f658cf9d0b1f4c577ff
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/4287
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
The KVM CPU sometimes needs to access devices when drain() is
called. This typically happens on ARM when synchronizing devices that
use the system register interface. When called from drain(), the event
queue isn't locked since drain is called from the outside when the
simulator isn't servicing any events. In such cases, performing a
migration to the device's queue will unlock a mutex that isn't
locked. This typically results in a deadlock when resuming the system
since the lock will be in an undefined state.
Change-Id: Ibdcc2e034e916a929124f297e72aae306cf66728
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/4286
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
A KVM VM is typically a child of the System object already, but for
solving future issues with configuration graph resolution, the most
logical way to keep track of this object is for it to be an actual
parameter of the System object.
Change-Id: I965ded22203ff8667db9ca02de0042ff1c772220
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
This patch enables timing accesses for KVM cpu. A new state,
RunningMMIOPending, is added to indicate that there are outstanding timing
requests generated by KVM in the system. KVM's tick() is disabled and the
simulation does not enter into KVM until all outstanding timing requests have
completed. The main motivation for this is to allow KVM CPU to perform MMIO
in Ruby, since Ruby does not support atomic accesses.
In general, the ThreadID parameter is unnecessary in the memory system
as the ContextID is what is used for the purposes of locks/wakeups.
Since we allocate sequential ContextIDs for each thread on MT-enabled
CPUs, ThreadID is unnecessary as the CPUs can identify the requesting
thread through sideband info (SenderState / LSQ entries) or ContextID
offset from the base ContextID for a cpu.
This is a re-spin of 20264eb after the revert (bd1c6789) and includes
some fixes of that commit.
The following patches had unexpected interactions with the current
upstream code and have been reverted for now:
e07fd01651f3: power: Add support for power models
831c7f2f9e39: power: Low-power idle power state for idle CPUs
4f749e00b667: power: Add power states to ClockedObject
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
--HG--
extra : amend_source : 0b6fb073c6bbc24be533ec431eb51fbf1b269508
In general, the ThreadID parameter is unnecessary in the memory system
as the ContextID is what is used for the purposes of locks/wakeups.
Since we allocate sequential ContextIDs for each thread on MT-enabled
CPUs, ThreadID is unnecessary as the CPUs can identify the requesting
thread through sideband info (SenderState / LSQ entries) or ContextID
offset from the base ContextID for a cpu.
This changeset adds an option to force the kvm-based CPUs to always
synchronize the gem5 thread context representation on entry/exit into
the kernel. This is very useful for debugging. Unfortunately, it is
also the only way to get reliable register contents when using remote
gdb functionality. The long-term solution for the latter would be to
implement a kvm-specific thread context.
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Alexandru Dutu <alexandru.dutu@amd.com>
We can't/shouldn't use KVM after a fork since the child and parent
probably point to the same VM. Knowing the exact effects of this is
hard, but they are likely to be messy. We also disconnect the
performance counters attached to the guest. This works around what
seems to be a kernel bug where spurious SIGIOs get delivered to the
forked child process.
Signed-off-by: Andreas Sandberg <andreas@sandberg.pp.se>
[sascha.bischoff@arm.com: Rebased patches onto a newer gem5 version]
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
[andreas.sandberg@arm.com: Fatal if entering KVM in child process ]
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
The drain() call currently passes around a DrainManager pointer, which
is now completely pointless since there is only ever one global
DrainManager in the system. It also contains vestiges from the time
when SimObjects had to keep track of their child objects that needed
draining.
This changeset moves all of the DrainState handling to the Drainable
base class and changes the drain() and drainResume() calls to reflect
this. Particularly, the drain() call has been updated to take no
parameters (the DrainManager argument isn't needed) and return a
DrainState instead of an unsigned integer (there is no point returning
anything other than 0 or 1 any more). Drainable objects should return
either DrainState::Draining (equivalent to returning 1 in the old
system) if they need more time to drain or DrainState::Drained
(equivalent to returning 0 in the old system) if they are already in a
consistent state. Returning DrainState::Running is considered an
error.
Drain done signalling is now done through the signalDrainDone() method
in the Drainable class instead of using the DrainManager directly. The
new call checks if the state of the object is DrainState::Draining
before notifying the drain manager. This means that it is safe to call
signalDrainDone() without first checking if the simulator has
requested draining. The intention here is to reduce the code needed to
implement draining in simple objects.
Objects that are can be serialized are supposed to inherit from the
Serializable class. This class is meant to provide a unified API for
such objects. However, so far it has mainly been used by SimObjects
due to some fundamental design limitations. This changeset redesigns
to the serialization interface to make it more generic and hide the
underlying checkpoint storage. Specifically:
* Add a set of APIs to serialize into a subsection of the current
object. Previously, objects that needed this functionality would
use ad-hoc solutions using nameOut() and section name
generation. In the new world, an object that implements the
interface has the methods serializeSection() and
unserializeSection() that serialize into a named /subsection/ of
the current object. Calling serialize() serializes an object into
the current section.
* Move the name() method from Serializable to SimObject as it is no
longer needed for serialization. The fully qualified section name
is generated by the main serialization code on the fly as objects
serialize sub-objects.
* Add a scoped ScopedCheckpointSection helper class. Some objects
need to serialize data structures, that are not deriving from
Serializable, into subsections. Previously, this was done using
nameOut() and manual section name generation. To simplify this,
this changeset introduces a ScopedCheckpointSection() helper
class. When this class is instantiated, it adds a new /subsection/
and subsequent serialization calls during the lifetime of this
helper class happen inside this section (or a subsection in case
of nested sections).
* The serialize() call is now const which prevents accidental state
manipulation during serialization. Objects that rely on modifying
state can use the serializeOld() call instead. The default
implementation simply calls serialize(). Note: The old-style calls
need to be explicitly called using the
serializeOld()/serializeSectionOld() style APIs. These are used by
default when serializing SimObjects.
* Both the input and output checkpoints now use their own named
types. This hides underlying checkpoint implementation from
objects that need checkpointing and makes it easier to change the
underlying checkpoint storage code.
There are cases (particularly when attaching GDB) when instruction
events are scheduled at the current instruction tick. This used to
trigger an assertion error in kvm. This changeset adds a check for
this condition and forces KVM to do a quick entry that completes any
pending IO operations, but does not execute any new instructions,
before servicing the event. We could check if we need to enter KVM at
all, but forcing a quick entry is makes the code slightly cleaner and
does not hurt correctness (performance is hardly an issue in these
cases).
The register dumping code in kvm tries to print the bytes in large
registers (128 bits and larger) instead of printing them as hex. This
changeset fixes that.
This patch tidies up how we create and set the fields of a Request. In
essence it tries to use the constructor where possible (as opposed to
setPhys and setVirt), thus avoiding spreading the information across a
number of locations. In fact, setPhys is made private as part of this
patch, and a number of places where we callede setVirt instead uses
the appropriate constructor.
This patch adds methods in KvmCPU model to handle KVM exits caused by syscall
instructions and page faults. These types of exits will be encountered if
KvmCPU is run in SE mode.
activate(), suspend(), and halt() used on thread contexts had an optional
delay parameter. However this parameter was often ignored. Also, when used,
the delay was seemily arbitrarily set to 0 or 1 cycle (no other delays were
ever specified). This patch removes the delay parameter and 'Events'
associated with them across all ISAs and cores. Unused activate logic
is also removed.
Simulating a SMP or multicore requires devices to be shared between
multiple KVM vCPUs. This means that locking is required when accessing
devices. This changeset adds the necessary locking to allow devices to
execute correctly. It is implemented by temporarily migrating the KVM
CPU to the VM's (and devices) event queue when handling
MMIO. Similarly, the VM migrates to the interrupt controller's event
queue when delivering an interrupt.
The support for fast-forwarding of multicore simulations added by this
changeset assumes that all devices in a system are simulated in the
same thread and each vCPU has its own thread. Special care must be
taken to ensure that devices living under the CPU in the object
hierarchy (e.g., the interrupt controller) do not inherit the parent
CPUs thread and are assigned to device thread. The KvmVM object is
assumed to live in the same thread as the other devices in the system.
KVM used to use two signals, one for instruction count exits and one
for timer exits. There is really no need to distinguish between the
two since they only trigger exits from KVM. This changeset unifies and
renames the signals and adds a method, kick(), that can be used to
raise the control signal in the vCPU thread. It also removes the early
timer warning since we do not normally see if the signal was
delivered.
--HG--
extra : rebase_source : cd0e45ca90894c3d6f6aa115b9b06a1d8f0fda4d
This changeset adds support for INIT and STARTUP IPI handling. We
currently handle both of these interrupts in gem5 and transfer the
state to KVM. Since we do not have a BIOS loaded, we pretend that the
INIT interrupt suspends the CPU after reset.
--HG--
extra : rebase_source : 7f3b25f3801d68f668b6cd91eaf50d6f48ee2a6a
Signal handlers in KVM are controlled per thread and should be
initialized from the thread that is going to execute the CPU. This
changeset moves the initialization call from startup() to
startupThread().
The introduction of parallel event queues added most of the support
needed to run multiple VMs (systems) within the same gem5
instance. This changeset fixes up signal delivery so that KVM's
control signals are delivered to the thread that executes the CPU's
event queue. Specifically:
* Timers and counters are now initialized from a separate method
(startupThread) that is scheduled as the first event in the
thread-specific event queue. This ensures that they are
initialized from the thread that is going to execute the CPUs
event queue and enables signal delivery to the right thread when
exiting from KVM.
* The POSIX-timer-based KVM timer (used to force exits from KVM) has
been updated to deliver signals to the thread that's executing KVM
instead of the process (thread is undefined in that case). This
assumes that the timer is instantiated from the thread that is
going to execute the KVM vCPU.
* Signal masking is now done using pthread_sigmask instead of
sigprocmask. The behavior of the latter is undefined in threaded
applications.
* Since signal masks can be inherited, make sure to actively unmask
the control signals when setting up the KVM signal mask.
There are currently no facilities to multiplex between multiple KVM
CPUs in the same event queue, we are therefore limited to
configurations where there is only one KVM CPU per event queue. In
practice, this means that multi-system configurations can be
simulated, but not multiple CPUs in a shared-memory configuration.
