mem-ruby,sim-se: Support Maybe_Stale in functional reads

Functional reads can be satisfied by one of the following, in order:
1. Main memory (when the data is not present in the cache hierarchy);
2. Valid data block in cache;
3. Valid data block in coherence message;
4. Valid data block marked as Maybe_Stale;

Number 4 is not handled by the current implementation. A Maybe_Stale
block can be either truly stale or actually valid. When it is stale,
the memory read will be satisfied by either number 2 or number 3. When
it is valid, there will be no coherence message with valid data inside,
and the Maybe_Stale block will transition to a valid state after
receiving some kind of acknowledgement.

The main challenge to handle number 4 is how to know from which
Maybe_Stale block the data should be read from. For instance, in a two
level cache hierarchy, we might have a block marked as Maybe_Stale in
both L1 and L2. In this case, we should prioritize the cache controller
that is closest to the CPU. To define this priority, a new virtual
function 'functionalReadPriority' was added to the AbstractController
class.

Change-Id: I4774cd01aab7bb9ca53694cd9dc4f9416a8e4025

src/mem/ruby/slicc_interface/AbstractController.hh

@@ -134,6 +134,14 @@ class AbstractController : public ClockedObject, public Consumer
     virtual bool functionalReadBuffers(PacketPtr&) = 0;
     virtual void functionalRead(const Addr &addr, PacketPtr)
     { panic("functionalRead(Addr,PacketPtr) not implemented"); }
+    /**
+     * Returns the priority used by functional reads when deciding from which
+     * controller to read a Maybe_Stale data block.
+     * Lower positive values have higher priority, negative values are ignored.
+     *
+     * @return the controller's priority
+     */
+    virtual int functionalReadPriority() { return -1; }
 
     //! Functional read that reads only blocks not present in the mask.
     //! Return number of bytes read.

src/mem/ruby/system/RubySystem.cc

@@ -518,6 +518,7 @@ RubySystem::functionalRead(PacketPtr pkt)
 
     AbstractController *ctrl_ro = nullptr;
     AbstractController *ctrl_rw = nullptr;
+    AbstractController *ctrl_ms = nullptr;
     AbstractController *ctrl_backing_store = nullptr;
 
     // In this loop we count the number of controllers that have the given
@@ -534,9 +535,25 @@ RubySystem::functionalRead(PacketPtr pkt)
         }
         else if (access_perm == AccessPermission_Busy)
             num_busy++;
-        else if (access_perm == AccessPermission_Maybe_Stale)
+        else if (access_perm == AccessPermission_Maybe_Stale) {
+            int priority = cntrl->functionalReadPriority();
+            if (priority >= 0) {
+                if (ctrl_ms == nullptr) {
+                    ctrl_ms = cntrl;
+                } else {
+                    int current_priority = ctrl_ms->functionalReadPriority();
+                    if (ctrl_ms == nullptr || priority < current_priority) {
+                        ctrl_ms = cntrl;
+                    } else if (priority == current_priority) {
+                        warn("More than one Abstract Controller with "
+                             "Maybe_Stale permission and same priority (%d) "
+                             "for addr: %#x on cacheline: %#x.", priority,
+                             address, line_address);
+                    }
+                }
+            }
             num_maybe_stale++;
-        else if (access_perm == AccessPermission_Backing_Store) {
+        } else if (access_perm == AccessPermission_Backing_Store) {
             // See RubySlicc_Exports.sm for details, but Backing_Store is meant
             // to represent blocks in memory *for Broadcast/Snooping protocols*,
             // where memory has no idea whether it has an exclusive copy of data
@@ -606,6 +623,13 @@ RubySystem::functionalRead(PacketPtr pkt)
             if (network->functionalRead(pkt))
                 return true;
         }
+        if (ctrl_ms != nullptr) {
+            // No copy in transit or buffered indicates that a block marked
+            // as Maybe_Stale is actually up-to-date, just waiting an Ack or
+            // similar type of message which carries no data.
+            ctrl_ms->functionalRead(line_address, pkt);
+            return true;
+        }
     }
 
     return false;