stdlib: Cache the resources.json download

"resources.json" is referenced when a resource is requested. The
"resources.json" file may be updated at any time and therefore, the
downloader was engineered to retrieve this file from the gem5-resources
repo on each request. However, this can lead to excessively frequently
pulls over short periods of time which is uncessary given how
infrequently this file changes.

To combat this, this patch caches the "resources.json" file and will
use it for up to an hour after creation before re-retrieving it.

Change-Id: I3b4907cbadce8a54df21d85f8021bf3603ae0f6f
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/57275
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>

src/python/gem5/resources/downloader.py

@@ -59,17 +59,23 @@ def _get_resources_json_uri() -> str:
 
     return uri
 
-def _get_resources_json_at_url(url: str) -> Dict:
+def _get_resources_json_at_url(url: str, use_caching: bool = True) -> Dict:
     '''
     Returns a resource JSON, in the form of a Python Dict. The URL location
     of the JSON must be specified.
 
+    If `use_caching` is True, a copy of the JSON will be cached locally, and
+    used for up to an hour after retrieval.
+
     **Note**: The URL is assumed to be the location within a Google Source
     repository. Special processing is done to handle this. This is the primary
     reason there are separate functions for handling the retrieving of the
     resources JSON comapared to just using the `_download` function directly.
 
     :param url: The URL of the JSON file.
+    :param use_caching: True if a cached file is to be used (up to an hour),
+    otherwise the file will be retrieved from the URL regardless. True by
+    default.
     '''
 
     file_path = os.path.join(
@@ -77,7 +83,19 @@ def _get_resources_json_at_url(url: str) -> Dict:
         f"gem5-resources-{hashlib.md5(url.encode()).hexdigest()}.base64",
     )
 
-    _download(url, file_path)
+    # The resources.json file can change at any time, but to avoid excessive
+    # retrieval we cache a version locally and use it for up to an hour before
+    # obtaining a fresh copy.
+    #
+    # `time.time()` and `os.path.getmtime(..)` both return an unix epoch time
+    # in seconds. Therefore, the value of "3600" here represents an hour
+    # difference between the two values. `time.time()` gets the current time,
+    # and `os.path.getmtime(<file>)` gets the modification time of the file.
+    # This is the most portable solution as other ideas, like "file creation
+    # time", are  not always the same concept between operating systems.
+    if not use_caching or not os.path.exists(file_path) or \
+        (time.time() - os.path.getmtime(file_path)) > 3600:
+                _download(url, file_path)
 
     # Note: Google Source does not properly support obtaining files as raw
     # text. Therefore when we open the URL we receive the JSON in base64