stdlib: Refactor gem5 Vision/gem5-resources code

This patch includes several changes to the gem5 tools interface to the
gem5-resources infrastructure. These are:

* The old download and JSON query functions have been removed from the
  downloader module. These functions were used for directly downloading
  and inspecting the resource JSON file, hosted at
  https://resources.gem5.org/resources. This information is now obtained
  via `gem5.client`. If a resources JSON file is specified as a client,
  it should conform to the new schema:
  https//resources.gem5.org/gem5-resources-schema.json. The old schema
  (pre-v23.0) is no longer valid. Tests have been updated to reflect
  this change. Those which tested these old functions have been removed.
* Unused imports have been removed.
* For the resource query functions, and those tasked with obtaining the
  resources, the parameter `gem5_version` has been added. In all cases
  it does the same thing:
    * It will filter results based on compatibility to the
      `gem5_version` specified. If no resources are compatible the
      latest version of that resource is chosen (though a warning is
      thrown).
    * By default it is set to the current gem5 version.
    * It is optional. If `None`, this filtering functionality is not
      carried out.
    * Tests have been updated to fix the version to “develop” so the
      they do not break between versions.
* The `gem5_version` parameters will filter using a logic which will
  base compatibility on the specificity of the gem5-version specified in
  a resource’s data. If a resource has a compatible gem5-version of
  “v18.4” it will be compatible with any minor/hotfix version within the
  v18.4 release (this can be seen as matching on “v18.4.*.*”.) Likewise,
  if a resource has a compatible gem5-version of “v18.4.1” then it’s
  only compatible with the v18.4.1 release but any of it’s hot fix
  releases (“v18.4.1.*”).
* The ‘list_resources’ function has been updated to use the
  “gem5.client” APIs to get resource information from the clients
  (MongoDB or a JSON file). This has been designed to remain backwards
  compatible to as much as is possible, though, due to schema changes,
  the function does search across all versions of gem5.
* `get_resources` function was added to the `AbstractClient`. This is a
   more general function than `get_resource_by_id`. It was
  primarily created to handle the `list_resources` update but is a
  useful update to the API. The `get_resource_by_id` function has been
  altered to function as a wrapped to the `get_resources` function.
* Removed “GEM5_RESOURCE_JSON” code has been removed. This is no longer
  used.
* Tests have been cleaned up a little bit to be easier to read.
* Some docstrings have been updated.

Things that are left TODO with this code:

* The client_wrapper/client/abstract_client abstractions are rather
  pointless. In particular the client_wrapper and client classes could
  be merged.
* The downloader module no longer does much and should have its
  functions merged into other modules.
* With the addition of the `get_resources` function, much of the code in
  the `AbstractClient` could be simplified.

Change-Id: I0ce48e88b93a2b9db53d4749861fa0b5f9472053
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71506
Reviewed-by: Kunal Pai <kunpai@ucdavis.edu>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
(cherry picked from commit 82587ce71b)
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71739
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
This commit is contained in:
Bobby R. Bruce
2023-06-12 14:09:15 -07:00
committed by Bobby Bruce
parent b22afde3be
commit aff1e7b64c
21 changed files with 445 additions and 603 deletions

View File

@@ -10,6 +10,7 @@
"source": "src/linux-kernel",
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -25,6 +26,7 @@
"root_partition": "1",
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -39,6 +41,7 @@
"source": "src/simple",
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -51,6 +54,7 @@
"url": "http://dist.gem5.org/dist/develop/test-progs/hello/bin/arm/linux/hello64-static",
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -66,6 +70,7 @@
"url": "http://dist.gem5.org/dist/develop/checkpoints/riscv-hello-example-checkpoint.tar",
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -75,10 +80,11 @@
"description": null,
"is_zipped": false,
"is_tar_archive": true,
"md5sum": "71b2cb004fe2cda4556f0b1a38638af6",
"md5sum": "3a57c1bb1077176c4587b8a3bf4f8ace",
"url": "http://dist.gem5.org/dist/develop/checkpoints/riscv-hello-example-checkpoint.tar",
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -87,11 +93,12 @@
"id": "file-example",
"description": null,
"is_zipped": false,
"md5sum": "71b2cb004fe2cda4556f0b1a38638af6",
"md5sum": "2efd144c11829ab18d54eae6371e120a",
"url": "http://dist.gem5.org/dist/develop/checkpoints/riscv-hello-example-checkpoint.tar",
"source": null,
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -106,6 +113,7 @@
"url": "http://dist.gem5.org/dist/develop/checkpoints/riscv-hello-example-checkpoint.tar",
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -125,6 +133,7 @@
"workload_name": "Example Workload",
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -148,6 +157,7 @@
],
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -161,6 +171,7 @@
"source": null,
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
},
@@ -170,11 +181,12 @@
"description": "A looppoint json file resource.",
"is_zipped": false,
"region_id": "1",
"md5sum": "a71ed64908b082ea619b26b940a643c1",
"md5sum": "efb85ebdf90c5cee655bf2e05ae7692a",
"url": "http://dist.gem5.org/dist/develop/looppoints/x86-matrix-multiply-omp-100-8-looppoint-json-20230128",
"source": null,
"resource_version": "1.0.0",
"gem5_versions": [
"develop",
"23.0"
]
}