misc: Merge branch 'release-staging-v20.1.0.0' into develop

Change-Id: I3694b251855b969c7bd3807f34e1b4241d47d586
This commit is contained in:
Bobby R. Bruce
2020-09-30 20:39:06 -07:00
33 changed files with 198 additions and 91 deletions

View File

@@ -86,20 +86,20 @@ To clone the gem5 repository:
git clone https://gem5.googlesource.com/public/gem5
```
By default, the master branch is checked out. The master branch is stable,
containing the latest released version of gem5. To obtain code still
under-development (and which contributions can be made):
By default, the stable branch is checked out. The stable branch contains the
latest released version of gem5. To obtain code still under-development (and
which contributions can be made):
```
cd gem5
git checkout --track origin/develop
```
Changes should be made to this develop branch. Changes to the master branch
Changes should be made to this develop branch. Changes to the stable branch
will be blocked. Once a change on the develop branch is properly incorporated
into the gem5 repo it will be merged into the master Branch upon the next
into the gem5 repo it will be merged into the stable Branch upon the next
release of gem5. New releases of gem5 occur three times a year. Ergo, changes
made to the develop branch should appear on the master branch within three to
made to the develop branch should appear on the stable branch within three to
four months as part of a stable release.
Other gem5 repositories
@@ -322,7 +322,7 @@ Branches
========
By default, contributions to gem5 should be made on the develop branch. The
master branch is maintained as a stable release branch (i.e., it can be pulled
stable branch is maintained as a stable release branch (i.e., it can be pulled
to obtain the latest official release of gem5). Creation of additional branches
is generally discouraged due to their tendency to bloat git repositories with
abandoned code. However, the creation of new branches is permitted for
@@ -491,7 +491,7 @@ release are submitted to the develop branch.
2. When a release is ready, a new staging branch shall be created by a project
maintainer, from develop, with the name "release-staging-{VERSION}". The
gem5-dev mailing list will be notified that the staging branch will be merged
into the master branch after two weeks, thus marking the new release.
into the stable branch after two weeks, thus marking the new release.
3. The staging branch will have the full suite of gem5 tests run on it to
ensure all tests pass and the to-be-released code is in a decent state.
4. If a user submits a changeset to the staging branch, it will be considered
@@ -504,8 +504,8 @@ change may be submitted directly to the staging branch. All other submissions
to gem5 will continue to be made to the develop branch. Patches submitted
into the staging branch do not need to be re-added to the develop branch.
5. Once signed off by members of the PMC the staging branch shall be merged
into the master and develop branch. The staging branch will then be deleted.
6. The master branch shall be tagged with the correct version number for that
into the stable and develop branch. The staging branch will then be deleted.
6. The stable branch shall be tagged with the correct version number for that
release. gem5 conforms to a "v{YY}.{MAJOR}.{MINOR}.{HOTFIX}" versioning system.
E.g., the first major release of 2022 will be "v22.0.0.0", followed by
"v22.1.0.0". All the releases (with the exception of hotfixes) are considered
@@ -527,14 +527,14 @@ whether the issue is worthy of a hotfix, and the final decision should be
made by members of the PMC if there is no consensus. Assuming the hotfix is
permitted, the following steps will be taken:
1. A new branch with the prefix "hotfix-" will be created from the master
1. A new branch with the prefix "hotfix-" will be created from the stable
branch. Only gem5 maintainers can create branches. If a non-maintainer requires
the creation of a hotfix branch then they should contact a gem5 maintainer.
2. The change shall be submitted to the hotfix branch via gerrit. Full review,
as with any other change, will be required.
3. Once fully submitted, the hotfix branch shall be merged into both the
develop and the master branch by a gem5 maintainer.
4. The master branch will be tagged with the new version number; the same as
develop and the stable branch by a gem5 maintainer.
4. The stable branch will be tagged with the new version number; the same as
the last but with an incremented hotfix number (e.g., "v20.2.0.0" would
transition to "v20.2.0.1").
4. The hotfix branch will then be deleted.

View File

@@ -1,6 +1,124 @@
# Version 20.1.0.0
* m5.stats.dump() root argument renamed to roots to reflect the fact that it now takes a list of SimObjects
Thank you to everyone that made this release possible!
This has been a very productive release with [150 issues](https://gem5.atlassian.net/), over 650 commits (a 25% increase from the 20.0 release), and 58 unique contributors (a 100% increase!).
## Process changes
We are no longer using the "master" branch.
Instead, we will have two branches:
* "stable": This will point to the latest stable release (gem5-20.1 as of today)
* "develop": This is the latest development code that will be merged in to the "stable" branch at each release.
We suggest all *users* use the stable (default) branch.
However, to contribute your fixes and new changes to gem5, it should be contributed to the develop branch.
See CONTRIBUTING.md for more details.
gem5 has also implemented a project code of conduct.
See the CODE-OF-CONDUCT.md file for details.
In the code of conduct "we pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community."
## New features in 20.1
### New DRAM interface: Contributed by *Wendy Elsasser*
You can find details about this on the [gem5 blog](http://www.gem5.org/2020/05/27/memory-controller.html) or Wendy's talks on YouTube: [Talk on new interface and NVM](https://www.youtube.com/watch?v=t2PRoZPwwpk) and the [talk on LPDDR5](https://www.youtube.com/watch?v=ttJ9_I_Avyc)
* **[PYTHON API CHANGE]**: The DRAM models are now *DRAM interfaces* which is a child of the *memory controller*. Example change shown [in the blog post](http://www.gem5.org/project/2020/07/18/gem5-20-1.html).
* The DRAM is split into a memory controller and a DRAM interface
* `SimpleMemory` is no longer a drop-in replacement for a DRAM-based memory controller.
* LPDDR5 model added
* NVM model added
* New memory controller model that can use both NVM and DRAM
### Improved on-chip interconnect model, HeteroGarnet: Contributed by *Srikant Bharadwaj*
You can find details about this on the [gem5 blog](http://www.gem5.org/2020/05/27/heterogarnet.html) and [Srikant's talk on YouTube](https://www.youtube.com/watch?v=AH9r44r2lHA).
* **[USER-FACING CHANGE]**: The network type options are now "simple" and "garnet" instead of "garnet2.0". (If "garnet2.0" is used, you will get a warning until gem5-20.2)
* Added models for clock domain crossings and serialization/deserialization (SERDES)
### Transactional memory support: Contributed by *Timothy Hayes*
You can find details on the [Jira issue](https://gem5.atlassian.net/browse/GEM5-587)
* gem5 now supports Arm TME (transactional memory extensions)
* Transactional memory is only implemented in the `MESI_Three_Level_HTM` Ruby protocol, and it is only implemented in Ruby.
* This implements a checkpointing mechanism for the architectural state and buffering of speculative memory updates.
* IBM POWER and x86 HTM extensions have *not* been implemented.
### Other new features
* External simulator integrations
* Added support for DRAMSim3
* Added back support for DRAMSim2
* Armv8-A Self Hosted Debug extension added
* KVM support for Armv8-A hosts without GICv2 hardware
* Implemented Secure EL2 for Armv8-A
## Removed features
* Dropped support for mercurial version control
## New supported platforms
* GCC up to 10.2 is now supported. Minimum GCC is now 5.0.
* Clang up to version 9. Minimum Clang is now 3.9.
## Platforms no longer support
* **[USER-FACING CHANGE]**: Python 2 is officially deprecated. We will drop support for Python 2 in the next release. In this release you will get a warning if you're using Python 2.
* **[USER-FACING CHANGE]**: We have dropped support for GCC 4.X
* **[USER-FACING CHANGE]**: We have dropped support for Scons 2.x (Note: this is the default in Ubuntu 16.04. Use pip to install a newer scons.)
See <http://www.gem5.org/documentation/general_docs/building> for gem5's current dependencies.
## Other changes
### Deprecating "master" and "slave"
* **[API CHANGE]**: The names "master" and "slave" have been deprecated
* Special thanks to Lakin Smith, Shivani Parekh, Eden Avivi, and Emily Brickey.
* Below is a guide to most of the name changes.
* The goal was to replace problematic language with more descriptive and precise terms.
* There may be some bugs introduced with this change as there were many places in the Python configurations which relied on "duck typing".
* This change is mostly backwards compatible and warning will be issued until at least gem5-20.2.
```
MasterPort -> RequestorPort
SlavePort -> ResponsePort
xbar.slave -> xbar.cpu_side
xbar.master -> xbar.mem_side
MasterId -> RequestorId
```
### Testing improvements
* We now have Jenkins server (<http://jenkins.gem5.org/>) running nightly and other large tests. Special thanks to Mike Upton for setting this up!
* Nightly tests run the "long" regression tests (many tests added).
* Compiler tests run gem5 build for many targets and all compilers once a week.
* Updated CI tester (kokoro) to use a more up-to-date environment.
* Improved the testing infrastructure.
* Almost all testing resources now available in [gem5-resources repository](https://gem5.googlesource.com/public/gem5-resources/).
* Generally cleaned up the `tests/` directory in mainline gem5.
* Many general testlib improvements.
### More changes
* **[PYTHON API CHANGE]**: m5.stats.dump() root argument renamed to roots to reflect the fact that it now takes a list of SimObjects
* **[USER-FACING CHANGE]**: Checkpoint compatibility may be broken by the following changes
* <https://gem5-review.googlesource.com/c/public/gem5/+/25145>
* <https://gem5-review.googlesource.com/c/public/gem5/+/31874>
* **[API CHANGE]** Changed `setCPU` to `setThreadContext` in Interrupts
* Added a `Compressor` namespace.
* **[API CHANGE]** The `Callback` class was removed and replaced with C++ lambdas.
* Many objects' stats have been updated to the "new" stats style.
* Many more objects have had their APIs formalized. See <http://www.gem5.org/documentation/general_docs/gem5-apis>
----------------------------------------------------------------------------------------------------
# Version 20.0.0.3

View File

@@ -238,7 +238,8 @@ global_vars.AddVariables(
('MARSHAL_CCFLAGS_EXTRA', 'Extra C and C++ marshal compiler flags', ''),
('MARSHAL_LDFLAGS_EXTRA', 'Extra marshal linker flags', ''),
('PYTHON_CONFIG', 'Python config binary to use',
[ 'python2.7-config', 'python-config', 'python3-config' ]),
[ 'python3-config', 'python-config', 'python2.7-config', 'python2-config']
),
('PROTOC', 'protoc tool', environ.get('PROTOC', 'protoc')),
('BATCH', 'Use batch pool for build and tests', False),
('BATCH_CMD', 'Batch pool submission command name', 'qdo'),
@@ -336,12 +337,6 @@ if main['GCC'] or main['CLANG']:
main.Append(PSHLINKFLAGS=shared_partial_flags)
main.Append(PLINKFLAGS=shared_partial_flags)
# Treat warnings as errors but white list some warnings that we
# want to allow (e.g., deprecation warnings).
main.Append(CCFLAGS=['-Werror',
'-Wno-error=deprecated-declarations',
'-Wno-error=deprecated',
])
else:
error('\n'.join((
"Don't know what compiler options to use for your compiler.",
@@ -636,6 +631,10 @@ if main['USE_PYTHON']:
main['PYTHON_CONFIG'])
print("Info: Using Python config: %s" % (python_config, ))
if python_config != 'python3-config':
warning('python3-config could not be found.\n'
'Future releases of gem5 will drop support for python2.')
py_includes = readCommand([python_config, '--includes'],
exception='').split()
py_includes = list(filter(

View File

@@ -136,7 +136,7 @@ def setup_memory_controllers(system, ruby, dir_cntrls, options):
mem_ctrl = m5.objects.MemCtrl(dram = dram_intf)
if options.access_backing_store:
mem_ctrl.kvm_map=False
dram_intf.kvm_map=False
mem_ctrls.append(mem_ctrl)
dir_ranges.append(mem_ctrl.dram.range)

View File

@@ -12,6 +12,7 @@
#ifdef __clang__
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wunused-value"
#pragma clang diagnostic warning "-Wrange-loop-analysis"
#endif
#if defined(__INTEL_COMPILER)

View File

@@ -214,7 +214,7 @@ def define_defaults(defaults):
os.pardir,
os.pardir))
defaults.result_path = os.path.join(os.getcwd(), 'testing-results')
defaults.resource_url = 'http://dist.gem5.org/dist/develop'
defaults.resource_url = 'http://dist.gem5.org/dist/v20-1'
defaults.resource_path = os.path.abspath(os.path.join(defaults.base_dir,
'tests',
'gem5',

View File

@@ -31,7 +31,7 @@ PROJECT_NAME = gem5
# This could be handy for archiving the generated documentation or
# if some version control system is used.
PROJECT_NUMBER = DEVELOP-FOR-V20.1
PROJECT_NUMBER = v20.1.0.0
# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute)
# base path where the generated documentation will be put.

View File

@@ -109,15 +109,16 @@ Tstart64::completeAcc(PacketPtr pkt, ExecContext *xc,
// checkpointing occurs in the outer transaction only
if (htm_depth == 1) {
auto new_cpt = new HTMCheckpoint();
BaseHTMCheckpointPtr& cpt = xc->tcBase()->getHtmCheckpointPtr();
new_cpt->save(tc);
new_cpt->destinationRegister(dest);
HTMCheckpoint *armcpt =
dynamic_cast<HTMCheckpoint*>(cpt.get());
assert(armcpt != nullptr);
armcpt->save(tc);
armcpt->destinationRegister(dest);
ArmISA::globalClearExclusive(tc);
xc->tcBase()->setHtmCheckpointPtr(
std::unique_ptr<BaseHTMCheckpoint>(new_cpt));
}
xc->setIntRegOperand(this, 0, (Dest64) & mask(intWidth));

View File

@@ -38,6 +38,7 @@
#include "arch/arm/isa.hh"
#include "arch/arm/faults.hh"
#include "arch/arm/htm.hh"
#include "arch/arm/interrupts.hh"
#include "arch/arm/pmu.hh"
#include "arch/arm/self_debug.hh"
@@ -439,9 +440,15 @@ ISA::startup()
{
BaseISA::startup();
if (tc)
if (tc) {
setupThreadContext();
if (haveTME) {
std::unique_ptr<BaseHTMCheckpoint> cpt(new HTMCheckpoint());
tc->setHtmCheckpointPtr(std::move(cpt));
}
}
afterStartup = true;
}

View File

@@ -29,4 +29,4 @@
/**
* @ingroup api_base_utils
*/
const char *gem5Version = "[DEVELOP-FOR-V20.1]";
const char *gem5Version = "20.1.0.0";

View File

@@ -347,7 +347,6 @@ template <class Impl>
void
O3ThreadContext<Impl>::setHtmCheckpointPtr(BaseHTMCheckpointPtr new_cpt)
{
assert(!thread->htmCheckpoint->valid());
thread->htmCheckpoint = std::move(new_cpt);
}

View File

@@ -196,6 +196,5 @@ SimpleThread::getHtmCheckpointPtr()
void
SimpleThread::setHtmCheckpointPtr(BaseHTMCheckpointPtr new_cpt)
{
assert(!_htmCheckpoint->valid());
_htmCheckpoint = std::move(new_cpt);
}

View File

@@ -92,7 +92,8 @@ AbstractMemory::initState()
panic_if(!image_range.isSubset(range), "%s: memory image %s doesn't fit.",
name(), file);
PortProxy proxy([this](PacketPtr pkt) { functionalAccess(pkt); }, size());
PortProxy proxy([this](PacketPtr pkt) { functionalAccess(pkt); },
system()->cacheLineSize());
panic_if(!image.write(proxy), "%s: Unable to write image.");
}

View File

@@ -41,8 +41,8 @@
#include <vector>
std::ostream& operator<<(std::ostream& os, const BoolVec& myvector) {
for (const auto& it: myvector) {
os << " " << it;
for (const bool e: myvector) {
os << " " << e;
}
return os;
}

View File

@@ -256,8 +256,8 @@ inline int
countBoolVec(BoolVec bVec)
{
int count = 0;
for (const auto &it: bVec) {
if (it) {
for (const bool e: bVec) {
if (e) {
count++;
}
}

View File

@@ -167,7 +167,7 @@ Sequencer::wakeup()
int total_outstanding = 0;
for (const auto &table_entry : m_RequestTable) {
for (const auto seq_req : table_entry.second) {
for (const auto &seq_req : table_entry.second) {
if (current_time - seq_req.issue_time < m_deadlock_threshold)
continue;

View File

@@ -175,7 +175,15 @@ def simulate(*args, **kwargs):
if _drain_manager.isDrained():
_drain_manager.resume()
return _m5.event.simulate(*args, **kwargs)
# We flush stdout and stderr before and after the simulation to ensure the
# output arrive in order.
sys.stdout.flush()
sys.stderr.flush()
sim_out = _m5.event.simulate(*args, **kwargs)
sys.stdout.flush()
sys.stderr.flush()
return sim_out
def drain():
"""Drain the simulator in preparation of a checkpoint or memory mode

View File

@@ -103,7 +103,7 @@ const std::string DIST_SIZE = "dist-size";
static inline void
panicFsOnlyPseudoInst(const char *name)
{
panic("Pseudo inst \"%s\" is only available in Full System mode.");
panic("Pseudo inst \"%s\" is only available in Full System mode.", name);
}
void

View File

@@ -1,6 +1,2 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000000 ticks per second
-50000

View File

@@ -1,6 +1,2 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000000 ticks per second
-776.000061

View File

@@ -1,7 +1,3 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000000 ticks per second
--- Done DRAM low power sweep ---
Fixed params -

View File

@@ -42,6 +42,7 @@ import shutil
import sys
import socket
import threading
import gzip
from six.moves import urllib
@@ -260,11 +261,11 @@ class DownloadedProgram(UniqueFixture):
and downloads an updated version if it is needed.
"""
def __new__(cls, url, path, filename):
def __new__(cls, url, path, filename, gzip_decompress=False):
target = joinpath(path, filename)
return super(DownloadedProgram, cls).__new__(cls, target)
def _init(self, url, path, filename, **kwargs):
def _init(self, url, path, filename, gzip_decompress=False, **kwargs):
"""
url: string
The url of the archive
@@ -272,12 +273,16 @@ class DownloadedProgram(UniqueFixture):
The absolute path of the directory containing the archive
filename: string
The name of the archive
gzip_decompress: boolean
True if this target resource have been compressed using gzip and
is to be decompressed prior to usage.
"""
self.url = url
self.path = path
self.filename = joinpath(path, filename)
self.name = "Downloaded:" + self.filename
self.gzip_decompress = gzip_decompress
def _download(self):
import errno
@@ -288,7 +293,17 @@ class DownloadedProgram(UniqueFixture):
except OSError as e:
if e.errno != errno.EEXIST:
raise
urllib.request.urlretrieve(self.url, self.filename)
if self.gzip_decompress:
gzipped_filename = self.filename + ".gz"
urllib.request.urlretrieve(self.url, gzipped_filename)
with open(self.filename, 'w') as outfile:
with gzip.open(gzipped_filename, 'r') as infile:
shutil.copyfileobj(infile, outfile)
os.remove(gzipped_filename)
else:
urllib.request.urlretrieve(self.url, self.filename)
def _getremotetime(self):
import datetime, time

View File

@@ -1,7 +1,3 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000000 ticks per second
**** REAL SIMULATION ****
Hello world!

View File

@@ -1,7 +1,3 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000000 ticks per second
**** REAL SIMULATION ****
Begining test of difficult SPARC instructions...

View File

@@ -1,7 +1,3 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000000 ticks per second
Beginning simulation!
Hello world!

View File

@@ -1,7 +1,3 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000000 ticks per second
Beginning simulation!
Exiting @ tick 10944163 because Goodbye hello!! Goodbye hello!! Goodbye hello!! Goodbye hello!! Goodbye hello!! Goodbye hello!! Goo

View File

@@ -1,7 +1,3 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000000 ticks per second
Hello World! From a SimObject!
Beginning simulation!

View File

@@ -1,7 +1,3 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000 ticks per second
Beginning simulation!
Exiting @ tick 9981 because Ruby Tester completed

View File

@@ -1,7 +1,3 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000000 ticks per second
Beginning simulation!
Running on 2 cores. with 100 values

View File

@@ -1,7 +1,3 @@
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
Global frequency set at 1000000000000 ticks per second
Init done
[Iteration 1, Thread 1] Got lock

View File

@@ -115,6 +115,9 @@ class DerivedGoldStandard(MatchGoldStandard):
class MatchStdout(DerivedGoldStandard):
_file = constants.gem5_simulation_stdout
_default_ignore_regex = [
re.compile('^\s+$'), # Remove blank lines.
re.compile('^gem5 Simulator System'),
re.compile('^gem5 is copyrighted software'),
re.compile('^Redirecting (stdout|stderr) to'),
re.compile('^gem5 version '),
re.compile('^gem5 compiled '),

View File

@@ -34,13 +34,13 @@ else:
base_path = joinpath(absdirpath(__file__), '..', 'resources',
'ubuntu-boot')
image_url = config.resource_url + '/images/x86/ubuntu-18-04/base.img'
image_url = config.resource_url + '/images/x86/ubuntu-18-04/base.img.gz'
kernel_url = config.resource_url + '/kernels/x86/static/vmlinux-4.19.83'
image_name = 'ubuntu-18-04-base.img'
kernel_name = 'vmlinux-4.19.83' # 4.19 is LTS (Projected EOL: Dec, 2020)
image = DownloadedProgram(image_url, base_path, image_name)
image = DownloadedProgram(image_url, base_path, image_name, True)
kernel = DownloadedProgram(kernel_url, base_path, kernel_name)