derek/gem5
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

misc: Merge branch 'release-staging-v22-0' into stable

Browse Source
This commit is contained in:
Bobby R. Bruce
2022-06-17 20:21:37 -04:00
parent e4fae58da6 962de4c6ce
commit 7b9364b5c0
1416 changed files with 99642 additions and 54541 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
CONTRIBUTING.md
View File

@@ -144,13 +144,17 @@ changes, we require all change descriptions be strictly formatted.
A canonical commit message consists of three parts:
* A short summary line describing the change. This line starts with one or
more keywords (found in the MAINTAINERS file) separated by commas followed
by a colon and a description of the change. This line should be no more than
65 characters long since version control systems usually add a prefix that
causes line-wrapping for longer lines.
by a colon and a description of the change. This short description is
written in the imperative mood, and should say what happens when the patch
is applied. Keep it short and simple. Write it in sentence case preferably
not ending in a period. This line should be no more than 65 characters long
since version control systems usually add a prefix that causes line-wrapping
for longer lines.
* (Optional, but highly recommended) A detailed description. This describes
what you have done and why. If the change isn't obvious, you might want to
motivate why it is needed. Lines need to be wrapped to 72 characters or
less.
less. Leave a blank line between the first short summary line and this
detailed description.
* Tags describing patch metadata. You are highly recommended to use
tags to acknowledge reviewers for their work. Gerrit will automatically add
most tags.

114
KCONFIG.md Normal file
View File

@@ -0,0 +1,114 @@
This file explains how to work with gem5's implementation of the kconfig
configuration system, very similar to what's used by the linux kernel. It talks
about how to work with the Kconfig files themselves which define what user
adjustable configuration parameters there are, and how they work and
interoperate.
This file does *not*:
* Describe how kconfig works generally. This is well documented elsewhere, for
instance [here](
https://www.kernel.org/doc/html/latest/kbuild/kconfig-language.html):
* The settings in gem5's kconfig files. These should be documented with help
text within the kconfig files, and in the code where these options are
consumed.
* The various tools which can manipulate a given configuration. These are
documented in gem5's SCons help text.
# kconfiglib
gem5 uses the kconfiglib python library for consuming and manipulating kconfig
configurations. It is very similar to the kconfig implementation used by the
linux kernel, but is slightly different and has some small extensions added to
it. Almost all kconfig documentation written for the kernel's implementation
should apply here as well, but it may occasionally be necessary to refer to the
kconfiglib documentation.
Also, because gem5's build system is more modular than kconfig's design
supports out of the box, particularly for "choice" blocks, we have extended
kconfiglib and added a "cont_choice" keyword. This keyword is very similar to
"choice", except "cont_choice" blocks can be re-opened and extended with more
options further into the config.
This can be used to set up a central point where the user can choose between
mutually exclusive options, and still allow new Kconfig files to add new
options without modifying the original source.
Having to choose between mutually exclusive options should be avoided in
general, but is unavoidable in a few key places in gem5 at the moment. Once
those areas have been addressed, this keyword may be removed in the future.
# The 'CONF' dict in the SCons environment
In "env" SCons environment in SConscript files, or the "main" environment in
SConsopts files, can hold many variables which help SCons operate generally,
like setting what include paths to use, what the compiler command line is, etc.
These environments each have a 'CONF' sub-dict which holds all the variables
which are actually used to configure gem5, and not to configure SCons and the
build process itself.
All variables in this dict are automatically available to include in c++. To
access the value of env['CONF']['FOO'], you would #include "config/foo.hh".
Because these variables are in a shared namespace, their names should be unique
and distinctive.
These values are available in config scripts through the m5.defines.buildEnv
dict.
# Automatic/measured configuration values.
Some configuration values are not set by the user, and are measured from the
host environment. These could reflect the availability of a header file,
library or tool, whether the compiler supports a particular option or language
feature, etc.
These values should be measured in SConsopts files, and stored in the 'CONF'
dict described above. Like any other variable in 'CONF', they are then
available to C++ through generated header files, to config scripts through
buildEnv, etc. They are also available in the kconfig files themselves through
a mechanism discussed below.
# Accessing 'CONF' values in Kconfig files.
When the gem5 Kconfig files are processed to either manipulate a configuration
through a tool, or to apply a configuration to the gem5 build, all the values
in 'CONF' are temporarily put into environment variables. In the Kconfig files
themselves, these environment variables can be accessed using $(FOO) syntax,
which is described in kconfiglib's documentation.
Note that this is slightly different from the kernel's Kconfig syntax, where
the environment variables would have to be imported in using other keywords
first.
This is generally used to make automatic/measured settings which were
determined in SConsopts files available in Kconfig files. They can then be used
to compute dependencies, or to set default values, etc.
# Structure of the Kconfig hierarchy
Unlike SConscript files, gem5 does not find Kconfig files automatically, and
they are only used if they are included explicitly in other Kconfig files.
Kconfig options should be defined as close as possible to where they are used.
This makes them easier to find, keeps related functionality grouped
together in the source tree, and minimizes conflicts from modifying the same
few, central files when changing unrelated parts of gem5.
When including a Kconfig file in another, you should use the "rsource" keyword
which is a kconfiglib extension. This lets you include the other file using a
path which is relative to the current file, and also helps make the kconfig
files more modular and self contained.
# EXTRAS directories.
The EXTRAS variable can be set to a list of directories which hold additional
source that should be built into gem5. Because there's no way to know what (if
any) paths will be in EXTRAS ahead of time, it is not possible to explicitly
include kconfig files in those directories from a static file.
Instead, gem5's real root Kconfig file, which includes the one in src, is
automatically generated as part of the build. It uses the kconfiglib extension
"osource" to optionally source a file called Kconfig in the base of each EXTRAS
directory after it has sourced gem5's main Kconfig. If you want to add Kconfig
options to your EXTRAS directory, you can create that file, and then rsource
any additional internal Kconfig files as needed.

115
RELEASE-NOTES.md
View File

@@ -1,3 +1,118 @@
# Version 22.0.0.0
gem5 version 22.0 has been slightly delayed, but we a have a very strong release!
This release has 660 changes from 48 unique contributors.
While there are not too many big ticket features, the community has done a lot to improve the stablity and add bugfixes to gem5 over this release.
That said, we have a few cool new features like full system GPU support, a huge number of Arm improvements, and an improved HBM model.
See below for more details!
## New features
- [Arm now models DVM messages for TLBIs and DSBs accurately](https://gem5.atlassian.net/browse/GEM5-1097). This is implemented in the CHI protocol.
- EL2/EL3 support on by default in ArmSystem
- HBM controller which supports pseudo channels
- [Improved Ruby's SimpleNetwork routing](https://gem5.atlassian.net/browse/GEM5-920)
- Added x86 bare metal workload and better real mode support
- [Added round-robin arbitration when using multiple prefetchers](https://gem5.atlassian.net/browse/GEM5-1169)
- [KVM Emulation added for ARM GIGv3](https://gem5.atlassian.net/browse/GEM5-1138)
- Many improvements to the CHI protocol
## Many RISC-V instructions added
The following RISCV instructions have been added to gem5's RISC-V ISA:
* Zba instructions: add.uw, sh1add, sh1add.uw, sh2add, sh2add.uw, sh3add, sh3add.uw, slli.uw
* Zbb instructions: andn, orn, xnor, clz, clzw, ctz, ctzw, cpop, cpopw, max, maxu, min, minu, sext.b, sext.h, zext.h, rol, rolw, ror, rori, roriw, rorw, orc.b, rev8
* Zbc instructions: clmul, clmulh, clmulr
* Zbs instructions: bclr, bclri, bext, bexti, binv, binvi, bset, bseti
* Zfh instructions: flh, fsh, fmadd.h, fmsub.h, fnmsub.h, fnmadd.h, fadd.h, fsub.h, fmul.h, fdiv.h, fsqrt.h, fsgnj.h, fsgnjn.h, fsgnjx.h, fmin.h, fmax.h, fcvt.s.h, fcvt.h.s, fcvt.d.h, fcvt.h.d, fcvt.w.h, fcvt.h.w, fcvt.wu.h, fcvt.h.wu
### Improvements to the stdlib automatic resource downloader
The gem5 standard library's downloader has been re-engineered to more efficiently obtain the `resources.json` file.
It is now cached instead of retrieved on each resource retrieval.
The `resources.json` directory has been moved to a more permament URL at <http://resources.gem5.org/resources.json>.
Tests have also been added to ensure the resources module continues to function correctly.
### gem5 in SystemC support revamped
The gem5 in SystemC has been revamped to accomodate new research needs.
These changes include stability improvements and bugs fixes.
The gem5 testing suite has also been expanded to include gem5 in SystemC tests.
### Improved GPU support
Users may now simulate an AMD GPU device in full system mode using the ROCm 4.2 compute stack.
Until v21.2, gem5 only supported GPU simulation in Syscall-Emulation mode with ROCm 4.0.
See [`src/gpu-fs/README.md`](https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/gpu-fs/) in gem5-resources and example scripts in [`configs/example/gpufs/`](https://gem5.googlesource.com/public/gem5/+/refs/tags/v22.0.0.0/configs/example/gpufs/) for example scripts which run GPU full system simulations.
A [GPU Ruby random tester has been added](https://gem5-review.googlesource.com/c/public/gem5/+/59272) to help validate the correctness of the CPU and GPU Ruby coherence protocols as part of every kokoro check-in.
This helps validate the correctness of the protocols before new changes are checked in.
Currently the tester focuses on the protocols used with the GPU, but the ideas are extensible to other protocols.
The work is based on "Autonomous Data-Race-Free GPU Testing", IISWC 2019, Tuan Ta, Xianwei Zhang, Anthony Gutierrez, and Bradford M. Beckmann.
### An Arm board has been added to the gem5 Standard Library
Via [this change](https://gem5-review.googlesource.com/c/public/gem5/+/58910), an ARM Board, `ArmBoard`, has been added to the gem5 standard library.
This allows for an ARM system to be run using the gem5 stdlib components.
An example gem5 configuration script using this board can be found in `configs/example/gem5_library/arm-ubuntu-boot-exit.py`.
### `createAddrRanges` now supports NUMA configurations
When the system is configured for NUMA, it has multiple memory ranges, and each memory range is mapped to a corresponding NUMA node. For this, the change enables `createAddrRanges` to map address ranges to only a given HNFs.
Jira ticker here: https://gem5.atlassian.net/browse/GEM5-1187.
## API (user-facing) changes
### CPU model types are no longer simply the model name, but they are specialized for each ISA
For instance, the `O3CPU` is now the `X86O3CPU` and `ArmO3CPU`, etc.
This requires a number of changes if you have your own CPU models.
See https://gem5-review.googlesource.com/c/public/gem5/+/52490 for details.
Additionally, this requires changes in any configuration script which inherits from the old CPU types.
In many cases, if there is only a single ISA compiled the old name will still work.
However, this is not 100% true.
Finally, `CPU_MODELS` is no longer a parameter in `build_opts/`.
Now, if you want to compile a CPU model for a particular ISA you will have to add a new file for the CPU model in the `arch/` directory.
### Many changes in the CPU and ISA APIs
If you have any specialized CPU models or any ISAs which are not in the mainline, expect many changes when rebasing on this release.
- No longer use read/setIntReg (e.g., see https://gem5-review.googlesource.com/c/public/gem5/+/49766)
- InvalidRegClass has changed (e.g., see https://gem5-review.googlesource.com/c/public/gem5/+/49745)
- All of the register classes have changed (e.g., see https://gem5-review.googlesource.com/c/public/gem5/+/49764/)
- `initiateSpecialMemCmd` renamed to `initiateMemMgmtCmd` to generalize to other command beyond HTM (e.g., DVM/TLBI)
- `OperandDesc` class added (e.g., see https://gem5-review.googlesource.com/c/public/gem5/+/49731)
- Many cases of `TheISA` have been removed
## Bug Fixes
- [Fixed RISC-V call/ret instruction decoding](https://gem5-review.googlesource.com/c/public/gem5/+/58209). The fix adds IsReturn` and `IsCall` flags for RISC-V jump instructions by defining a new `JumpConstructor` in "standard.isa". Jira Ticket here: https://gem5.atlassian.net/browse/GEM5-1139.
- [Fixed x86 Read-Modify-Write behavior in multiple timing cores with classic caches](https://gem5-review.googlesource.com/c/public/gem5/+/55744). Jira Ticket here: https://gem5.atlassian.net/browse/GEM5-1105.
- [The circular buffer for the O3 LSQ has been fixed](https://gem5-review.googlesource.com/c/public/gem5/+/58649). This issue affected running the O3 CPU with large workloaders. Jira Ticket here: https://gem5.atlassian.net/browse/GEM5-1203.
- [Removed "memory-leak"-like error in RISC-V lr/sc implementation](https://gem5-review.googlesource.com/c/public/gem5/+/55663). Jira issue here: https://gem5.atlassian.net/browse/GEM5-1170.
- [Resolved issues with Ruby's memtest](https://gem5-review.googlesource.com/c/public/gem5/+/56811). In gem5 v21.2, If the size of the address range was smaller than the maximum number of outstandnig requests allowed downstream, the tester would get stuck trying to find a unique address. This has been resolved.
## Build-related changes
- Variable in `env` in the SConscript files now requires you to use `env['CONF']` to access them. Anywhere that `env['<VARIABLE>']` appeared should noe be `env['CONF']['<VARIABLE>']`
- Internal build files are now in a per-target `gem5.build` directory
- All build variable are per-target and there are no longer any shared variables.
## Other changes
- New bootloader is required for Arm VExpress_GEM5_Foundation platform. See https://gem5.atlassian.net/browse/GEM5-1222 for details.
- The MemCtrl interface has been updated to use more inheritance to make extending it to other memory types (e.g., HBM pseudo channels) easier.
# Version 21.2.1.1
**[HOTFIX]** In order to ensure v21 of gem5 remains compatible with future changes, the gem5 stdlib downloader has been updated to obtain the resources.json file from <https://resources.gem5.org/resources.json>.

53
SConsopts Normal file
View File

@@ -0,0 +1,53 @@
# Copyright (c) 2013, 2015-2020 ARM Limited
# All rights reserved.
#
# The license below extends only to copyright in the software and shall
# not be construed as granting a license to any other intellectual
# property including but not limited to intellectual property relating
# to a hardware implementation of the functionality of the software
# licensed hereunder. You may use the software subject to the license
# terms below provided that you ensure that this notice is replicated
# unmodified and in its entirety in all distributions of the software,
# modified or unmodified, in source code or in binary form.
#
# Copyright (c) 2011 Advanced Micro Devices, Inc.
# Copyright (c) 2009 The Hewlett-Packard Development Company
# Copyright (c) 2004-2005 The Regents of The University of Michigan
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
import os
import os.path
from gem5_scons import warning
Import('*')
sticky_vars.AddVariables(
('BATCH', 'Use batch pool for build and tests', False),
('BATCH_CMD', 'Batch pool submission command name', 'qdo'),
('M5_BUILD_CACHE', 'Cache built objects in this directory', False),
('USE_EFENCE', 'Link with Electric Fence malloc debugger', False),
)

673
SConstruct
View File

@@ -80,17 +80,28 @@ import atexit
import os
import sys
from os import mkdir, environ
from os import mkdir, remove, environ
from os.path import abspath, dirname, expanduser
from os.path import isdir, isfile
from os.path import join, split
import logging
logging.basicConfig()
# SCons imports
import SCons
import SCons.Node
import SCons.Node.FS
import SCons.Tool
if getattr(SCons, '__version__', None) in ('3.0.0', '3.0.1'):
# Monkey patch a fix which appears in version 3.0.2, since we only
# require version 3.0.0
def __hash__(self):
return hash(self.lstr)
import SCons.Subst
SCons.Subst.Literal.__hash__ = __hash__
########################################################################
#
@@ -98,6 +109,8 @@ import SCons.Tool
#
########################################################################
linker_options = ('bfd', 'gold', 'lld', 'mold')
AddOption('--no-colors', dest='use_colors', action='store_false',
help="Don't add color to abbreviated scons output")
AddOption('--with-cxx-config', action='store_true',
@@ -106,7 +119,10 @@ AddOption('--default',
help='Override which build_opts file to use for defaults')
AddOption('--ignore-style', action='store_true',
help='Disable style checking hooks')
AddOption('--gold-linker', action='store_true', help='Use the gold linker')
AddOption('--linker', action='store', default=None, choices=linker_options,
help=f'Select which linker to use ({", ".join(linker_options)})')
AddOption('--gold-linker', action='store_const', const='gold', dest='linker',
help='Use the gold linker. Deprecated: Use --linker=gold')
AddOption('--no-compress-debug', action='store_true',
help="Don't compress debug info in build files")
AddOption('--with-lto', action='store_true',
@@ -233,47 +249,6 @@ if not isdir(build_root):
mkdir(build_root)
main['BUILDROOT'] = build_root
main.SConsignFile(os.path.join(build_root, "sconsign"))
########################################################################
#
# Set up global sticky variables... these are common to an entire build
# tree (not specific to a particular build like X86)
#
########################################################################
global_vars_file = os.path.join(build_root, 'variables.global')
global_vars = Variables(global_vars_file, args=ARGUMENTS)
global_vars.AddVariables(
('CC', 'C compiler', environ.get('CC', main['CC'])),
('CXX', 'C++ compiler', environ.get('CXX', main['CXX'])),
('CCFLAGS_EXTRA', 'Extra C and C++ compiler flags', ''),
('GEM5PY_CCFLAGS_EXTRA', 'Extra C and C++ gem5py compiler flags', ''),
('GEM5PY_LINKFLAGS_EXTRA', 'Extra marshal gem5py flags', ''),
('LINKFLAGS_EXTRA', 'Extra linker flags', ''),
('PYTHON_CONFIG', 'Python config binary to use',
[ 'python3-config', 'python-config']
),
('PROTOC', 'protoc tool', environ.get('PROTOC', 'protoc')),
('BATCH', 'Use batch pool for build and tests', False),
('BATCH_CMD', 'Batch pool submission command name', 'qdo'),
('M5_BUILD_CACHE', 'Cache built objects in this directory', False),
('EXTRAS', 'Add extra directories to the compilation', '')
)
# Update main environment with values from ARGUMENTS & global_vars_file
global_vars.Update(main)
Help('''
Global build variables:
{help}
'''.format(help=global_vars.GenerateHelpText(main)), append=True)
# Save sticky variable settings back to current variables file
global_vars.Save(global_vars_file, main)
########################################################################
#
@@ -281,16 +256,8 @@ global_vars.Save(global_vars_file, main)
#
########################################################################
# Parse EXTRAS variable to build list of all directories where we're
# look for sources etc. This list is exported as extras_dir_list.
base_dir = Dir('#src').abspath
if main['EXTRAS']:
extras_dir_list = makePathListAbsolute(main['EXTRAS'].split(':'))
else:
extras_dir_list = []
Export('base_dir')
Export('extras_dir_list')
# the ext directory should be on the #includes path
main.Append(CPPPATH=[Dir('ext')])
@@ -323,151 +290,6 @@ main['CLANG'] = CXX_version and CXX_version.find('clang') >= 0
if main['GCC'] + main['CLANG'] > 1:
error('Two compilers enabled at once?')
# Set up default C++ compiler flags
if main['GCC'] or main['CLANG']:
# As gcc and clang share many flags, do the common parts here
main.Append(CCFLAGS=['-pipe'])
main.Append(CCFLAGS=['-fno-strict-aliasing'])
# Enable -Wall and -Wextra and then disable the few warnings that
# we consistently violate
main.Append(CCFLAGS=['-Wall', '-Wundef', '-Wextra',
'-Wno-sign-compare', '-Wno-unused-parameter'])
# We always compile using C++17
main.Append(CXXFLAGS=['-std=c++17'])
if sys.platform.startswith('freebsd'):
main.Append(CCFLAGS=['-I/usr/local/include'])
main.Append(CXXFLAGS=['-I/usr/local/include'])
# On FreeBSD we need libthr.
main.Append(LIBS=['thr'])
with gem5_scons.Configure(main) as conf:
conf.CheckLinkFlag('-Wl,--as-needed')
if GetOption('gold_linker'):
main.Append(LINKFLAGS='-fuse-ld=gold')
else:
error('\n'.join((
"Don't know what compiler options to use for your compiler.",
"compiler: " + main['CXX'],
"version: " + CXX_version.replace('\n', '<nl>') if
CXX_version else 'COMMAND NOT FOUND!',
"If you're trying to use a compiler other than GCC",
"or clang, there appears to be something wrong with your",
"environment.",
"",
"If you are trying to use a compiler other than those listed",
"above you will need to ease fix SConstruct and ",
"src/SConscript to support that compiler.")))
if main['GCC']:
if compareVersions(main['CXXVERSION'], "7") < 0:
error('gcc version 7 or newer required.\n'
'Installed version:', main['CXXVERSION'])
with gem5_scons.Configure(main) as conf:
# This warning has a false positive in the systemc code in g++ 11.1.
conf.CheckCxxFlag('-Wno-free-nonheap-object')
# Add the appropriate Link-Time Optimization (LTO) flags if `--with-lto` is
# set.
if GetOption('with_lto'):
# g++ uses "make" to parallelize LTO. The program can be overriden with
# the environment variable "MAKE", but we currently make no attempt to
# plumb that variable through.
parallelism = ''
if main.Detect('make'):
parallelism = '=%d' % GetOption('num_jobs')
else:
warning('"make" not found, link time optimization will be '
'single threaded.')
for var in 'LTO_CCFLAGS', 'LTO_LINKFLAGS':
# Use the same amount of jobs for LTO as we are running scons with.
main[var] = ['-flto%s' % parallelism]
main.Append(TCMALLOC_CCFLAGS=['-fno-builtin-malloc', '-fno-builtin-calloc',
'-fno-builtin-realloc', '-fno-builtin-free'])
elif main['CLANG']:
if compareVersions(main['CXXVERSION'], "6") < 0:
error('clang version 6 or newer required.\n'
'Installed version:', main['CXXVERSION'])
# Set the Link-Time Optimization (LTO) flags if enabled.
if GetOption('with_lto'):
for var in 'LTO_CCFLAGS', 'LTO_LINKFLAGS':
main[var] = ['-flto']
# clang has a few additional warnings that we disable.
with gem5_scons.Configure(main) as conf:
conf.CheckCxxFlag('-Wno-c99-designator')
conf.CheckCxxFlag('-Wno-defaulted-function-deleted')
main.Append(TCMALLOC_CCFLAGS=['-fno-builtin'])
# On Mac OS X/Darwin we need to also use libc++ (part of XCode) as
# opposed to libstdc++, as the later is dated.
if sys.platform == "darwin":
main.Append(CXXFLAGS=['-stdlib=libc++'])
main.Append(LIBS=['c++'])
# Add sanitizers flags
sanitizers=[]
if GetOption('with_ubsan'):
sanitizers.append('undefined')
if GetOption('with_asan'):
# Available for gcc >= 5 or llvm >= 3.1 both a requirement
# by the build system
sanitizers.append('address')
suppressions_file = Dir('util').File('lsan-suppressions').get_abspath()
suppressions_opt = 'suppressions=%s' % suppressions_file
suppressions_opts = ':'.join([suppressions_opt, 'print_suppressions=0'])
main['ENV']['LSAN_OPTIONS'] = suppressions_opts
print()
warning('To suppress false positive leaks, set the LSAN_OPTIONS '
'environment variable to "%s" when running gem5' %
suppressions_opts)
warning('LSAN_OPTIONS=%s' % suppressions_opts)
print()
if sanitizers:
sanitizers = ','.join(sanitizers)
if main['GCC'] or main['CLANG']:
main.Append(CCFLAGS=['-fsanitize=%s' % sanitizers,
'-fno-omit-frame-pointer'],
LINKFLAGS='-fsanitize=%s' % sanitizers)
else:
warning("Don't know how to enable %s sanitizer(s) for your "
"compiler." % sanitizers)
# Do this after we save setting back, or else we'll tack on an
# extra 'qdo' every time we run scons.
if main['BATCH']:
main['CC'] = main['BATCH_CMD'] + ' ' + main['CC']
main['CXX'] = main['BATCH_CMD'] + ' ' + main['CXX']
main['AS'] = main['BATCH_CMD'] + ' ' + main['AS']
main['AR'] = main['BATCH_CMD'] + ' ' + main['AR']
main['RANLIB'] = main['BATCH_CMD'] + ' ' + main['RANLIB']
if sys.platform == 'cygwin':
# cygwin has some header file issues...
main.Append(CCFLAGS=["-Wno-uninitialized"])
# Cache build files in the supplied directory.
if main['M5_BUILD_CACHE']:
print('Using build cache located at', main['M5_BUILD_CACHE'])
CacheDir(main['M5_BUILD_CACHE'])
if not GetOption('no_compress_debug'):
with gem5_scons.Configure(main) as conf:
if not conf.CheckCxxFlag('-gz'):
warning("Can't enable object file debug section compression")
if not conf.CheckLinkFlag('-gz'):
warning("Can't enable executable debug section compression")
########################################################################
#
@@ -532,112 +354,6 @@ def config_embedded_python(env):
warning('Embedded python library too new. '
f'Python 3 expected, found {ver_string}.')
if main['USE_PYTHON']:
config_embedded_python(main)
gem5py_env = main.Clone()
else:
gem5py_env = main.Clone()
config_embedded_python(gem5py_env)
# Bare minimum environment that only includes python
gem5py_env.Append(CCFLAGS=['${GEM5PY_CCFLAGS_EXTRA}'])
gem5py_env.Append(LINKFLAGS=['${GEM5PY_LINKFLAGS_EXTRA}'])
if GetOption('gprof') and GetOption('pprof'):
error('Only one type of profiling should be enabled at a time')
if GetOption('gprof'):
main.Append(CCFLAGS=['-g', '-pg'], LINKFLAGS=['-pg'])
if GetOption('pprof'):
main.Append(CCFLAGS=['-g'],
LINKFLAGS=['-Wl,--no-as-needed', '-lprofiler', '-Wl,--as-needed'])
main['HAVE_PKG_CONFIG'] = main.Detect('pkg-config')
with gem5_scons.Configure(main) as conf:
# On Solaris you need to use libsocket for socket ops
if not conf.CheckLibWithHeader(
[None, 'socket'], 'sys/socket.h', 'C++', 'accept(0,0,0);'):
error("Can't find library with socket calls (e.g. accept()).")
if not conf.CheckLibWithHeader('z', 'zlib.h', 'C++','zlibVersion();'):
error('Did not find needed zlib compression library '
'and/or zlib.h header file.\n'
'Please install zlib and try again.')
if not GetOption('without_tcmalloc'):
with gem5_scons.Configure(main) as conf:
if conf.CheckLib('tcmalloc'):
conf.env.Append(CCFLAGS=conf.env['TCMALLOC_CCFLAGS'])
elif conf.CheckLib('tcmalloc_minimal'):
conf.env.Append(CCFLAGS=conf.env['TCMALLOC_CCFLAGS'])
else:
warning("You can get a 12% performance improvement by "
"installing tcmalloc (libgoogle-perftools-dev package "
"on Ubuntu or RedHat).")
########################################################################
#
# Read and process SConsopts files. These can add new settings which
# affect each variant directory independently.
#
########################################################################
# Register a callback which is called after all SConsopts files have been read.
after_sconsopts_callbacks = []
def AfterSConsopts(cb):
after_sconsopts_callbacks.append(cb)
Export('AfterSConsopts')
# Sticky variables get saved in the variables file so they persist from
# one invocation to the next (unless overridden, in which case the new
# value becomes sticky).
sticky_vars = Variables(args=ARGUMENTS)
Export('sticky_vars')
# Sticky variables that should be exported to #defines in config/*.hh
# (see src/SConscript).
export_vars = []
Export('export_vars')
# Walk the tree and execute all SConsopts scripts that wil add to the
# above variables
if GetOption('verbose'):
print("Reading SConsopts")
for bdir in [ base_dir ] + extras_dir_list:
if not isdir(bdir):
error("Directory '%s' does not exist." % bdir)
for root, dirs, files in os.walk(bdir):
if 'SConsopts' in files:
if GetOption('verbose'):
print("Reading", os.path.join(root, 'SConsopts'))
SConscript(os.path.join(root, 'SConsopts'))
# Call any callbacks which the SConsopts files registered.
for cb in after_sconsopts_callbacks:
cb()
# Add any generic sticky variables here.
sticky_vars.Add(BoolVariable('USE_EFENCE',
'Link with Electric Fence malloc debugger', False))
########################################################################
#
# Find and process all the SConscript files in ext. These are shared by
# all variants in a build root.
#
########################################################################
ext_dir = Dir('#ext').abspath
ext_build_dirs = []
for root, dirs, files in os.walk(ext_dir):
if 'SConscript' in files:
build_dir = os.path.relpath(root, ext_dir)
ext_build_dirs.append(build_dir)
main.SConscript(os.path.join(root, 'SConscript'),
variant_dir=os.path.join(build_root, build_dir))
########################################################################
#
@@ -646,38 +362,260 @@ for root, dirs, files in os.walk(ext_dir):
########################################################################
for variant_path in variant_paths:
if not GetOption('silent'):
print("Building in", variant_path)
# Make a copy of the build-root environment to use for this config.
env = main.Clone()
env['BUILDDIR'] = variant_path
gem5_build = os.path.join(build_root, variant_path, 'gem5.build')
env['GEM5BUILD'] = gem5_build
Execute(Mkdir(gem5_build))
env.SConsignFile(os.path.join(gem5_build, 'sconsign'))
# Set up default C++ compiler flags
if env['GCC'] or env['CLANG']:
# As gcc and clang share many flags, do the common parts here
env.Append(CCFLAGS=['-pipe'])
env.Append(CCFLAGS=['-fno-strict-aliasing'])
# Enable -Wall and -Wextra and then disable the few warnings that
# we consistently violate
env.Append(CCFLAGS=['-Wall', '-Wundef', '-Wextra',
'-Wno-sign-compare', '-Wno-unused-parameter'])
# We always compile using C++17
env.Append(CXXFLAGS=['-std=c++17'])
if sys.platform.startswith('freebsd'):
env.Append(CCFLAGS=['-I/usr/local/include'])
env.Append(CXXFLAGS=['-I/usr/local/include'])
# On FreeBSD we need libthr.
env.Append(LIBS=['thr'])
with gem5_scons.Configure(env) as conf:
conf.CheckLinkFlag('-Wl,--as-needed')
linker = GetOption('linker')
if linker:
with gem5_scons.Configure(env) as conf:
if not conf.CheckLinkFlag(f'-fuse-ld={linker}'):
# check mold support for gcc older than 12.1.0
if linker == 'mold' and \
(env['GCC'] and \
compareVersions(env['CXXVERSION'],
"12.1.0") < 0) and \
((isdir('/usr/libexec/mold') and \
conf.CheckLinkFlag('-B/usr/libexec/mold')) or \
(isdir('/usr/local/libexec/mold') and \
conf.CheckLinkFlag('-B/usr/local/libexec/mold'))):
pass # support mold
else:
error(f'Linker "{linker}" is not supported')
if linker == 'gold' and not GetOption('with_lto'):
# Tell the gold linker to use threads. The gold linker
# segfaults if both threads and LTO are enabled.
conf.CheckLinkFlag('-Wl,--threads')
conf.CheckLinkFlag(
'-Wl,--thread-count=%d' % GetOption('num_jobs'))
else:
error('\n'.join((
"Don't know what compiler options to use for your compiler.",
"compiler: " + env['CXX'],
"version: " + CXX_version.replace('\n', '<nl>') if
CXX_version else 'COMMAND NOT FOUND!',
"If you're trying to use a compiler other than GCC",
"or clang, there appears to be something wrong with your",
"environment.",
"",
"If you are trying to use a compiler other than those listed",
"above you will need to ease fix SConstruct and ",
"src/SConscript to support that compiler.")))
if env['GCC']:
if compareVersions(env['CXXVERSION'], "7") < 0:
error('gcc version 7 or newer required.\n'
'Installed version:', env['CXXVERSION'])
with gem5_scons.Configure(env) as conf:
# This warning has a false positive in the systemc in g++ 11.1.
conf.CheckCxxFlag('-Wno-free-nonheap-object')
# Add the appropriate Link-Time Optimization (LTO) flags if
# `--with-lto` is set.
if GetOption('with_lto'):
# g++ uses "make" to parallelize LTO. The program can be overriden
# with the environment variable "MAKE", but we currently make no
# attempt to plumb that variable through.
parallelism = ''
if env.Detect('make'):
parallelism = '=%d' % GetOption('num_jobs')
else:
warning('"make" not found, link time optimization will be '
'single threaded.')
for var in 'LTO_CCFLAGS', 'LTO_LINKFLAGS':
# Use the same amount of jobs for LTO as scons.
env[var] = ['-flto%s' % parallelism]
env.Append(TCMALLOC_CCFLAGS=[
'-fno-builtin-malloc', '-fno-builtin-calloc',
'-fno-builtin-realloc', '-fno-builtin-free'])
elif env['CLANG']:
if compareVersions(env['CXXVERSION'], "6") < 0:
error('clang version 6 or newer required.\n'
'Installed version:', env['CXXVERSION'])
# Set the Link-Time Optimization (LTO) flags if enabled.
if GetOption('with_lto'):
for var in 'LTO_CCFLAGS', 'LTO_LINKFLAGS':
env[var] = ['-flto']
# clang has a few additional warnings that we disable.
with gem5_scons.Configure(env) as conf:
conf.CheckCxxFlag('-Wno-c99-designator')
conf.CheckCxxFlag('-Wno-defaulted-function-deleted')
env.Append(TCMALLOC_CCFLAGS=['-fno-builtin'])
# On Mac OS X/Darwin we need to also use libc++ (part of XCode) as
# opposed to libstdc++, as the later is dated.
if sys.platform == "darwin":
env.Append(CXXFLAGS=['-stdlib=libc++'])
env.Append(LIBS=['c++'])
# Add sanitizers flags
sanitizers=[]
if GetOption('with_ubsan'):
sanitizers.append('undefined')
if GetOption('with_asan'):
# Available for gcc >= 5 or llvm >= 3.1 both a requirement
# by the build system
sanitizers.append('address')
suppressions_file = Dir('util').File('lsan-suppressions').get_abspath()
suppressions_opt = 'suppressions=%s' % suppressions_file
suppressions_opts = ':'.join([suppressions_opt,
'print_suppressions=0'])
env['ENV']['LSAN_OPTIONS'] = suppressions_opts
print()
warning('To suppress false positive leaks, set the LSAN_OPTIONS '
'environment variable to "%s" when running gem5' %
suppressions_opts)
warning('LSAN_OPTIONS=%s' % suppressions_opts)
print()
if sanitizers:
sanitizers = ','.join(sanitizers)
if env['GCC'] or env['CLANG']:
env.Append(CCFLAGS=['-fsanitize=%s' % sanitizers,
'-fno-omit-frame-pointer'],
LINKFLAGS='-fsanitize=%s' % sanitizers)
else:
warning("Don't know how to enable %s sanitizer(s) for your "
"compiler." % sanitizers)
if sys.platform == 'cygwin':
# cygwin has some header file issues...
env.Append(CCFLAGS=["-Wno-uninitialized"])
if not GetOption('no_compress_debug'):
with gem5_scons.Configure(env) as conf:
if not conf.CheckCxxFlag('-gz'):
warning("Can't enable object file debug section compression")
if not conf.CheckLinkFlag('-gz'):
warning("Can't enable executable debug section compression")
if env['USE_PYTHON']:
config_embedded_python(env)
gem5py_env = env.Clone()
else:
gem5py_env = env.Clone()
config_embedded_python(gem5py_env)
# Bare minimum environment that only includes python
gem5py_env.Append(CCFLAGS=['${GEM5PY_CCFLAGS_EXTRA}'])
gem5py_env.Append(LINKFLAGS=['${GEM5PY_LINKFLAGS_EXTRA}'])
if GetOption('gprof') and GetOption('pprof'):
error('Only one type of profiling should be enabled at a time')
if GetOption('gprof'):
env.Append(CCFLAGS=['-g', '-pg'], LINKFLAGS=['-pg'])
if GetOption('pprof'):
env.Append(CCFLAGS=['-g'],
LINKFLAGS=['-Wl,--no-as-needed', '-lprofiler',
'-Wl,--as-needed'])
env['HAVE_PKG_CONFIG'] = env.Detect('pkg-config')
with gem5_scons.Configure(env) as conf:
# On Solaris you need to use libsocket for socket ops
if not conf.CheckLibWithHeader(
[None, 'socket'], 'sys/socket.h', 'C++', 'accept(0,0,0);'):
error("Can't find library with socket calls (e.g. accept()).")
if not conf.CheckLibWithHeader('z', 'zlib.h', 'C++','zlibVersion();'):
error('Did not find needed zlib compression library '
'and/or zlib.h header file.\n'
'Please install zlib and try again.')
if not GetOption('without_tcmalloc'):
with gem5_scons.Configure(env) as conf:
if conf.CheckLib('tcmalloc'):
conf.env.Append(CCFLAGS=conf.env['TCMALLOC_CCFLAGS'])
elif conf.CheckLib('tcmalloc_minimal'):
conf.env.Append(CCFLAGS=conf.env['TCMALLOC_CCFLAGS'])
else:
warning("You can get a 12% performance improvement by "
"installing tcmalloc (libgoogle-perftools-dev package "
"on Ubuntu or RedHat).")
if not GetOption('silent'):
print("Building in", variant_path)
# variant_dir is the tail component of build path, and is used to
# determine the build parameters (e.g., 'X86')
(build_root, variant_dir) = os.path.split(variant_path)
####################################################################
#
# Read and process SConsopts files. These can add new settings which
# affect each variant directory independently.
#
####################################################################
# Register a callback to call after all SConsopts files have been read.
after_sconsopts_callbacks = []
def AfterSConsopts(cb):
after_sconsopts_callbacks.append(cb)
Export('AfterSConsopts')
# Sticky variables get saved in the variables file so they persist from
# one invocation to the next (unless overridden, in which case the new
# value becomes sticky).
sticky_vars = Variables(args=ARGUMENTS)
Export('sticky_vars')
# EXTRAS is special since it affects what SConsopts need to be read.
sticky_vars.Add(('EXTRAS', 'Add extra directories to the compilation', ''))
# Set env variables according to the build directory config.
sticky_vars.files = []
# Variables for $BUILD_ROOT/$VARIANT_DIR are stored in
# $BUILD_ROOT/variables/$VARIANT_DIR so you can nuke
# $BUILD_ROOT/$VARIANT_DIR without losing your variables settings.
current_vars_file = os.path.join(build_root, 'variables', variant_dir)
if isfile(current_vars_file):
sticky_vars.files.append(current_vars_file)
# $BUILD_ROOT/$VARIANT_DIR/gem5.build/variables
gem5_build_vars = os.path.join(gem5_build, 'variables')
build_root_vars = os.path.join(build_root, 'variables', variant_dir)
current_vars_files = [gem5_build_vars, build_root_vars]
existing_vars_files = list(filter(isfile, current_vars_files))
if existing_vars_files:
sticky_vars.files.extend(existing_vars_files)
if not GetOption('silent'):
print("Using saved variables file %s" % current_vars_file)
elif variant_dir in ext_build_dirs:
# Things in ext are built without a variant directory.
continue
print('Using saved variables file(s) %s' %
', '.join(existing_vars_files))
else:
# Variant specific variables file doesn't exist.
# Make sure the directory is there so we can create the file later.
opt_dir = dirname(current_vars_file)
if not isdir(opt_dir):
mkdir(opt_dir)
# Get default build variables from source tree. Variables are
# normally determined by name of $VARIANT_DIR, but can be
# overridden by '--default=' arg on command line.
@@ -685,23 +623,64 @@ for variant_path in variant_paths:
opts_dir = Dir('#build_opts').abspath
if default:
default_vars_files = [
os.path.join(build_root, 'variables', default),
gem5_build_vars,
build_root_vars,
os.path.join(opts_dir, default)
]
else:
default_vars_files = [os.path.join(opts_dir, variant_dir)]
existing_files = list(filter(isfile, default_vars_files))
if existing_files:
default_vars_file = existing_files[0]
existing_default_files = list(filter(isfile, default_vars_files))
if existing_default_files:
default_vars_file = existing_default_files[0]
sticky_vars.files.append(default_vars_file)
print("Variables file %s not found,\n using defaults in %s"
% (current_vars_file, default_vars_file))
print("Variables file(s) %s not found,\n using defaults in %s" %
(' or '.join(current_vars_files), default_vars_file))
else:
error("Cannot find variables file %s or default file(s) %s"
% (current_vars_file, ' or '.join(default_vars_files)))
error("Cannot find variables file(s) %s or default file(s) %s" %
(' or '.join(current_vars_files),
' or '.join(default_vars_files)))
Exit(1)
# Apply current variable settings to env
# Apply current settings for EXTRAS to env.
sticky_vars.Update(env)
# Parse EXTRAS variable to build list of all directories where we're
# look for sources etc. This list is exported as extras_dir_list.
if env['EXTRAS']:
extras_dir_list = makePathListAbsolute(env['EXTRAS'].split(':'))
else:
extras_dir_list = []
Export('extras_dir_list')
# Variables which were determined with Configure.
env['CONF'] = {}
# Walk the tree and execute all SConsopts scripts that wil add to the
# above variables
if GetOption('verbose'):
print("Reading SConsopts")
def trySConsopts(dir):
sconsopts_path = os.path.join(dir, 'SConsopts')
if not isfile(sconsopts_path):
return
if GetOption('verbose'):
print("Reading", sconsopts_path)
SConscript(sconsopts_path, exports={'main': env})
trySConsopts(Dir('#').abspath)
for bdir in [ base_dir ] + extras_dir_list:
if not isdir(bdir):
error("Directory '%s' does not exist." % bdir)
for root, dirs, files in os.walk(bdir):
trySConsopts(root)
# Call any callbacks which the SConsopts files registered.
for cb in after_sconsopts_callbacks:
cb()
# Update env for new variables added by the SConsopts.
sticky_vars.Update(env)
Help('''
@@ -710,21 +689,45 @@ Build variables for {dir}:
'''.format(dir=variant_dir, help=sticky_vars.GenerateHelpText(env)),
append=True)
# Process variable settings.
if env['USE_EFENCE']:
env.Append(LIBS=['efence'])
# If the old vars file exists, delete it to avoid confusion/stale values.
if isfile(build_root_vars):
warning(f'Deleting old variant variables file "{build_root_vars}"')
remove(build_root_vars)
# Save sticky variables back to the gem5.build variant variables file.
sticky_vars.Save(gem5_build_vars, env)
if env['KVM_ISA'] != env['TARGET_ISA']:
env['USE_KVM'] = False
# Pull all the sticky variables into the CONF dict.
env['CONF'].update({key: env[key] for key in sticky_vars.keys()})
# Do this after we save setting back, or else we'll tack on an
# extra 'qdo' every time we run scons.
if env['CONF']['BATCH']:
env['CC'] = env['CONF']['BATCH_CMD'] + ' ' + env['CC']
env['CXX'] = env['CONF']['BATCH_CMD'] + ' ' + env['CXX']
env['AS'] = env['CONF']['BATCH_CMD'] + ' ' + env['AS']
env['AR'] = env['CONF']['BATCH_CMD'] + ' ' + env['AR']
env['RANLIB'] = env['CONF']['BATCH_CMD'] + ' ' + env['RANLIB']
# Cache build files in the supplied directory.
if env['CONF']['M5_BUILD_CACHE']:
print('Using build cache located at', env['CONF']['M5_BUILD_CACHE'])
CacheDir(env['CONF']['M5_BUILD_CACHE'])
# Save sticky variable settings back to current variables file
sticky_vars.Save(current_vars_file, env)
env.Append(CCFLAGS='$CCFLAGS_EXTRA')
env.Append(LINKFLAGS='$LINKFLAGS_EXTRA')
exports=['env', 'gem5py_env']
ext_dir = Dir('#ext').abspath
variant_ext = os.path.join(variant_path, 'ext')
for root, dirs, files in os.walk(ext_dir):
if 'SConscript' in files:
build_dir = os.path.relpath(root, ext_dir)
SConscript(os.path.join(root, 'SConscript'),
variant_dir=os.path.join(variant_ext, build_dir),
exports=exports)
# The src/SConscript file sets up the build rules in 'env' according
# to the configured variables. It returns a list of environments,
# one for each variant build (debug, opt, etc.)

1
build_opts/ARM
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'arm'
CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,O3CPU,MinorCPU'
PROTOCOL = 'CHI'

1
build_opts/ARM_MESI_Three_Level
View File

@@ -2,5 +2,4 @@
# All rights reserved.
TARGET_ISA = 'arm'
CPU_MODELS = 'TimingSimpleCPU,O3CPU'
PROTOCOL = 'MESI_Three_Level'

1
build_opts/ARM_MESI_Three_Level_HTM
View File

@@ -2,5 +2,4 @@
# All rights reserved.
TARGET_ISA = 'arm'
CPU_MODELS = 'TimingSimpleCPU,O3CPU'
PROTOCOL = 'MESI_Three_Level_HTM'

1
build_opts/ARM_MOESI_hammer
View File

@@ -2,5 +2,4 @@
# All rights reserved.
TARGET_ISA = 'arm'
CPU_MODELS = 'TimingSimpleCPU,O3CPU'
PROTOCOL = 'MOESI_hammer'

1
build_opts/GCN3_X86
View File

@@ -2,4 +2,3 @@ PROTOCOL = 'GPU_VIPER'
TARGET_ISA = 'x86'
TARGET_GPU_ISA = 'gcn3'
BUILD_GPU = True
CPU_MODELS = 'AtomicSimpleCPU,O3CPU,TimingSimpleCPU'

1
build_opts/Garnet_standalone
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
CPU_MODELS = ''
PROTOCOL = 'Garnet_standalone'

1
build_opts/MIPS
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'mips'
CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,O3CPU'
PROTOCOL = 'MI_example'

1
build_opts/NULL
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
CPU_MODELS = ''
PROTOCOL='MI_example'

1
build_opts/NULL_MESI_Two_Level
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
CPU_MODELS = ''
PROTOCOL = 'MESI_Two_Level'

1
build_opts/NULL_MOESI_CMP_directory
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
CPU_MODELS = ''
PROTOCOL='MOESI_CMP_directory'

1
build_opts/NULL_MOESI_CMP_token
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
CPU_MODELS = ''
PROTOCOL='MOESI_CMP_token'

1
build_opts/NULL_MOESI_hammer
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
CPU_MODELS = ''
PROTOCOL='MOESI_hammer'

1
build_opts/POWER
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'power'
CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,O3CPU'
PROTOCOL = 'MI_example'

1
build_opts/RISCV
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'riscv'
CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,MinorCPU,O3CPU'
PROTOCOL = 'MI_example'

1
build_opts/SPARC
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'sparc'
CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,O3CPU'
PROTOCOL = 'MI_example'

1
build_opts/VEGA_X86
View File

@@ -2,4 +2,3 @@ PROTOCOL = 'GPU_VIPER'
TARGET_ISA = 'x86'
TARGET_GPU_ISA = 'vega'
BUILD_GPU = True
CPU_MODELS = 'AtomicSimpleCPU,O3CPU,TimingSimpleCPU'

3
build_opts/X86
View File

@@ -1,4 +1,3 @@
TARGET_ISA = 'x86'
CPU_MODELS = 'TimingSimpleCPU,O3CPU,AtomicSimpleCPU'
PROTOCOL = 'MESI_Two_Level'
NUMBER_BITS_PER_SET = '128'
NUMBER_BITS_PER_SET = '128'

1
build_opts/X86_MESI_Two_Level
View File

@@ -1,4 +1,3 @@
TARGET_ISA = 'x86'
CPU_MODELS = 'TimingSimpleCPU,O3CPU,AtomicSimpleCPU'
PROTOCOL = 'MESI_Two_Level'
NUMBER_BITS_PER_SET = '128'

1
build_opts/X86_MI_example
View File

@@ -1,3 +1,2 @@
TARGET_ISA = 'x86'
CPU_MODELS = 'TimingSimpleCPU,O3CPU,AtomicSimpleCPU'
PROTOCOL = 'MI_example'

1
build_opts/X86_MOESI_AMD_Base
View File

@@ -1,3 +1,2 @@
PROTOCOL = 'MOESI_AMD_Base'
TARGET_ISA = 'x86'
CPU_MODELS = 'AtomicSimpleCPU,O3CPU,TimingSimpleCPU'

299
build_tools/cxx_config_cc.py Normal file
View File

@@ -0,0 +1,299 @@
# Copyright 2004-2006 The Regents of The University of Michigan
# Copyright 2010-20013 Advanced Micro Devices, Inc.
# Copyright 2013 Mark D. Hill and David A. Wood
# Copyright 2017-2020 ARM Limited
# Copyright 2021 Google, Inc.
#
# The license below extends only to copyright in the software and shall
# not be construed as granting a license to any other intellectual
# property including but not limited to intellectual property relating
# to a hardware implementation of the functionality of the software
# licensed hereunder. You may use the software subject to the license
# terms below provided that you ensure that this notice is replicated
# unmodified and in its entirety in all distributions of the software,
# modified or unmodified, in source code or in binary form.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
import argparse
import importlib
import os.path
import sys
import importer
from code_formatter import code_formatter
parser = argparse.ArgumentParser()
parser.add_argument('modpath', help='module the simobject belongs to')
parser.add_argument('cxx_config_cc', help='cxx config cc file to generate')
args = parser.parse_args()
basename = os.path.basename(args.cxx_config_cc)
sim_object_name = os.path.splitext(basename)[0]
importer.install()
module = importlib.import_module(args.modpath)
sim_object = getattr(module, sim_object_name)
from m5.params import isSimObjectClass
import m5.params
code = code_formatter()
entry_class = 'CxxConfigDirectoryEntry_%s' % sim_object_name
param_class = '%sCxxConfigParams' % sim_object_name
def cxx_bool(b):
return 'true' if b else 'false'
code('#include "params/%s.hh"' % sim_object_name)
for param in sim_object._params.values():
if isSimObjectClass(param.ptype):
code('#include "%s"' % param.ptype._value_dict['cxx_header'])
code('#include "params/%s.hh"' % param.ptype.__name__)
else:
param.ptype.cxx_ini_predecls(code)
code('''#include "${{sim_object._value_dict['cxx_header']}}"
#include "base/str.hh"
#include "cxx_config/${sim_object_name}.hh"
namespace gem5
{
${param_class}::DirectoryEntry::DirectoryEntry()
{
''')
code.indent()
for param in sim_object._params.values():
is_vector = isinstance(param, m5.params.VectorParamDesc)
is_simobj = issubclass(param.ptype, m5.SimObject.SimObject)
code('parameters["%s"] = new ParamDesc("%s", %s, %s);' %
(param.name, param.name, cxx_bool(is_vector),
cxx_bool(is_simobj)));
for port in sim_object._ports.values():
is_vector = isinstance(port, m5.params.VectorPort)
is_requestor = port.role == 'GEM5 REQUESTOR'
code('ports["%s"] = new PortDesc("%s", %s, %s);' %
(port.name, port.name, cxx_bool(is_vector),
cxx_bool(is_requestor)))
code.dedent()
code('''}
bool
${param_class}::setSimObject(const std::string &name, SimObject *simObject)
{
bool ret = true;
if (false) {
''')
code.indent()
for param in sim_object._params.values():
is_vector = isinstance(param, m5.params.VectorParamDesc)
is_simobj = issubclass(param.ptype, m5.SimObject.SimObject)
if is_simobj and not is_vector:
code('} else if (name == "${{param.name}}") {')
code.indent()
code('this->${{param.name}} = '
'dynamic_cast<${{param.ptype.cxx_type}}>(simObject);')
code('if (simObject && !this->${{param.name}})')
code(' ret = false;')
code.dedent()
code.dedent()
code('''
} else {
ret = false;
}
return ret;
}
bool
${param_class}::setSimObjectVector(const std::string &name,
const std::vector<SimObject *> &simObjects)
{
bool ret = true;
if (false) {
''')
code.indent()
for param in sim_object._params.values():
is_vector = isinstance(param, m5.params.VectorParamDesc)
is_simobj = issubclass(param.ptype, m5.SimObject.SimObject)
if is_simobj and is_vector:
code('} else if (name == "${{param.name}}") {')
code.indent()
code('this->${{param.name}}.clear();')
code('for (auto i = simObjects.begin(); '
'ret && i != simObjects.end(); i ++)')
code('{')
code.indent()
code('${{param.ptype.cxx_type}} object = '
'dynamic_cast<${{param.ptype.cxx_type}}>(*i);')
code('if (*i && !object)')
code(' ret = false;')
code('else')
code(' this->${{param.name}}.push_back(object);')
code.dedent()
code('}')
code.dedent()
code.dedent()
code('''
} else {
ret = false;
}
return ret;
}
void
${param_class}::setName(const std::string &name_)
{
this->name = name_;
}
bool
${param_class}::setParam(const std::string &name,
const std::string &value, const Flags flags)
{
bool ret = true;
if (false) {
''')
code.indent()
for param in sim_object._params.values():
is_vector = isinstance(param, m5.params.VectorParamDesc)
is_simobj = issubclass(param.ptype, m5.SimObject.SimObject)
if not is_simobj and not is_vector:
code('} else if (name == "${{param.name}}") {')
code.indent()
param.ptype.cxx_ini_parse(code,
'value', 'this->%s' % param.name, 'ret =')
code.dedent()
code.dedent()
code('''
} else {
ret = false;
}
return ret;
}
bool
${param_class}::setParamVector(const std::string &name,
const std::vector<std::string> &values, const Flags flags)
{
bool ret = true;
if (false) {
''')
code.indent()
for param in sim_object._params.values():
is_vector = isinstance(param, m5.params.VectorParamDesc)
is_simobj = issubclass(param.ptype, m5.SimObject.SimObject)
if not is_simobj and is_vector:
code('} else if (name == "${{param.name}}") {')
code.indent()
code('${{param.name}}.clear();')
code('for (auto i = values.begin(); '
'ret && i != values.end(); i ++)')
code('{')
code.indent()
code('${{param.ptype.cxx_type}} elem;')
param.ptype.cxx_ini_parse(code,
'*i', 'elem', 'ret =')
code('if (ret)')
code(' this->${{param.name}}.push_back(elem);')
code.dedent()
code('}')
code.dedent()
code.dedent()
code('''
} else {
ret = false;
}
return ret;
}
bool
${param_class}::setPortConnectionCount(const std::string &name,
unsigned int count)
{
bool ret = true;
if (false) {
''')
code.indent()
for port in sim_object._ports.values():
code('} else if (name == "${{port.name}}") {')
code(' this->port_${{port.name}}_connection_count = count;')
code.dedent()
code('''
} else {
ret = false;
}
return ret;
}
SimObject *
${param_class}::simObjectCreate()
{
''')
code.indent()
if hasattr(sim_object, 'abstract') and sim_object.abstract:
code('return nullptr;')
else:
code('return this->create();')
code.dedent()
code('''}
} // namespace gem5
''')
code.write(args.cxx_config_cc)

115
build_tools/cxx_config_hh.py Normal file
View File

@@ -0,0 +1,115 @@
# Copyright 2004-2006 The Regents of The University of Michigan
# Copyright 2010-20013 Advanced Micro Devices, Inc.
# Copyright 2013 Mark D. Hill and David A. Wood
# Copyright 2017-2020 ARM Limited
# Copyright 2021 Google, Inc.
#
# The license below extends only to copyright in the software and shall
# not be construed as granting a license to any other intellectual
# property including but not limited to intellectual property relating
# to a hardware implementation of the functionality of the software
# licensed hereunder. You may use the software subject to the license
# terms below provided that you ensure that this notice is replicated
# unmodified and in its entirety in all distributions of the software,
# modified or unmodified, in source code or in binary form.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
import argparse
import importlib
import os.path
import sys
import importer
from code_formatter import code_formatter
parser = argparse.ArgumentParser()
parser.add_argument('modpath', help='module the simobject belongs to')
parser.add_argument('cxx_config_hh', help='cxx config header file to generate')
args = parser.parse_args()
basename = os.path.basename(args.cxx_config_hh)
sim_object_name = os.path.splitext(basename)[0]
importer.install()
module = importlib.import_module(args.modpath)
sim_object = getattr(module, sim_object_name)
code = code_formatter()
entry_class = 'CxxConfigDirectoryEntry_%s' % sim_object_name
param_class = '%sCxxConfigParams' % sim_object_name
code('''#include "params/${sim_object_name}.hh"
#include "sim/cxx_config.hh"
namespace gem5
{
class ${param_class} : public CxxConfigParams, public ${sim_object_name}Params
{
private:
class DirectoryEntry : public CxxConfigDirectoryEntry
{
public:
DirectoryEntry();
CxxConfigParams *
makeParamsObject() const
{
return new ${param_class};
}
};
static inline AddToConfigDir dirEntry
{"${sim_object_name}", new DirectoryEntry};
public:
bool setSimObject(const std::string &name, SimObject *simObject);
bool setSimObjectVector(const std::string &name,
const std::vector<SimObject *> &simObjects);
void setName(const std::string &name_);
const std::string &getName() { return this->name; }
bool setParam(const std::string &name, const std::string &value,
const Flags flags);
bool setParamVector(const std::string &name,
const std::vector<std::string> &values, const Flags flags);
bool setPortConnectionCount(const std::string &name, unsigned int count);
SimObject *simObjectCreate();
};
} // namespace gem5
''')
code.write(args.cxx_config_hh)

160
build_tools/enum_cc.py Normal file
View File

@@ -0,0 +1,160 @@
# Copyright 2004-2006 The Regents of The University of Michigan
# Copyright 2010-20013 Advanced Micro Devices, Inc.
# Copyright 2013 Mark D. Hill and David A. Wood
# Copyright 2017-2020 ARM Limited
# Copyright 2021 Google, Inc.
#
# The license below extends only to copyright in the software and shall
# not be construed as granting a license to any other intellectual
# property including but not limited to intellectual property relating
# to a hardware implementation of the functionality of the software
# licensed hereunder. You may use the software subject to the license
# terms below provided that you ensure that this notice is replicated
# unmodified and in its entirety in all distributions of the software,
# modified or unmodified, in source code or in binary form.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
import argparse
import importlib
import os.path
import sys
import importer
from code_formatter import code_formatter
parser = argparse.ArgumentParser()
parser.add_argument('modpath', help='module the enum belongs to')
parser.add_argument('enum_cc', help='enum cc file to generate')
parser.add_argument('use_python',
help='whether python is enabled in gem5 (True or False)')
args = parser.parse_args()
use_python = args.use_python.lower()
if use_python == 'true':
use_python = True
elif use_python == 'false':
use_python = False
else:
print(f'Unrecognized "use_python" value {use_python}', file=sys.stderr)
sys.exit(1)
basename = os.path.basename(args.enum_cc)
enum_name = os.path.splitext(basename)[0]
importer.install()
module = importlib.import_module(args.modpath)
enum = getattr(module, enum_name)
code = code_formatter()
wrapper_name = enum.wrapper_name
file_name = enum.__name__
name = enum.__name__ if enum.enum_name is None else enum.enum_name
code('''#include "base/compiler.hh"
#include "enums/$file_name.hh"
namespace gem5
{
''')
if enum.wrapper_is_struct:
code('const char *${wrapper_name}::${name}Strings'
'[Num_${name}] =')
else:
if enum.is_class:
code('''\
const char *${name}Strings[static_cast<int>(${name}::Num_${name})] =
''')
else:
code('''GEM5_DEPRECATED_NAMESPACE(Enums, enums);
namespace enums
{''')
code.indent(1)
code('const char *${name}Strings[Num_${name}] =')
code('{')
code.indent(1)
for val in enum.vals:
code('"$val",')
code.dedent(1)
code('};')
if not enum.wrapper_is_struct and not enum.is_class:
code.dedent(1)
code('} // namespace enums')
code('} // namespace gem5')
if use_python:
name = enum.__name__
enum_name = enum.__name__ if enum.enum_name is None else enum.enum_name
wrapper_name = enum_name if enum.is_class else enum.wrapper_name
code('''#include "pybind11/pybind11.h"
#include "pybind11/stl.h"
#include <sim/init.hh>
namespace py = pybind11;
namespace gem5
{
static void
module_init(py::module_ &m_internal)
{
py::module_ m = m_internal.def_submodule("enum_${name}");
''')
if enum.is_class:
code('py::enum_<${enum_name}>(m, "enum_${name}")')
else:
code('py::enum_<${wrapper_name}::${enum_name}>(m, "enum_${name}")')
code.indent()
code.indent()
for val in enum.vals:
code('.value("${val}", ${wrapper_name}::${val})')
code('.value("Num_${name}", ${wrapper_name}::Num_${enum_name})')
if not enum.is_class:
code('.export_values()')
code(';')
code.dedent()
code('}')
code.dedent()
code('''
static EmbeddedPyBind embed_enum("enum_${name}", module_init);
} // namespace gem5
''')
code.write(args.enum_cc)

116
build_tools/enum_hh.py Normal file
View File

@@ -0,0 +1,116 @@
# Copyright 2004-2006 The Regents of The University of Michigan
# Copyright 2010-20013 Advanced Micro Devices, Inc.
# Copyright 2013 Mark D. Hill and David A. Wood
# Copyright 2017-2020 ARM Limited
# Copyright 2021 Google, Inc.
#
# The license below extends only to copyright in the software and shall
# not be construed as granting a license to any other intellectual
# property including but not limited to intellectual property relating
# to a hardware implementation of the functionality of the software
# licensed hereunder. You may use the software subject to the license
# terms below provided that you ensure that this notice is replicated
# unmodified and in its entirety in all distributions of the software,
# modified or unmodified, in source code or in binary form.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
import argparse
import importlib
import os.path
import sys
import importer
from code_formatter import code_formatter
parser = argparse.ArgumentParser()
parser.add_argument('modpath', help='module the enum belongs to')
parser.add_argument('enum_hh', help='enum header file to generate')
args = parser.parse_args()
basename = os.path.basename(args.enum_hh)
enum_name = os.path.splitext(basename)[0]
importer.install()
module = importlib.import_module(args.modpath)
enum = getattr(module, enum_name)
code = code_formatter()
# Generate C++ class declaration for this enum type.
# Note that we wrap the enum in a class/struct to act as a namespace,
# so that the enum strings can be brief w/o worrying about collisions.
wrapper_name = enum.wrapper_name
wrapper = 'struct' if enum.wrapper_is_struct else 'namespace'
name = enum.__name__ if enum.enum_name is None else enum.enum_name
idem_macro = '__ENUM__%s__%s__' % (wrapper_name, name)
code('''\
#ifndef $idem_macro
#define $idem_macro
namespace gem5
{
''')
if enum.is_class:
code('''\
enum class $name
{
''')
else:
code('''\
$wrapper $wrapper_name {
enum $name
{
''')
code.indent(1)
code.indent(1)
for val in enum.vals:
code('$val = ${{enum.map[val]}},')
code('Num_$name = ${{len(enum.vals)}}')
code.dedent(1)
code('};')
if enum.is_class:
code('''\
extern const char *${name}Strings[static_cast<int>(${name}::Num_${name})];
''')
elif enum.wrapper_is_struct:
code('static const char *${name}Strings[Num_${name}];')
else:
code('extern const char *${name}Strings[Num_${name}];')
if not enum.is_class:
code.dedent(1)
code('}; // $wrapper_name')
code()
code('} // namespace gem5')
code()
code('#endif // $idem_macro')
code.write(args.enum_hh)

213
build_tools/sim_object_param_struct_cc.py
View File

@@ -1,5 +1,18 @@
# Copyright 2004-2006 The Regents of The University of Michigan
# Copyright 2010-20013 Advanced Micro Devices, Inc.
# Copyright 2013 Mark D. Hill and David A. Wood
# Copyright 2017-2020 ARM Limited
# Copyright 2021 Google, Inc.
#
# The license below extends only to copyright in the software and shall
# not be construed as granting a license to any other intellectual
# property including but not limited to intellectual property relating
# to a hardware implementation of the functionality of the software
# licensed hereunder. You may use the software subject to the license
# terms below provided that you ensure that this notice is replicated
# unmodified and in its entirety in all distributions of the software,
# modified or unmodified, in source code or in binary form.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
@@ -57,6 +70,204 @@ importer.install()
module = importlib.import_module(args.modpath)
sim_object = getattr(module, sim_object_name)
from m5.objects.SimObject import PyBindProperty
code = code_formatter()
sim_object.params_create_decl(code, use_python)
py_class_name = sim_object.pybind_class
# The 'local' attribute restricts us to the params declared in
# the object itself, not including inherited params (which
# will also be inherited from the base class's param struct
# here). Sort the params based on their key
params = list(map(lambda k_v: k_v[1],
sorted(sim_object._params.local.items())))
ports = sim_object._ports.local
# only include pybind if python is enabled in the build
if use_python:
code('''#include "pybind11/pybind11.h"
#include "pybind11/stl.h"
#include <type_traits>
#include "base/compiler.hh"
#include "params/$sim_object.hh"
#include "sim/init.hh"
#include "sim/sim_object.hh"
#include "${{sim_object.cxx_header}}"
''')
else:
code('''
#include <type_traits>
#include "base/compiler.hh"
#include "params/$sim_object.hh"
#include "${{sim_object.cxx_header}}"
''')
# only include the python params code if python is enabled.
if use_python:
for param in params:
param.pybind_predecls(code)
code('''namespace py = pybind11;
namespace gem5
{
static void
module_init(py::module_ &m_internal)
{
py::module_ m = m_internal.def_submodule("param_${sim_object}");
''')
code.indent()
if sim_object._base:
code('py::class_<${sim_object}Params, ' \
'${{sim_object._base.type}}Params, ' \
'std::unique_ptr<${{sim_object}}Params, py::nodelete>>(' \
'm, "${sim_object}Params")')
else:
code('py::class_<${sim_object}Params, ' \
'std::unique_ptr<${sim_object}Params, py::nodelete>>(' \
'm, "${sim_object}Params")')
code.indent()
if not hasattr(sim_object, 'abstract') or not sim_object.abstract:
code('.def(py::init<>())')
code('.def("create", &${sim_object}Params::create)')
param_exports = sim_object.cxx_param_exports + [
PyBindProperty(k)
for k, v in sorted(sim_object._params.local.items())
] + [
PyBindProperty(f"port_{port.name}_connection_count")
for port in ports.values()
]
for exp in param_exports:
exp.export(code, f"{sim_object}Params")
code(';')
code()
code.dedent()
bases = []
if 'cxx_base' in sim_object._value_dict:
# If the c++ base class implied by python inheritance was
# overridden, use that value.
if sim_object.cxx_base:
bases.append(sim_object.cxx_base)
elif sim_object._base:
# If not and if there was a SimObject base, use its c++ class
# as this class' base.
bases.append(sim_object._base.cxx_class)
# Add in any extra bases that were requested.
bases.extend(sim_object.cxx_extra_bases)
if bases:
base_str = ", ".join(bases)
code('py::class_<${{sim_object.cxx_class}}, ${base_str}, ' \
'std::unique_ptr<${{sim_object.cxx_class}}, py::nodelete>>(' \
'm, "${py_class_name}")')
else:
code('py::class_<${{sim_object.cxx_class}}, ' \
'std::unique_ptr<${{sim_object.cxx_class}}, py::nodelete>>(' \
'm, "${py_class_name}")')
code.indent()
for exp in sim_object.cxx_exports:
exp.export(code, sim_object.cxx_class)
code(';')
code.dedent()
code()
code.dedent()
code('}')
code()
code('static EmbeddedPyBind '
'embed_obj("${0}", module_init, "${1}");',
sim_object, sim_object._base.type if sim_object._base else "")
code()
code('} // namespace gem5')
# include the create() methods whether or not python is enabled.
if not hasattr(sim_object, 'abstract') or not sim_object.abstract:
if 'type' in sim_object.__dict__:
code('''
namespace gem5
{
namespace
{
/*
* If we can't define a default create() method for this params
* struct because the SimObject doesn't have the right
* constructor, use template magic to make it so we're actually
* defining a create method for this class instead.
*/
class Dummy${sim_object}ParamsClass
{
public:
${{sim_object.cxx_class}} *create() const;
};
template <class CxxClass, class Enable=void>
class Dummy${sim_object}Shunt;
/*
* This version directs to the real Params struct and the
* default behavior of create if there's an appropriate
* constructor.
*/
template <class CxxClass>
class Dummy${sim_object}Shunt<CxxClass, std::enable_if_t<
std::is_constructible_v<CxxClass, const ${sim_object}Params &>>>
{
public:
using Params = ${sim_object}Params;
static ${{sim_object.cxx_class}} *
create(const Params &p)
{
return new CxxClass(p);
}
};
/*
* This version diverts to the DummyParamsClass and a dummy
* implementation of create if the appropriate constructor does
* not exist.
*/
template <class CxxClass>
class Dummy${sim_object}Shunt<CxxClass, std::enable_if_t<
!std::is_constructible_v<CxxClass, const ${sim_object}Params &>>>
{
public:
using Params = Dummy${sim_object}ParamsClass;
static ${{sim_object.cxx_class}} *
create(const Params &p)
{
return nullptr;
}
};
} // anonymous namespace
/*
* An implementation of either the real Params struct's create
* method, or the Dummy one. Either an implementation is
* mandantory since this was shunted off to the dummy class, or
* one is optional which will override this weak version.
*/
[[maybe_unused]] ${{sim_object.cxx_class}} *
Dummy${sim_object}Shunt<${{sim_object.cxx_class}}>::Params::create() const
{
return Dummy${sim_object}Shunt<${{sim_object.cxx_class}}>::create(*this);
}
} // namespace gem5
''')
code.write(args.param_cc)

176
build_tools/sim_object_param_struct_hh.py
View File

@@ -1,5 +1,18 @@
# Copyright 2004-2006 The Regents of The University of Michigan
# Copyright 2010-20013 Advanced Micro Devices, Inc.
# Copyright 2013 Mark D. Hill and David A. Wood
# Copyright 2017-2020 ARM Limited
# Copyright 2021 Google, Inc.
#
# The license below extends only to copyright in the software and shall
# not be construed as granting a license to any other intellectual
# property including but not limited to intellectual property relating
# to a hardware implementation of the functionality of the software
# licensed hereunder. You may use the software subject to the license
# terms below provided that you ensure that this notice is replicated
# unmodified and in its entirety in all distributions of the software,
# modified or unmodified, in source code or in binary form.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
@@ -45,6 +58,167 @@ importer.install()
module = importlib.import_module(args.modpath)
sim_object = getattr(module, sim_object_name)
from m5.objects.SimObject import SimObject
from m5.params import Enum
code = code_formatter()
sim_object.cxx_param_decl(code)
# The 'local' attribute restricts us to the params declared in
# the object itself, not including inherited params (which
# will also be inherited from the base class's param struct
# here). Sort the params based on their key
params = list(map(lambda k_v: k_v[1],
sorted(sim_object._params.local.items())))
ports = sim_object._ports.local
try:
ptypes = [p.ptype for p in params]
except:
print(sim_object)
print(params)
raise
warned_about_nested_templates = False
class CxxClass(object):
def __init__(self, sig, template_params=[]):
# Split the signature into its constituent parts. This could
# potentially be done with regular expressions, but
# it's simple enough to pick appart a class signature
# manually.
parts = sig.split('<', 1)
base = parts[0]
t_args = []
if len(parts) > 1:
# The signature had template arguments.
text = parts[1].rstrip(' \t\n>')
arg = ''
# Keep track of nesting to avoid splitting on ","s embedded
# in the arguments themselves.
depth = 0
for c in text:
if c == '<':
depth = depth + 1
if depth > 0 and not warned_about_nested_templates:
warned_about_nested_templates = True
print('Nested template argument in cxx_class.'
' This feature is largely untested and '
' may not work.')
elif c == '>':
depth = depth - 1
elif c == ',' and depth == 0:
t_args.append(arg.strip())
arg = ''
else:
arg = arg + c
if arg:
t_args.append(arg.strip())
# Split the non-template part on :: boundaries.
class_path = base.split('::')
# The namespaces are everything except the last part of the class path.
self.namespaces = class_path[:-1]
# And the class name is the last part.
self.name = class_path[-1]
self.template_params = template_params
self.template_arguments = []
# Iterate through the template arguments and their values. This
# will likely break if parameter packs are used.
for arg, param in zip(t_args, template_params):
type_keys = ('class', 'typename')
# If a parameter is a type, parse it recursively. Otherwise
# assume it's a constant, and store it verbatim.
if any(param.strip().startswith(kw) for kw in type_keys):
self.template_arguments.append(CxxClass(arg))
else:
self.template_arguments.append(arg)
def declare(self, code):
# First declare any template argument types.
for arg in self.template_arguments:
if isinstance(arg, CxxClass):
arg.declare(code)
# Re-open the target namespace.
for ns in self.namespaces:
code('namespace $ns {')
# If this is a class template...
if self.template_params:
code('template <${{", ".join(self.template_params)}}>')
# The actual class declaration.
code('class ${{self.name}};')
# Close the target namespaces.
for ns in reversed(self.namespaces):
code('} // namespace $ns')
code('''\
#ifndef __PARAMS__${sim_object}__
#define __PARAMS__${sim_object}__
''')
# The base SimObject has a couple of params that get
# automatically set from Python without being declared through
# the normal Param mechanism; we slip them in here (needed
# predecls now, actual declarations below)
if sim_object == SimObject:
code('''#include <string>''')
cxx_class = CxxClass(sim_object._value_dict['cxx_class'],
sim_object._value_dict['cxx_template_params'])
# A forward class declaration is sufficient since we are just
# declaring a pointer.
cxx_class.declare(code)
for param in params:
param.cxx_predecls(code)
for port in ports.values():
port.cxx_predecls(code)
code()
if sim_object._base:
code('#include "params/${{sim_object._base.type}}.hh"')
code()
for ptype in ptypes:
if issubclass(ptype, Enum):
code('#include "enums/${{ptype.__name__}}.hh"')
code()
code('namespace gem5')
code('{')
code('')
# now generate the actual param struct
code("struct ${sim_object}Params")
if sim_object._base:
code(" : public ${{sim_object._base.type}}Params")
code("{")
if not hasattr(sim_object, 'abstract') or not sim_object.abstract:
if 'type' in sim_object.__dict__:
code(" ${{sim_object.cxx_type}} create() const;")
code.indent()
if sim_object == SimObject:
code('''
SimObjectParams() {}
virtual ~SimObjectParams() {}
std::string name;
''')
for param in params:
param.cxx_decl(code)
for port in ports.values():
port.cxx_decl(code)
code.dedent()
code('};')
code()
code('} // namespace gem5')
code()
code('#endif // __PARAMS__${sim_object}__')
code.write(args.param_hh)

5
configs/common/FSConfig.py
View File

@@ -74,8 +74,7 @@ def attach_9p(parent, bus):
viodir = os.path.realpath(os.path.join(m5.options.outdir, '9p'))
viopci.vio.root = os.path.join(viodir, 'share')
viopci.vio.socketPath = os.path.join(viodir, 'socket')
if not os.path.exists(viopci.vio.root):
os.makedirs(viopci.vio.root)
os.makedirs(viopci.vio.root, exist_ok=True)
if os.path.exists(viopci.vio.socketPath):
os.remove(viopci.vio.socketPath)
parent.viopci = viopci
@@ -445,6 +444,8 @@ def connectX86RubySystem(x86_sys):
def makeX86System(mem_mode, numCPUs=1, mdesc=None, workload=None, Ruby=False):
self = System()
self.m5ops_base = 0xffff0000
if workload is None:
workload = X86FsWorkload()
self.workload = workload

12
configs/common/FileSystemConfig.py
View File

@@ -140,7 +140,7 @@ def config_filesystem(system, options = None):
# Set up /sys/devices/system/cpu
cpudir = joinpath(sysdir, 'devices', 'system', 'cpu')
makedirs(cpudir)
makedirs(cpudir, exist_ok=True)
file_append((cpudir, 'online'), '0-%d' % (len(cpus) - 1))
file_append((cpudir, 'possible'), '0-%d' % (len(cpus) - 1))
@@ -168,7 +168,7 @@ def register_node(cpu_list, mem, node_number):
'system', 'node')
nodedir = joinpath(nodebasedir,'node%d' % node_number)
makedirs(nodedir)
makedirs(nodedir, exist_ok=True)
file_append((nodedir, 'cpumap'), hex_mask(cpu_list))
file_append((nodedir, 'meminfo'),
@@ -180,10 +180,8 @@ def register_cpu(physical_package_id, core_siblings,
cpudir = joinpath(m5.options.outdir, 'fs', 'sys', 'devices', 'system',
'cpu', 'cpu%d' % core_id)
if not isdir(joinpath(cpudir, 'topology')):
makedirs(joinpath(cpudir, 'topology'))
if not isdir(joinpath(cpudir, 'cache')):
makedirs(joinpath(cpudir, 'cache'))
makedirs(joinpath(cpudir, 'topology'), exist_ok=True)
makedirs(joinpath(cpudir, 'cache'))
file_append((cpudir, 'online'), '1')
file_append((cpudir, 'topology', 'physical_package_id'),
@@ -204,7 +202,7 @@ def register_cache(level, idu_type, size, line_size, assoc, cpus):
while isdir(joinpath(cachedir, 'index%d' % j)):
j += 1
indexdir = joinpath(cachedir, 'index%d' % j)
makedirs(indexdir)
makedirs(indexdir, exist_ok=True)
file_append((indexdir, 'level'), level)
file_append((indexdir, 'type'), idu_type)

85
configs/common/GPUTLBConfig.py
View File

@@ -34,41 +34,69 @@
import m5
from m5.objects import *
def TLB_constructor(level):
def TLB_constructor(options, level, gpu_ctrl=None, full_system=False):
constructor_call = "X86GPUTLB(size = options.L%(level)dTLBentries, \
assoc = options.L%(level)dTLBassoc, \
hitLatency = options.L%(level)dAccessLatency,\
missLatency2 = options.L%(level)dMissLatency,\
maxOutstandingReqs = options.L%(level)dMaxOutstandingReqs,\
accessDistance = options.L%(level)dAccessDistanceStat,\
clk_domain = SrcClockDomain(\
clock = options.gpu_clock,\
voltage_domain = VoltageDomain(\
voltage = options.gpu_voltage)))" % locals()
return constructor_call
def Coalescer_constructor(level):
constructor_call = "TLBCoalescer(probesPerCycle = \
options.L%(level)dProbesPerCycle, \
coalescingWindow = options.L%(level)dCoalescingWindow,\
disableCoalescing = options.L%(level)dDisableCoalescing,\
if full_system:
constructor_call = "VegaGPUTLB(\
gpu_device = gpu_ctrl, \
size = options.L%(level)dTLBentries, \
assoc = options.L%(level)dTLBassoc, \
hitLatency = options.L%(level)dAccessLatency,\
missLatency1 = options.L%(level)dMissLatency,\
missLatency2 = options.L%(level)dMissLatency,\
maxOutstandingReqs = options.L%(level)dMaxOutstandingReqs,\
clk_domain = SrcClockDomain(\
clock = options.gpu_clock,\
voltage_domain = VoltageDomain(\
voltage = options.gpu_voltage)))" % locals()
else:
constructor_call = "X86GPUTLB(size = options.L%(level)dTLBentries, \
assoc = options.L%(level)dTLBassoc, \
hitLatency = options.L%(level)dAccessLatency,\
missLatency2 = options.L%(level)dMissLatency,\
maxOutstandingReqs = options.L%(level)dMaxOutstandingReqs,\
accessDistance = options.L%(level)dAccessDistanceStat,\
clk_domain = SrcClockDomain(\
clock = options.gpu_clock,\
voltage_domain = VoltageDomain(\
voltage = options.gpu_voltage)))" % locals()
return constructor_call
def Coalescer_constructor(options, level, full_system):
if full_system:
constructor_call = "VegaTLBCoalescer(probesPerCycle = \
options.L%(level)dProbesPerCycle, \
tlb_level = %(level)d ,\
coalescingWindow = options.L%(level)dCoalescingWindow,\
disableCoalescing = options.L%(level)dDisableCoalescing,\
clk_domain = SrcClockDomain(\
clock = options.gpu_clock,\
voltage_domain = VoltageDomain(\
voltage = options.gpu_voltage)))" % locals()
else:
constructor_call = "TLBCoalescer(probesPerCycle = \
options.L%(level)dProbesPerCycle, \
coalescingWindow = options.L%(level)dCoalescingWindow,\
disableCoalescing = options.L%(level)dDisableCoalescing,\
clk_domain = SrcClockDomain(\
clock = options.gpu_clock,\
voltage_domain = VoltageDomain(\
voltage = options.gpu_voltage)))" % locals()
return constructor_call
def create_TLB_Coalescer(options, my_level, my_index, tlb_name,
coalescer_name):
coalescer_name, gpu_ctrl=None, full_system=False):
# arguments: options, TLB level, number of private structures for this
# Level, TLB name and Coalescer name
for i in range(my_index):
tlb_name.append(eval(TLB_constructor(my_level)))
coalescer_name.append(eval(Coalescer_constructor(my_level)))
tlb_name.append(
eval(TLB_constructor(options, my_level, gpu_ctrl, full_system)))
coalescer_name.append(
eval(Coalescer_constructor(options, my_level, full_system)))
def config_tlb_hierarchy(options, system, shader_idx):
def config_tlb_hierarchy(options, system, shader_idx, gpu_ctrl=None,
full_system=False):
n_cu = options.num_compute_units
if options.TLB_config == "perLane":
@@ -121,7 +149,7 @@ def config_tlb_hierarchy(options, system, shader_idx):
options.L1TLBassoc = options.L1TLBentries
# call the constructors for the TLB and the Coalescer
create_TLB_Coalescer(options, level, TLB_index,\
TLB_array, Coalescer_array)
TLB_array, Coalescer_array, gpu_ctrl, full_system)
system_TLB_name = TLB_type['name'] + '_tlb'
system_Coalescer_name = TLB_type['name'] + '_coalescer'
@@ -198,8 +226,17 @@ def config_tlb_hierarchy(options, system, shader_idx):
system.l2_coalescer[0].cpu_side_ports[%d]' % \
(name, index, l2_coalescer_index))
l2_coalescer_index += 1
# L2 <-> L3
system.l2_tlb[0].mem_side_ports[0] = \
system.l3_coalescer[0].cpu_side_ports[0]
# L3 TLB Vega page table walker to memory for full system only
if full_system:
for TLB_type in L3:
name = TLB_type['name']
for index in range(TLB_type['width']):
exec('system._dma_ports.append(system.%s_tlb[%d].walker)' % \
(name, index))
return system

2
configs/common/MemConfig.py
View File

@@ -235,7 +235,7 @@ def config_mem(options, system):
# Create a controller if not sharing a channel with DRAM
# in which case the controller has already been created
if not opt_hybrid_channel:
mem_ctrl = m5.objects.MemCtrl()
mem_ctrl = m5.objects.HeteroMemCtrl()
mem_ctrl.nvm = nvm_intf
mem_ctrls.append(mem_ctrl)

1
configs/common/Simulation.py
View File

@@ -488,6 +488,7 @@ def run(options, root, testsys, cpu_class):
options.indirect_bp_type)
switch_cpus[i].branchPred.indirectBranchPred = \
IndirectBPClass()
switch_cpus[i].createThreads()
# If elastic tracing is enabled attach the elastic trace probe
# to the switch CPUs

2
configs/common/cores/arm/HPI.py
View File

@@ -1379,7 +1379,7 @@ class HPI_L2(Cache):
write_buffers = 16
# prefetcher FIXME
class HPI(MinorCPU):
class HPI(ArmMinorCPU):
# Inherit the doc string from the module to avoid repeating it
# here.
__doc__ = __doc__

2
configs/common/cores/arm/O3_ARM_v7a.py
View File

@@ -99,7 +99,7 @@ class O3_ARM_v7a_BP(BiModeBP):
RASSize = 16
instShiftAmt = 2
class O3_ARM_v7a_3(DerivO3CPU):
class O3_ARM_v7a_3(ArmO3CPU):
LQEntries = 16
SQEntries = 16
LSQDepCheckShift = 0

2
configs/common/cores/arm/ex5_LITTLE.py
View File

@@ -88,7 +88,7 @@ class ex5_LITTLE_FUP(MinorFUPool):
ex5_LITTLE_FP(), ex5_LITTLE_MemFU(),
ex5_LITTLE_MiscFU()]
class ex5_LITTLE(MinorCPU):
class ex5_LITTLE(ArmMinorCPU):
executeFuncUnits = ex5_LITTLE_FUP()
class L1Cache(Cache):

2
configs/common/cores/arm/ex5_big.py
View File

@@ -99,7 +99,7 @@ class ex5_big_BP(BiModeBP):
RASSize = 48
instShiftAmt = 2
class ex5_big(DerivO3CPU):
class ex5_big(ArmO3CPU):
LQEntries = 16
SQEntries = 16
LSQDepCheckShift = 0

3
configs/example/apu_se.py
View File

@@ -161,7 +161,7 @@ parser.add_argument('--fast-forward-pseudo-op', action='store_true',
' m5_switchcpu pseudo-ops will toggle back and forth')
parser.add_argument("--num-hw-queues", type=int, default=10,
help="number of hw queues in packet processor")
parser.add_argument("--reg-alloc-policy", type=str, default="simple",
parser.add_argument("--reg-alloc-policy", type=str, default="dynamic",
help="register allocation policy (simple/dynamic)")
parser.add_argument("--dgpu", action="store_true", default=False,
@@ -544,6 +544,7 @@ if fast_forward:
have_kvm_support = 'BaseKvmCPU' in globals()
if have_kvm_support and buildEnv['TARGET_ISA'] == "x86":
system.vm = KvmVM()
system.m5ops_base = 0xffff0000
for i in range(len(host_cpu.workload)):
host_cpu.workload[i].useArchPT = True
host_cpu.workload[i].kvmInSE = True

2
configs/example/arm/baremetal.py
View File

@@ -141,8 +141,6 @@ def create(args):
system.realview.gic.gicv4 = False
system.highest_el_is_64 = True
system.release.add(ArmExtension('SECURITY'))
system.release.add(ArmExtension('VIRTUALIZATION'))
workload_class = workloads.workload_list.get(args.workload)
system.workload = workload_class(

4
configs/example/arm/devices.py
View File

@@ -176,7 +176,7 @@ class CpuCluster(SubSystem):
class AtomicCluster(CpuCluster):
def __init__(self, system, num_cpus, cpu_clock, cpu_voltage="1.0V"):
cpu_config = [ ObjectList.cpu_list.get("AtomicSimpleCPU"), None,
None, None, None ]
None, None ]
super(AtomicCluster, self).__init__(system, num_cpus, cpu_clock,
cpu_voltage, *cpu_config)
def addL1(self):
@@ -185,7 +185,7 @@ class AtomicCluster(CpuCluster):
class KvmCluster(CpuCluster):
def __init__(self, system, num_cpus, cpu_clock, cpu_voltage="1.0V"):
cpu_config = [ ObjectList.cpu_list.get("ArmV8KvmCPU"), None, None,
None, None ]
None ]
super(KvmCluster, self).__init__(system, num_cpus, cpu_clock,
cpu_voltage, *cpu_config)
def addL1(self):

24
configs/example/arm/ruby_fs.py
View File

@@ -1,4 +1,4 @@
# Copyright (c) 2016-2017, 2020-2021 Arm Limited
# Copyright (c) 2016-2017, 2020-2022 Arm Limited
# All rights reserved.
#
# The license below extends only to copyright in the software and shall
@@ -46,7 +46,7 @@ from common import MemConfig
from common import ObjectList
from common import Options
from common import SysPaths
from common.cores.arm import HPI
from common.cores.arm import O3_ARM_v7a, HPI
from ruby import Ruby
import devices
@@ -57,18 +57,12 @@ default_disk = 'linaro-minimal-aarch64.img'
default_root_device = '/dev/vda1'
# Pre-defined CPU configurations. Each tuple must be ordered as : (cpu_class,
# l1_icache_class, l1_dcache_class, walk_cache_class, l2_Cache_class). Any of
# the cache class may be 'None' if the particular cache is not present.
# Pre-defined CPU configurations.
cpu_types = {
"noncaching" : ( NonCachingSimpleCPU, None, None, None),
"minor" : (MinorCPU,
devices.L1I, devices.L1D,
devices.L2),
"hpi" : ( HPI.HPI,
HPI.HPI_ICache, HPI.HPI_DCache,
HPI.HPI_L2)
"noncaching" : NonCachingSimpleCPU,
"minor" : MinorCPU,
"hpi" : HPI.HPI,
"o3" : O3_ARM_v7a.O3_ARM_v7a_3,
}
def create_cow_image(name):
@@ -100,7 +94,7 @@ def create(args):
print("Error: Bootscript %s does not exist" % args.script)
sys.exit(1)
cpu_class = cpu_types[args.cpu][0]
cpu_class = cpu_types[args.cpu]
mem_mode = cpu_class.memory_mode()
system = devices.ArmRubySystem(args.mem_size,
@@ -115,7 +109,7 @@ def create(args):
devices.CpuCluster(system,
args.num_cpus,
args.cpu_freq, "1.0V",
*cpu_types[args.cpu]),
cpu_class, None, None, None),
]
# Add the PCI devices we need for this system. The base system

23
configs/example/arm/starter_fs.py
View File

@@ -1,4 +1,4 @@
# Copyright (c) 2016-2017, 2020 ARM Limited
# Copyright (c) 2016-2017, 2020, 2022 Arm Limited
# All rights reserved.
#
# The license below extends only to copyright in the software and shall
@@ -50,7 +50,7 @@ m5.util.addToPath('../..')
from common import SysPaths
from common import ObjectList
from common import MemConfig
from common.cores.arm import HPI
from common.cores.arm import O3_ARM_v7a, HPI
import devices
@@ -61,17 +61,19 @@ default_root_device = '/dev/vda1'
# Pre-defined CPU configurations. Each tuple must be ordered as : (cpu_class,
# l1_icache_class, l1_dcache_class, walk_cache_class, l2_Cache_class). Any of
# l1_icache_class, l1_dcache_class, l2_Cache_class). Any of
# the cache class may be 'None' if the particular cache is not present.
cpu_types = {
"atomic" : ( AtomicSimpleCPU, None, None, None),
"atomic" : (AtomicSimpleCPU, None, None, None),
"minor" : (MinorCPU,
devices.L1I, devices.L1D,
devices.L2),
"hpi" : ( HPI.HPI,
HPI.HPI_ICache, HPI.HPI_DCache,
HPI.HPI_L2)
"hpi" : (HPI.HPI,
HPI.HPI_ICache, HPI.HPI_DCache,
HPI.HPI_L2),
"o3" : (O3_ARM_v7a.O3_ARM_v7a_3,
O3_ARM_v7a.O3_ARM_v7a_ICache, O3_ARM_v7a.O3_ARM_v7a_DCache,
O3_ARM_v7a.O3_ARM_v7aL2),
}
def create_cow_image(name):
@@ -148,6 +150,9 @@ def create(args):
os.path.join(m5.options.outdir, 'system.dtb')
system.generateDtb(system.workload.dtb_filename)
if args.initrd:
system.workload.initrd_filename = args.initrd
# Linux boot command flags
kernel_cmd = [
# Tell Linux to use the simulated serial port as a console
@@ -196,6 +201,8 @@ def main():
help="DTB file to load")
parser.add_argument("--kernel", type=str, default=default_kernel,
help="Linux kernel")
parser.add_argument("--initrd", type=str, default=None,
help="initrd/initramfs file to load")
parser.add_argument("--disk-image", type=str,
default=default_disk,
help="Disk to instantiate")

6
configs/example/gem5_library/arm-hello.py
View File

@@ -37,7 +37,7 @@ Usage
```
scons build/ARM/gem5.opt
./build/ARM/gem5.opt configs/gem5_library/arm-hello.py
./build/ARM/gem5.opt configs/example/gem5_library/arm-hello.py
```
"""
@@ -62,7 +62,7 @@ cache_hierarchy = NoCache()
memory = SingleChannelDDR3_1600(size="32MB")
# We use a simple Timing processor with one core.
processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, num_cores=1)
processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, isa=ISA.ARM, num_cores=1)
# The gem5 library simble board which can be used to run simple SE-mode
# simulations.
@@ -88,7 +88,7 @@ board.set_se_binary_workload(
)
# Lastly we run the simulation.
simulator = Simulator(board=board, full_system=False)
simulator = Simulator(board=board)
simulator.run()
print(

152
configs/example/gem5_library/arm-ubuntu-boot-exit.py Normal file
View File

@@ -0,0 +1,152 @@
# Copyright (c) 2022 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
This script shows an example of booting an ARM based full system Ubuntu
disk image using the gem5's standard library. This simulation boots the disk
image using 2 TIMING CPU cores. The simulation ends when the startup is
completed successfully (i.e. when an `m5_exit instruction is reached on
successful boot).
Usage
-----
```
scons build/ARM/gem5.opt -j<NUM_CPUS>
./build/ARM/gem5.opt configs/example/gem5_library/arm-ubuntu-boot-exit.py
```
"""
from gem5.isas import ISA
from m5.objects import ArmDefaultRelease
from gem5.utils.requires import requires
from gem5.resources.resource import Resource
from gem5.simulate.simulator import Simulator
from m5.objects import VExpress_GEM5_Foundation
from gem5.components.boards.arm_board import ArmBoard
from gem5.components.memory import DualChannelDDR4_2400
from gem5.components.processors.cpu_types import CPUTypes
from gem5.components.processors.simple_processor import SimpleProcessor
# This runs a check to ensure the gem5 binary is compiled for ARM.
requires(
isa_required=ISA.ARM,
)
# With ARM, we use simple caches.
from gem5.components.cachehierarchies.classic\
.private_l1_private_l2_cache_hierarchy import (
PrivateL1PrivateL2CacheHierarchy,
)
# Here we setup the parameters of the l1 and l2 caches.
cache_hierarchy = PrivateL1PrivateL2CacheHierarchy(
l1d_size="16kB",
l1i_size="16kB",
l2_size="256kB",
)
# Memory: Dual Channel DDR4 2400 DRAM device.
memory = DualChannelDDR4_2400(size = "2GB")
# Here we setup the processor. We use a simple TIMING processor. The config
# script was also tested with ATOMIC processor.
processor = SimpleProcessor(
cpu_type=CPUTypes.TIMING,
num_cores=2,
)
# The ArmBoard requires a `release` to be specified. This adds all the
# extensions or features to the system. We are setting this to Armv8
# (ArmDefaultRelease) in this example config script. However, the ArmBoard
# currently does not support SECURITY extension.
release = ArmDefaultRelease()
# Removing the SECURITY extension.
release.extensions.remove(release.extensions[2])
# The platform sets up the memory ranges of all the on-chip and off-chip
# devices present on the ARM system.
platform = VExpress_GEM5_Foundation()
# Here we setup the board. The ArmBoard allows for Full-System ARM simulations.
board = ArmBoard(
clk_freq = "3GHz",
processor = processor,
memory = memory,
cache_hierarchy = cache_hierarchy,
release = release,
platform = platform
)
# Here we set the Full System workload.
# The `set_kernel_disk_workload` function on the ArmBoard accepts an ARM
# kernel, a disk image, and, path to the bootloader.
board.set_kernel_disk_workload(
# The ARM kernel will be automatically downloaded to the `~/.cache/gem5`
# directory if not already present. The arm-ubuntu-boot-exit was tested
# with `vmlinux.arm64`
kernel = Resource("arm64-linux-kernel-5.4.49"),
# The ARM ubuntu image will be automatically downloaded to the
# `~/.cache/gem5` directory if not already present.
disk_image = Resource("arm64-ubuntu-18.04-img"),
# We need to specify the path for the bootloader file.
bootloader = Resource("arm64-bootloader-foundation"),
# For the arm64-ubuntu-18.04.img, we need to specify the readfile content
readfile_contents = "m5 exit"
)
# We define the system with the aforementioned system defined.
simulator = Simulator(board = board)
# Once the system successfully boots, it encounters an
# `m5_exit instruction encountered`. We stop the simulation then. When the
# simulation has ended you may inspect `m5out/board.terminal` to see
# the stdout.
simulator.run()

112
configs/example/gem5_library/checkpoints/riscv-hello-restore-checkpoint.py Normal file
View File

@@ -0,0 +1,112 @@
# Copyright (c) 2022 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
This gem5 configuation script creates a simple board sharing the same
structure as the one in
configs/example/gem5_library/checkpoint/riscv-hello-save-checkpoint.py.
This script restores the checkpoint generated by the above script, and
runs the rest of "riscv-hello" binary simulation.
This configuration serves as an example of restoring a checkpoint.
This is setup is the close to the simplest setup possible using the gem5
library. It does not contain any kind of caching, IO, or any non-essential
components.
Usage
-----
```
scons build/RISCV/gem5.opt
./build/RISCV/gem5.opt \
configs/example/gem5_library/checkpoint/riscv-hello-restore-checkpoint.py
```
"""
from gem5.isas import ISA
from gem5.utils.requires import requires
from gem5.resources.resource import Resource
from gem5.components.memory import SingleChannelDDR3_1600
from gem5.components.processors.cpu_types import CPUTypes
from gem5.components.boards.simple_board import SimpleBoard
from gem5.components.cachehierarchies.classic.no_cache import NoCache
from gem5.components.processors.simple_processor import SimpleProcessor
from gem5.simulate.simulator import Simulator
# This check ensures the gem5 binary is compiled to the RISCV ISA target.
# If not, an exception will be thrown.
requires(isa_required=ISA.RISCV)
# In this setup we don't have a cache. `NoCache` can be used for such setups.
cache_hierarchy = NoCache()
# We use a single channel DDR3_1600 memory system
memory = SingleChannelDDR3_1600(size="32MB")
# We use a simple Timing processor with one core.
processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, isa=ISA.RISCV,
num_cores=1)
# The gem5 library simble board which can be used to run simple SE-mode
# simulations.
board = SimpleBoard(
clk_freq="3GHz",
processor=processor,
memory=memory,
cache_hierarchy=cache_hierarchy,
)
# Here we set the workload. In this case we want to run a simple "Hello World!"
# program compiled to the RISCV ISA. The `Resource` class will automatically
# download the binary from the gem5 Resources cloud bucket if it's not already
# present.
board.set_se_binary_workload(
# the workload should be the same as the save-checkpoint script
Resource("riscv-hello")
)
# Getting the pre-taken checkpoint from gem5-resources. This checkpoint
# was taken from running this gem5 configuration script,
# configs/example/gem5_library/checkpoints/riscv-hello-save-checkpoint.py
checkpoint_resource = Resource("riscv-hello-example-checkpoint")
# Now we restore the checkpoint by passing the path to the checkpoint to
# the Simulator object. The checkpoint_path could be a string containing
# the path to the checkpoint folder. However, here, we use gem5 resources
# to automatically download the checkpoint folder, and use .get_local_path()
# to obtain the path to that folder.
checkpoint_path = checkpoint_resource.get_local_path()
print("Restore a checkpoint at", checkpoint_path)
simulator = Simulator(board=board, full_system=False,
checkpoint_path=checkpoint_path)
simulator.run()
print(
"Exiting @ tick {} because {}.".format(
simulator.get_current_tick(),
simulator.get_last_exit_event_cause(),
)
)

108
configs/example/gem5_library/checkpoints/riscv-hello-save-checkpoint.py Normal file
View File

@@ -0,0 +1,108 @@
# Copyright (c) 2022 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
This gem5 configuation script creates a simple board to run the first
10^6 ticks of "riscv-hello" binary simulation and saves a checkpoint.
This configuration serves as an example of taking a checkpoint.
This is setup is the close to the simplest setup possible using the gem5
library. It does not contain any kind of caching, IO, or any non-essential
components.
Usage
-----
```
scons build/RISCV/gem5.opt
./build/RISCV/gem5.opt \
configs/example/gem5_library/checkpoint/riscv-hello-save-checkpoint.py
```
"""
from gem5.isas import ISA
from gem5.utils.requires import requires
from gem5.resources.resource import Resource
from gem5.components.memory import SingleChannelDDR3_1600
from gem5.components.processors.cpu_types import CPUTypes
from gem5.components.boards.simple_board import SimpleBoard
from gem5.components.cachehierarchies.classic.no_cache import NoCache
from gem5.components.processors.simple_processor import SimpleProcessor
from gem5.simulate.simulator import Simulator
# This check ensures the gem5 binary is compiled to the RISCV ISA target.
# If not, an exception will be thrown.
requires(isa_required=ISA.RISCV)
# In this setup we don't have a cache. `NoCache` can be used for such setups.
cache_hierarchy = NoCache()
# We use a single channel DDR3_1600 memory system
memory = SingleChannelDDR3_1600(size="32MB")
# We use a simple Timing processor with one core.
processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, isa=ISA.RISCV,
num_cores=1)
# The gem5 library simble board which can be used to run simple SE-mode
# simulations.
board = SimpleBoard(
clk_freq="3GHz",
processor=processor,
memory=memory,
cache_hierarchy=cache_hierarchy,
)
# Here we set the workload. In this case we want to run a simple "Hello World!"
# program compiled to the RISCV ISA. The `Resource` class will automatically
# download the binary from the gem5 Resources cloud bucket if it's not already
# present.
board.set_se_binary_workload(
# The `Resource` class reads the `resources.json` file from the gem5
# resources repository:
# https://gem5.googlesource.com/public/gem5-resource.
# Any resource specified in this file will be automatically retrieved.
# At the time of writing, this file is a WIP and does not contain all
# resources. Jira ticket: https://gem5.atlassian.net/browse/GEM5-1096
Resource("riscv-hello")
)
# Lastly we run the simulation.
max_ticks = 10**6
simulator = Simulator(board=board, full_system=False)
simulator.run(max_ticks = max_ticks)
print(
"Exiting @ tick {} because {}.".format(
simulator.get_current_tick(),
simulator.get_last_exit_event_cause(),
)
)
checkpoint_path = "riscv-hello-checkpoint/"
print("Taking a checkpoint at", checkpoint_path)
simulator.save_checkpoint(checkpoint_path)
print("Done taking a checkpoint")

4
configs/example/gem5_library/riscv-fs.py
View File

@@ -66,7 +66,9 @@ cache_hierarchy = PrivateL1PrivateL2CacheHierarchy(
memory = SingleChannelDDR3_1600()
# Setup a single core Processor.
processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, num_cores=1)
processor = SimpleProcessor(
cpu_type=CPUTypes.TIMING, isa=ISA.RISCV, num_cores=1
)
# Setup the board.
board = RiscvBoard(

35
configs/example/gem5_library/riscv-ubuntu-run.py
View File

@@ -53,6 +53,7 @@ from gem5.components.processors.cpu_types import CPUTypes
from gem5.isas import ISA
from gem5.coherence_protocol import CoherenceProtocol
from gem5.resources.resource import Resource
from gem5.simulate.simulator import Simulator
# This runs a check to ensure the gem5 binary is compiled for RISCV.
@@ -79,6 +80,7 @@ memory = DualChannelDDR4_2400(size = "3GB")
# Here we setup the processor. We use a simple processor.
processor = SimpleProcessor(
cpu_type=CPUTypes.TIMING,
isa=ISA.RISCV,
num_cores=2,
)
@@ -113,34 +115,5 @@ board.set_kernel_disk_workload(
),
)
root = Root(full_system=True, system=board)
m5.instantiate()
# We simulate the system till we encounter `m5_exit instruction encountered`.
exit_event = m5.simulate()
# We check whether the simulation ended with `m5_exit instruction encountered`
if exit_event.getCause() == "m5_exit instruction encountered":
# We acknowledge the user that the boot was successful.
print("Successfully completed booting!")
else:
# `m5_exit instruction encountered` was never encountered. We exit the
# program unsuccessfully.
print("The startup was not completed successfully!",)
print(
"Exiting @ tick {} because {}."\
.format(m5.curTick(), exit_event.getCause())
)
exit(-1)
# We are done with the simulation. We exit the program now.
print(
"Exiting @ tick {} because {}."\
.format(m5.curTick(), exit_event.getCause())
)
simulator = Simulator(board=board)
simulator.run()

1
configs/example/gem5_library/x86-gapbs-benchmarks.py
View File

@@ -147,6 +147,7 @@ memory = DualChannelDDR4_2400(size="3GB")
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
isa=ISA.X86,
num_cores=2,
)

1
configs/example/gem5_library/x86-npb-benchmarks.py
View File

@@ -164,6 +164,7 @@ memory = DualChannelDDR4_2400(size = "3GB")
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
isa=ISA.X86,
num_cores=2,
)

1
configs/example/gem5_library/x86-parsec-benchmarks.py
View File

@@ -137,6 +137,7 @@ memory = DualChannelDDR4_2400(size = "3GB")
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
isa=ISA.X86,
num_cores=2,
)

1
configs/example/gem5_library/x86-spec-cpu2006-benchmarks.py
View File

@@ -187,6 +187,7 @@ memory = DualChannelDDR4_2400(size = "3GB")
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
isa=ISA.X86,
num_cores=2,
)

1
configs/example/gem5_library/x86-spec-cpu2017-benchmarks.py
View File

@@ -193,6 +193,7 @@ memory = DualChannelDDR4_2400(size = "3GB")
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
isa=ISA.X86,
num_cores=2,
)

1
configs/example/gem5_library/x86-ubuntu-run-with-kvm.py
View File

@@ -89,6 +89,7 @@ memory = SingleChannelDDR3_1600(size="3GB")
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
isa=ISA.X86,
num_cores=2,
)

107
configs/example/gpufs/DisjointNetwork.py Normal file
View File

@@ -0,0 +1,107 @@
# Copyright (c) 2021 Advanced Micro Devices, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its
# contributors may be used to endorse or promote products derived from this
# software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
from m5.objects import *
from m5.util import fatal
from importlib import *
from network import Network
class DisjointSimple(SimpleNetwork):
def __init__(self, ruby_system):
super(DisjointSimple, self).__init__()
self.netifs = []
self.routers = []
self.int_links = []
self.ext_links = []
self.ruby_system = ruby_system
def connectCPU(self, opts, controllers):
# Setup parameters for makeTopology call for CPU network
topo_module = import_module("topologies.%s" % opts.cpu_topology)
topo_class = getattr(topo_module, opts.cpu_topology)
_topo = topo_class(controllers)
_topo.makeTopology(opts, self, SimpleIntLink,
SimpleExtLink, Switch)
self.initSimple(opts, self.int_links, self.ext_links)
def connectGPU(self, opts, controllers):
# Setup parameters for makeTopology call for GPU network
topo_module = import_module("topologies.%s" % opts.gpu_topology)
topo_class = getattr(topo_module, opts.gpu_topology)
_topo = topo_class(controllers)
_topo.makeTopology(opts, self, SimpleIntLink,
SimpleExtLink, Switch)
self.initSimple(opts, self.int_links, self.ext_links)
def initSimple(self, opts, int_links, ext_links):
# Attach links to network
self.int_links = int_links
self.ext_links = ext_links
self.setup_buffers()
class DisjointGarnet(GarnetNetwork):
def __init__(self, ruby_system):
super(DisjointGarnet, self).__init__()
self.netifs = []
self.ruby_system = ruby_system
def connectCPU(self, opts, controllers):
# Setup parameters for makeTopology call for CPU network
topo_module = import_module("topologies.%s" % opts.cpu_topology)
topo_class = getattr(topo_module, opts.cpu_topology)
_topo = topo_class(controllers)
_topo.makeTopology(opts, self, GarnetIntLink,
GarnetExtLink, GarnetRouter)
Network.init_network(opts, self, GarnetNetworkInterface)
def connectGPU(self, opts, controllers):
# Setup parameters for makeTopology call
topo_module = import_module("topologies.%s" % opts.gpu_topology)
topo_class = getattr(topo_module, opts.gpu_topology)
_topo = topo_class(controllers)
_topo.makeTopology(opts, self, GarnetIntLink,
GarnetExtLink, GarnetRouter)
Network.init_network(opts, self, GarnetNetworkInterface)

196
configs/example/gpufs/Disjoint_VIPER.py Normal file
View File

@@ -0,0 +1,196 @@
# Copyright (c) 2021 Advanced Micro Devices, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its
# contributors may be used to endorse or promote products derived from this
# software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
from m5.defines import buildEnv
from m5.objects import *
from m5.util import fatal
from example.gpufs.DisjointNetwork import *
from ruby.GPU_VIPER import *
from ruby import Ruby
class DummySystem():
def __init__(self, mem_ranges):
self.mem_ctrls = []
self.mem_ranges = mem_ranges
class Disjoint_VIPER(RubySystem):
def __init__(self):
if buildEnv['PROTOCOL'] != "GPU_VIPER":
fatal("This ruby config only supports the GPU_VIPER protocol")
super(Disjoint_VIPER, self).__init__()
def create(self, options, system, piobus, dma_devices):
# Disjoint network topology
if "garnet" in options.network:
self.network_cpu = DisjointGarnet(self)
self.network_gpu = DisjointGarnet(self)
else:
self.network_cpu = DisjointSimple(self)
self.network_gpu = DisjointSimple(self)
# Construct CPU controllers
cpu_dir_nodes = \
construct_dirs(options, system, self, self.network_cpu)
(cp_sequencers, cp_cntrl_nodes) = \
construct_corepairs(options, system, self, self.network_cpu)
# Construct GPU controllers
(tcp_sequencers, tcp_cntrl_nodes) = \
construct_tcps(options, system, self, self.network_gpu)
(sqc_sequencers, sqc_cntrl_nodes) = \
construct_sqcs(options, system, self, self.network_gpu)
(scalar_sequencers, scalar_cntrl_nodes) = \
construct_scalars(options, system, self, self.network_gpu)
tcc_cntrl_nodes = \
construct_tccs(options, system, self, self.network_gpu)
# Construct CPU memories
Ruby.setup_memory_controllers(system, self, cpu_dir_nodes, options)
# Construct GPU memories
(gpu_dir_nodes, gpu_mem_ctrls) = \
construct_gpudirs(options, system, self, self.network_gpu)
# Configure the directories based on which network they are in
for cpu_dir_node in cpu_dir_nodes:
cpu_dir_node.CPUonly = True
cpu_dir_node.GPUonly = False
for gpu_dir_node in gpu_dir_nodes:
gpu_dir_node.CPUonly = False
gpu_dir_node.GPUonly = True
# Set access backing store if specified
if options.access_backing_store:
self.access_backing_store = True
# Assign the memory controllers to the system
cpu_abstract_mems = []
for mem_ctrl in system.mem_ctrls:
cpu_abstract_mems.append(mem_ctrl.dram)
system.memories = cpu_abstract_mems
gpu_abstract_mems = []
for mem_ctrl in gpu_mem_ctrls:
gpu_abstract_mems.append(mem_ctrl.dram)
system.pc.south_bridge.gpu.memories = gpu_abstract_mems
# Setup DMA controllers
gpu_dma_types = ["VegaPagetableWalker", "AMDGPUMemoryManager"]
cpu_dma_ctrls = []
gpu_dma_ctrls = []
dma_cntrls = []
for i, dma_device in enumerate(dma_devices):
dma_seq = DMASequencer(version=i, ruby_system=self)
dma_cntrl = DMA_Controller(version=i, dma_sequencer=dma_seq,
ruby_system=self)
# Handle inconsistently named ports on various DMA devices:
if not hasattr(dma_device, 'type'):
# IDE doesn't have a .type but seems like everything else does.
dma_seq.in_ports = dma_device
elif dma_device.type in gpu_dma_types:
dma_seq.in_ports = dma_device.port
else:
dma_seq.in_ports = dma_device.dma
if hasattr(dma_device, 'type') and \
dma_device.type in gpu_dma_types:
dma_cntrl.requestToDir = MessageBuffer(buffer_size=0)
dma_cntrl.requestToDir.out_port = self.network_gpu.in_port
dma_cntrl.responseFromDir = MessageBuffer(buffer_size=0)
dma_cntrl.responseFromDir.in_port = self.network_gpu.out_port
dma_cntrl.mandatoryQueue = MessageBuffer(buffer_size = 0)
gpu_dma_ctrls.append(dma_cntrl)
else:
dma_cntrl.requestToDir = MessageBuffer(buffer_size=0)
dma_cntrl.requestToDir.out_port = self.network_cpu.in_port
dma_cntrl.responseFromDir = MessageBuffer(buffer_size=0)
dma_cntrl.responseFromDir.in_port = self.network_cpu.out_port
dma_cntrl.mandatoryQueue = MessageBuffer(buffer_size = 0)
cpu_dma_ctrls.append(dma_cntrl)
dma_cntrls.append(dma_cntrl)
system.dma_cntrls = dma_cntrls
# Collect CPU and GPU controllers into seperate lists
cpu_cntrls = cpu_dir_nodes + cp_cntrl_nodes + cpu_dma_ctrls
gpu_cntrls = tcp_cntrl_nodes + sqc_cntrl_nodes + \
scalar_cntrl_nodes + tcc_cntrl_nodes + gpu_dma_ctrls + \
gpu_dir_nodes
# Setup number of vnets
self.number_of_virtual_networks = 11
self.network_cpu.number_of_virtual_networks = 11
self.network_gpu.number_of_virtual_networks = 11
# Set up the disjoint topology
self.network_cpu.connectCPU(options, cpu_cntrls)
self.network_gpu.connectGPU(options, gpu_cntrls)
# Create port proxy for connecting system port. System port is used
# for loading from outside guest, e.g., binaries like vmlinux.
system.sys_port_proxy = RubyPortProxy(ruby_system = self)
system.sys_port_proxy.pio_request_port = piobus.cpu_side_ports
system.system_port = system.sys_port_proxy.in_ports
# Only CPU sequencers connect to PIO bus. This acts as the "default"
# destination for unknown address ranges. PCIe requests fall under
# this category.
for i in range(len(cp_sequencers)):
cp_sequencers[i].pio_request_port = piobus.cpu_side_ports
cp_sequencers[i].mem_request_port = piobus.cpu_side_ports
# The CorePairs in MOESI_AMD_Base round up when constructing
# sequencers, but if the CPU does not exit there would be no
# sequencer to send a range change, leading to assert.
if i < options.num_cpus:
cp_sequencers[i].pio_response_port = piobus.mem_side_ports
# Setup ruby port. Both CPU and GPU are actually connected here.
all_sequencers = cp_sequencers + tcp_sequencers + \
sqc_sequencers + scalar_sequencers
self._cpu_ports = all_sequencers
self.num_of_sequencers = len(all_sequencers)

131
configs/example/gpufs/hip_cookbook.py Normal file
View File

@@ -0,0 +1,131 @@
# Copyright (c) 2022 Advanced Micro Devices, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its
# contributors may be used to endorse or promote products derived from this
# software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
import m5
import runfs
import tempfile
import argparse
import sys
import os
from amd import AmdGPUOptions
from common import Options
from common import GPUTLBOptions
from ruby import Ruby
cookbook_runscript = '''\
export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
export HSA_ENABLE_SDMA=0
dmesg -n3
dd if=/root/roms/vega10.rom of=/dev/mem bs=1k seek=768 count=128
if [ ! -f /lib/modules/`uname -r`/updates/dkms/amdgpu.ko ]; then
echo "ERROR: Missing DKMS package for kernel `uname -r`. Exiting gem5."
/sbin/m5 exit
fi
modprobe -v amdgpu ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
echo "Running {}"
cd /opt/rocm/hip/samples/2_Cookbook/{}/
make clean
make
/sbin/m5 exit
'''
def addCookbookOptions(parser):
parser.add_argument("-a", "--app", default=None,
choices=['0_MatrixTranspose',
'1_hipEvent',
'3_shared_memory',
'4_shfl',
'5_2dshfl',
'6_dynamic_shared',
'7_streams',
'8_peer2peer',
'9_unroll',
'10_inline_asm',
'11_texture_driver',
'13_occupancy',
'14_gpu_arch',
'15_static_library'],
help="GPU application to run")
parser.add_argument("-o", "--opts", default="",
help="GPU application arguments")
if __name__ == "__m5_main__":
parser = argparse.ArgumentParser()
runfs.addRunFSOptions(parser)
Options.addCommonOptions(parser)
AmdGPUOptions.addAmdGPUOptions(parser)
Ruby.define_options(parser)
GPUTLBOptions.tlb_options(parser)
addCookbookOptions(parser)
# Parse now so we can override options
args = parser.parse_args()
# Create temp script to run application
if args.app is None:
print("No application given. Use %s -a <app>" % sys.argv[0])
sys.exit(1)
elif args.kernel is None:
print("No kernel path given. Use %s --kernel <vmlinux>" % sys.argv[0])
sys.exit(1)
elif args.disk_image is None:
print("No disk path given. Use %s --disk-image <linux>" % sys.argv[0])
sys.exit(1)
elif args.gpu_mmio_trace is None:
print("No MMIO trace path. Use %s --gpu-mmio-trace <path>"
% sys.argv[0])
sys.exit(1)
_, tempRunscript = tempfile.mkstemp()
with open(tempRunscript, 'w') as b64file:
runscriptStr = cookbook_runscript.format(args.app, args.app)
b64file.write(runscriptStr)
if args.second_disk == None:
args.second_disk = args.disk_image
# Defaults for Vega10
args.ruby = True
args.cpu_type = 'X86KvmCPU'
args.num_cpus = 1
args.mem_size = '3GB'
args.dgpu = True
args.dgpu_mem_size = '16GB'
args.dgpu_start = '0GB'
args.checkpoint_restore = 0
args.disjoint = True
args.timing_gpu = True
args.script = tempRunscript
args.dgpu_xor_low_bit = 0
print(args.disk_image)
# Run gem5
runfs.runGpuFSSystem(args)

139
configs/example/gpufs/hip_rodinia.py Normal file
View File

@@ -0,0 +1,139 @@
# Copyright (c) 2022 Advanced Micro Devices, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its
# contributors may be used to endorse or promote products derived from this
# software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
import m5
import runfs
import base64
import tempfile
import argparse
import sys
import os
from amd import AmdGPUOptions
from common import Options
from common import GPUTLBOptions
from ruby import Ruby
rodinia_runscript = '''\
export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
export HSA_ENABLE_SDMA=0
dmesg -n3
dd if=/root/roms/vega10.rom of=/dev/mem bs=1k seek=768 count=128
if [ ! -f /lib/modules/`uname -r`/updates/dkms/amdgpu.ko ]; then
echo "ERROR: Missing DKMS package for kernel `uname -r`. Exiting gem5."
/sbin/m5 exit
fi
modprobe -v amdgpu ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
echo "Running {}"
cd /home/gem5/HIP-Examples/rodinia_3.0/hip/{}/
make clean
make
make test
/sbin/m5 exit
'''
def addRodiniaOptions(parser):
parser.add_argument("-a", "--app", default=None,
choices=['b+tree',
'backprop',
'bfs',
'cfd',
'dwt2d',
'gaussian',
'heartwall',
'hotspot',
'hybridsort',
'kmeans',
'lavaMD',
'leukocyte',
'lud',
'myocyte',
'nn',
'nw',
'particlefilter',
'pathfinder',
'srad',
'streamcluster'],
help="GPU application to run")
parser.add_argument("-o", "--opts", default="",
help="GPU application arguments")
if __name__ == "__m5_main__":
parser = argparse.ArgumentParser()
runfs.addRunFSOptions(parser)
Options.addCommonOptions(parser)
AmdGPUOptions.addAmdGPUOptions(parser)
Ruby.define_options(parser)
GPUTLBOptions.tlb_options(parser)
addRodiniaOptions(parser)
# Parse now so we can override options
args = parser.parse_args()
# Create temp script to run application
if args.app is None:
print("No application given. Use %s -a <app>" % sys.argv[0])
sys.exit(1)
elif args.kernel is None:
print("No kernel path given. Use %s --kernel <vmlinux>" % sys.argv[0])
sys.exit(1)
elif args.disk_image is None:
print("No disk path given. Use %s --disk-image <linux>" % sys.argv[0])
sys.exit(1)
elif args.gpu_mmio_trace is None:
print("No MMIO trace path. Use %s --gpu-mmio-trace <path>"
% sys.argv[0])
sys.exit(1)
_, tempRunscript = tempfile.mkstemp()
with open(tempRunscript, 'w') as b64file:
runscriptStr = rodinia_runscript.format(args.app, args.app)
b64file.write(runscriptStr)
if args.second_disk == None:
args.second_disk = args.disk_image
# Defaults for Vega10
args.ruby = True
args.cpu_type = 'X86KvmCPU'
args.num_cpus = 1
args.mem_size = '3GB'
args.dgpu = True
args.dgpu_mem_size = '16GB'
args.dgpu_start = '0GB'
args.checkpoint_restore = 0
args.disjoint = True
args.timing_gpu = True
args.script = tempRunscript
args.dgpu_xor_low_bit = 0
print(args.disk_image)
# Run gem5
runfs.runGpuFSSystem(args)

129
configs/example/gpufs/hip_samples.py Normal file
View File

@@ -0,0 +1,129 @@
# Copyright (c) 2022 Advanced Micro Devices, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its
# contributors may be used to endorse or promote products derived from this
# software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
import m5
import runfs
import tempfile
import argparse
import sys
import os
from amd import AmdGPUOptions
from common import Options
from common import GPUTLBOptions
from ruby import Ruby
samples_runscript = '''\
export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
export HSA_ENABLE_SDMA=0
dmesg -n3
dd if=/root/roms/vega10.rom of=/dev/mem bs=1k seek=768 count=128
if [ ! -f /lib/modules/`uname -r`/updates/dkms/amdgpu.ko ]; then
echo "ERROR: Missing DKMS package for kernel `uname -r`. Exiting gem5."
/sbin/m5 exit
fi
modprobe -v amdgpu ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
echo "Running {}"
cd /home/gem5/HIP-Examples/HIP-Examples-Applications/{}/
make clean
make
/sbin/m5 exit
'''
def addSamplesOptions(parser):
parser.add_argument("-a", "--app", default=None,
choices=['BinomialOption',
'BitonicSort',
'FastWalshTransform',
'FloydWarshall',
'HelloWorld',
'Histogram',
'MatrixMultiplication',
'PrefixSum',
'RecursiveGaussian',
'SimpleConvolution',
'dct',
'dwtHaar1D'],
help="GPU application to run")
parser.add_argument("-o", "--opts", default="",
help="GPU application arguments")
if __name__ == "__m5_main__":
parser = argparse.ArgumentParser()
runfs.addRunFSOptions(parser)
Options.addCommonOptions(parser)
AmdGPUOptions.addAmdGPUOptions(parser)
Ruby.define_options(parser)
GPUTLBOptions.tlb_options(parser)
addSamplesOptions(parser)
# Parse now so we can override options
args = parser.parse_args()
# Create temp script to run application
if args.app is None:
print("No application given. Use %s -a <app>" % sys.argv[0])
sys.exit(1)
elif args.kernel is None:
print("No kernel path given. Use %s --kernel <vmlinux>" % sys.argv[0])
sys.exit(1)
elif args.disk_image is None:
print("No disk path given. Use %s --disk-image <linux>" % sys.argv[0])
sys.exit(1)
elif args.gpu_mmio_trace is None:
print("No MMIO trace path. Use %s --gpu-mmio-trace <path>"
% sys.argv[0])
sys.exit(1)
_, tempRunscript = tempfile.mkstemp()
with open(tempRunscript, 'w') as b64file:
runscriptStr = samples_runscript.format(args.app, args.app)
b64file.write(runscriptStr)
if args.second_disk == None:
args.second_disk = args.disk_image
# Defaults for Vega10
args.ruby = True
args.cpu_type = 'X86KvmCPU'
args.num_cpus = 1
args.mem_size = '3GB'
args.dgpu = True
args.dgpu_mem_size = '16GB'
args.dgpu_start = '0GB'
args.checkpoint_restore = 0
args.disjoint = True
args.timing_gpu = True
args.script = tempRunscript
args.dgpu_xor_low_bit = 0
print(args.disk_image)
# Run gem5
runfs.runGpuFSSystem(args)

15
configs/example/gpufs/runfs.py
View File

@@ -71,7 +71,20 @@ def addRunFSOptions(parser):
help="Take a checkpoint before driver sends MMIOs. "
"This is used to switch out of KVM mode and into "
"timing mode required to read the VGA ROM on boot.")
parser.add_argument("--cpu-topology", type=str, default="Crossbar",
help="Network topology to use for CPU side. "
"Check configs/topologies for complete set")
parser.add_argument("--gpu-topology", type=str, default="Crossbar",
help="Network topology to use for GPU side. "
"Check configs/topologies for complete set")
parser.add_argument("--dgpu-mem-size", action="store", type=str,
default="16GB", help="Specify the dGPU physical memory"
" size")
parser.add_argument("--dgpu-num-dirs", type=int, default=1, help="Set "
"the number of dGPU directories (memory controllers")
parser.add_argument("--dgpu-mem-type", default="HBM_1000_4H_1x128",
choices=ObjectList.mem_list.get_names(),
help="type of memory to use")
def runGpuFSSystem(args):
'''

90
configs/example/gpufs/system/system.py
View File

@@ -33,9 +33,12 @@ from m5.util import panic
from common.Benchmarks import *
from common.FSConfig import *
from common import GPUTLBConfig
from common import Simulation
from ruby import Ruby
from example.gpufs.Disjoint_VIPER import *
def makeGpuFSSystem(args):
# Boot options are standard gem5 options plus:
# - Framebuffer device emulation 0 to reduce driver code paths.
@@ -54,7 +57,10 @@ def makeGpuFSSystem(args):
# Use the common FSConfig to setup a Linux X86 System
(TestCPUClass, test_mem_mode, FutureClass) = Simulation.setCPUClass(args)
bm = SysConfig(disks=[args.disk_image], mem=args.mem_size)
disks = [args.disk_image]
if args.second_disk is not None:
disks.extend([args.second_disk])
bm = SysConfig(disks=disks, mem=args.mem_size)
system = makeLinuxX86System(test_mem_mode, args.num_cpus, bm, True,
cmdline=cmdline)
system.workload.object_file = binary(args.kernel)
@@ -77,45 +83,102 @@ def makeGpuFSSystem(args):
system.shadow_rom_ranges = [AddrRange(0xc0000, size = Addr('128kB'))]
# Create specified number of CPUs. GPUFS really only needs one.
system.cpu = [TestCPUClass(clk_domain=system.cpu_clk_domain, cpu_id=i)
system.cpu = [X86KvmCPU(clk_domain=system.cpu_clk_domain, cpu_id=i)
for i in range(args.num_cpus)]
if ObjectList.is_kvm_cpu(TestCPUClass) or \
ObjectList.is_kvm_cpu(FutureClass):
system.kvm_vm = KvmVM()
system.kvm_vm = KvmVM()
# Create AMDGPU and attach to southbridge
shader = createGPU(system, args)
connectGPU(system, args)
# The shader core will be whatever is after the CPU cores are accounted for
shader_idx = args.num_cpus
system.cpu.append(shader)
# This arbitrary address is something in the X86 I/O hole
hsapp_gpu_map_paddr = 0xe00000000
hsapp_pt_walker = VegaPagetableWalker()
gpu_hsapp = HSAPacketProcessor(pioAddr=hsapp_gpu_map_paddr,
numHWQueues=args.num_hw_queues)
numHWQueues=args.num_hw_queues,
walker=hsapp_pt_walker)
dispatcher = GPUDispatcher()
cp_pt_walker = VegaPagetableWalker()
gpu_cmd_proc = GPUCommandProcessor(hsapp=gpu_hsapp,
dispatcher=dispatcher)
dispatcher=dispatcher,
walker=cp_pt_walker)
shader.dispatcher = dispatcher
shader.gpu_cmd_proc = gpu_cmd_proc
system.pc.south_bridge.gpu.cp = gpu_cmd_proc
# GPU Interrupt Handler
device_ih = AMDGPUInterruptHandler()
system.pc.south_bridge.gpu.device_ih = device_ih
# Setup the SDMA engines
sdma0_pt_walker = VegaPagetableWalker()
sdma1_pt_walker = VegaPagetableWalker()
sdma0 = SDMAEngine(walker=sdma0_pt_walker)
sdma1 = SDMAEngine(walker=sdma1_pt_walker)
system.pc.south_bridge.gpu.sdma0 = sdma0
system.pc.south_bridge.gpu.sdma1 = sdma1
# Setup PM4 packet processor
pm4_pkt_proc = PM4PacketProcessor()
system.pc.south_bridge.gpu.pm4_pkt_proc = pm4_pkt_proc
# GPU data path
gpu_mem_mgr = AMDGPUMemoryManager()
system.pc.south_bridge.gpu.memory_manager = gpu_mem_mgr
# CPU data path (SystemHub)
system_hub = AMDGPUSystemHub()
shader.system_hub = system_hub
# GPU, HSAPP, and GPUCommandProc are DMA devices
system._dma_ports.append(gpu_hsapp)
system._dma_ports.append(gpu_cmd_proc)
system._dma_ports.append(system.pc.south_bridge.gpu)
system._dma_ports.append(sdma0)
system._dma_ports.append(sdma1)
system._dma_ports.append(device_ih)
system._dma_ports.append(pm4_pkt_proc)
system._dma_ports.append(system_hub)
system._dma_ports.append(gpu_mem_mgr)
system._dma_ports.append(hsapp_pt_walker)
system._dma_ports.append(cp_pt_walker)
system._dma_ports.append(sdma0_pt_walker)
system._dma_ports.append(sdma1_pt_walker)
gpu_hsapp.pio = system.iobus.mem_side_ports
gpu_cmd_proc.pio = system.iobus.mem_side_ports
system.pc.south_bridge.gpu.pio = system.iobus.mem_side_ports
sdma0.pio = system.iobus.mem_side_ports
sdma1.pio = system.iobus.mem_side_ports
device_ih.pio = system.iobus.mem_side_ports
pm4_pkt_proc.pio = system.iobus.mem_side_ports
system_hub.pio = system.iobus.mem_side_ports
# Create Ruby system using Ruby.py for now
Ruby.create_system(args, True, system, system.iobus,
system._dma_ports)
# Full system needs special TLBs for SQC, Scalar, and vector data ports
args.full_system = True
GPUTLBConfig.config_tlb_hierarchy(args, system, shader_idx,
system.pc.south_bridge.gpu, True)
# Create Ruby system using disjoint VIPER topology
system.ruby = Disjoint_VIPER()
system.ruby.create(args, system, system.iobus, system._dma_ports)
# Create a seperate clock domain for Ruby
system.ruby.clk_domain = SrcClockDomain(clock = args.ruby_clock,
voltage_domain = system.voltage_domain)
for (i, cpu) in enumerate(system.cpu):
# Break once we reach the shader "CPU"
if i == args.num_cpus:
break
#
# Tie the cpu ports to the correct ruby system ports
#
@@ -125,9 +188,8 @@ def makeGpuFSSystem(args):
system.ruby._cpu_ports[i].connectCpuPorts(cpu)
# The shader core will be whatever is after the CPU cores are accounted for
shader_idx = args.num_cpus
system.cpu.append(shader)
for j in range(len(system.cpu[i].isa)):
system.cpu[i].isa[j].vendor_string = "AuthenticAMD"
gpu_port_idx = len(system.ruby._cpu_ports) \
- args.num_compute_units - args.num_sqc \

124
configs/example/gpufs/vega10_kvm.py Normal file
View File

@@ -0,0 +1,124 @@
# Copyright (c) 2022 Advanced Micro Devices, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its
# contributors may be used to endorse or promote products derived from this
# software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
import m5
import runfs
import base64
import tempfile
import argparse
import sys
import os
from amd import AmdGPUOptions
from common import Options
from common import GPUTLBOptions
from ruby import Ruby
demo_runscript = '''\
export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
export HSA_ENABLE_SDMA=0
dmesg -n3
dd if=/root/roms/vega10.rom of=/dev/mem bs=1k seek=768 count=128
if [ ! -f /lib/modules/`uname -r`/updates/dkms/amdgpu.ko ]; then
echo "ERROR: Missing DKMS package for kernel `uname -r`. Exiting gem5."
/sbin/m5 exit
fi
modprobe -v amdgpu ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
echo "Running {} {}"
echo "{}" | base64 -d > myapp
chmod +x myapp
./myapp {}
/sbin/m5 exit
'''
def addDemoOptions(parser):
parser.add_argument("-a", "--app", default=None,
help="GPU application to run")
parser.add_argument("-o", "--opts", default="",
help="GPU application arguments")
if __name__ == "__m5_main__":
parser = argparse.ArgumentParser()
runfs.addRunFSOptions(parser)
Options.addCommonOptions(parser)
AmdGPUOptions.addAmdGPUOptions(parser)
Ruby.define_options(parser)
GPUTLBOptions.tlb_options(parser)
addDemoOptions(parser)
# Parse now so we can override options
args = parser.parse_args()
# Create temp script to run application
if args.app is None:
print("No application given. Use %s -a <app>" % sys.argv[0])
sys.exit(1)
elif args.kernel is None:
print("No kernel path given. Use %s --kernel <vmlinux>" % sys.argv[0])
sys.exit(1)
elif args.disk_image is None:
print("No disk path given. Use %s --disk-image <linux>" % sys.argv[0])
sys.exit(1)
elif args.gpu_mmio_trace is None:
print("No MMIO trace path. Use %s --gpu-mmio-trace <path>"
% sys.argv[0])
sys.exit(1)
elif not os.path.isfile(args.app):
print("Could not find applcation", args.app)
sys.exit(1)
with open(os.path.abspath(args.app), 'rb') as binfile:
encodedBin = base64.b64encode(binfile.read()).decode()
_, tempRunscript = tempfile.mkstemp()
with open(tempRunscript, 'w') as b64file:
runscriptStr = demo_runscript.format(args.app, args.opts, encodedBin,
args.opts)
b64file.write(runscriptStr)
if args.second_disk == None:
args.second_disk = args.disk_image
# Defaults for Vega10
args.ruby = True
args.cpu_type = 'X86KvmCPU'
args.num_cpus = 1
args.mem_size = '3GB'
args.dgpu = True
args.dgpu_mem_size = '16GB'
args.dgpu_start = '0GB'
args.checkpoint_restore = 0
args.disjoint = True
args.timing_gpu = True
args.script = tempRunscript
args.dgpu_xor_low_bit = 0
# Run gem5
runfs.runGpuFSSystem(args)

2
configs/example/hmc_hello.py
View File

@@ -49,7 +49,7 @@ options = parser.parse_args()
system = System()
# use timing mode for the interaction between requestor-responder ports
system.mem_mode = 'timing'
# set the clock fequency of the system
# set the clock frequency of the system
clk = '1GHz'
vd = VoltageDomain(voltage='1V')
system.clk_domain = SrcClockDomain(clock=clk, voltage_domain=vd)

2
configs/example/hmctest.py
View File

@@ -50,7 +50,7 @@ def build_system(options):
system = System()
# use timing mode for the interaction between requestor-responder ports
system.mem_mode = 'timing'
# set the clock fequency of the system
# set the clock frequency of the system
clk = '100GHz'
vd = VoltageDomain(voltage='1V')
system.clk_domain = SrcClockDomain(clock=clk, voltage_domain=vd)

7
configs/example/lupv/run_lupv.py
View File

@@ -36,7 +36,6 @@ Characteristics
import m5
from m5.objects import Root
from gem5.runtime import get_runtime_isa
from gem5.components.boards.experimental.lupv_board import LupvBoard
from gem5.components.memory.single_channel import SingleChannelDDR3_1600
from gem5.components.processors.simple_processor import SimpleProcessor
@@ -83,11 +82,11 @@ memory = SingleChannelDDR3_1600(size="128MB")
# Setup a single core Processor.
if args.cpu_type == "atomic":
processor = SimpleProcessor(
cpu_type=CPUTypes.ATOMIC, num_cores=args.num_cpus
cpu_type=CPUTypes.ATOMIC, num_cores=args.num_cpus, isa=ISA.RISCV
)
elif args.cpu_type == "timing":
processor = SimpleProcessor(
cpu_type=CPUTypes.TIMING, num_cores=args.num_cpus
cpu_type=CPUTypes.TIMING, num_cores=args.num_cpus, isa=ISA.RISCV
)
# Setup the board.
@@ -106,7 +105,7 @@ board.set_kernel_disk_workload(
# Begin running of the simulation.
print("Running with ISA: " + get_runtime_isa().name)
print("Running with ISA: " + processor.get_isa().name)
print()
root = Root(full_system=True, system=board)
m5.instantiate()

4
configs/example/noc_config/2x4.py
View File

@@ -60,6 +60,10 @@ class CHI_HNF(CHI_config.CHI_HNF):
class NoC_Params(CHI_config.CHI_HNF.NoC_Params):
router_list = [1, 2, 5, 6]
class CHI_MN(CHI_config.CHI_MN):
class NoC_Params(CHI_config.CHI_MN.NoC_Params):
router_list = [4]
class CHI_SNF_MainMem(CHI_config.CHI_SNF_MainMem):
class NoC_Params(CHI_config.CHI_SNF_MainMem.NoC_Params):
router_list = [0, 4]

38
configs/example/ruby_gpu_random_test.py
View File

@@ -79,7 +79,7 @@ parser.add_argument("--random-seed", type=int, default=0,
help="Random seed number. Default value (i.e., 0) means \
using runtime-specific value")
parser.add_argument("--log-file", type=str, default="gpu-ruby-test.log")
parser.add_argument("--num-dmas", type=int, default=0,
parser.add_argument("--num-dmas", type=int, default=None,
help="The number of DMA engines to use in tester config.")
args = parser.parse_args()
@@ -108,7 +108,7 @@ if (args.system_size == "small"):
args.wf_size = 1
args.wavefronts_per_cu = 1
args.num_cpus = 1
args.num_dmas = 1
n_DMAs = 1
args.cu_per_sqc = 1
args.cu_per_scalar_cache = 1
args.num_compute_units = 1
@@ -117,7 +117,7 @@ elif (args.system_size == "medium"):
args.wf_size = 16
args.wavefronts_per_cu = 4
args.num_cpus = 4
args.num_dmas = 2
n_DMAs = 2
args.cu_per_sqc = 4
args.cu_per_scalar_cache = 4
args.num_compute_units = 4
@@ -126,11 +126,19 @@ elif (args.system_size == "large"):
args.wf_size = 32
args.wavefronts_per_cu = 4
args.num_cpus = 4
args.num_dmas = 4
n_DMAs = 4
args.cu_per_sqc = 4
args.cu_per_scalar_cache = 4
args.num_compute_units = 8
# Number of DMA engines
if not(args.num_dmas is None):
n_DMAs = args.num_dmas
# currently the tester does not support requests returned as
# aliased, thus we need num_dmas to be 0 for it
if not(args.num_dmas == 0):
print("WARNING: num_dmas != 0 not supported with VIPER")
#
# Set address range - 2 options
# level 0: small
@@ -173,9 +181,6 @@ tester_deadlock_threshold = 1e9
# For now we're testing only GPU protocol, so we force num_cpus to be 0
args.num_cpus = 0
# Number of DMA engines
n_DMAs = args.num_dmas
# Number of CUs
n_CUs = args.num_compute_units
@@ -234,11 +239,12 @@ args.num_cp = 0
#
# Make generic DMA sequencer for Ruby to use
#
dma_devices = [TesterDma()] * n_DMAs
system.piobus = IOXBar()
for _, dma_device in enumerate(dma_devices):
dma_device.pio = system.piobus.mem_side_ports
system.dma_devices = dma_devices
if n_DMAs > 0:
dma_devices = [TesterDma()] * n_DMAs
system.piobus = IOXBar()
for _, dma_device in enumerate(dma_devices):
dma_device.pio = system.piobus.mem_side_ports
system.dma_devices = dma_devices
#
# Create the Ruby system
@@ -250,7 +256,8 @@ system.dma_devices = dma_devices
# size of system.cpu
cpu_list = [ system.cpu ] * args.num_cpus
Ruby.create_system(args, full_system = False,
system = system, dma_ports = system.dma_devices,
system = system,
dma_ports = system.dma_devices if n_DMAs > 0 else [],
cpus = cpu_list)
#
@@ -275,7 +282,10 @@ print("Attaching ruby ports to the tester")
for i, ruby_port in enumerate(system.ruby._cpu_ports):
ruby_port.no_retry_on_stall = True
ruby_port.using_ruby_tester = True
ruby_port.mem_request_port = system.piobus.cpu_side_ports
# piobus is only created if there are DMAs
if n_DMAs > 0:
ruby_port.mem_request_port = system.piobus.cpu_side_ports
if i < n_CUs:
tester.cu_vector_ports = ruby_port.in_ports

1
configs/example/se.py
View File

@@ -200,6 +200,7 @@ for cpu in system.cpu:
if ObjectList.is_kvm_cpu(CPUClass) or ObjectList.is_kvm_cpu(FutureClass):
if buildEnv['TARGET_ISA'] == 'x86':
system.kvm_vm = KvmVM()
system.m5ops_base = 0xffff0000
for process in multiprocesses:
process.useArchPT = True
process.kvmInSE = True

2
configs/learning_gem5/part1/simple.py
View File

@@ -43,7 +43,7 @@ from m5.objects import *
# create the system we are going to simulate
system = System()
# Set the clock fequency of the system (and all of its children)
# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()

2
configs/learning_gem5/part1/two_level.py
View File

@@ -70,7 +70,7 @@ args = SimpleOpts.parse_args()
# create the system we are going to simulate
system = System()
# Set the clock fequency of the system (and all of its children)
# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()

2
configs/learning_gem5/part2/simple_cache.py
View File

@@ -39,7 +39,7 @@ from m5.objects import *
# create the system we are going to simulate
system = System()
# Set the clock fequency of the system (and all of its children)
# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()

2
configs/learning_gem5/part2/simple_memobj.py
View File

@@ -39,7 +39,7 @@ from m5.objects import *
# create the system we are going to simulate
system = System()
# Set the clock fequency of the system (and all of its children)
# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()

2
configs/learning_gem5/part3/ruby_test.py
View File

@@ -44,7 +44,7 @@ from test_caches import TestCacheSystem
# create the system we are going to simulate
system = System()
# Set the clock fequency of the system (and all of its children)
# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()

2
configs/learning_gem5/part3/simple_ruby.py
View File

@@ -53,7 +53,7 @@ from msi_caches import MyCacheSystem
# create the system we are going to simulate
system = System()
# Set the clock fequency of the system (and all of its children)
# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()

7
configs/network/Network.py
View File

@@ -80,6 +80,10 @@ def define_options(parser):
"--garnet-deadlock-threshold", action="store",
type=int, default=50000,
help="network-level deadlock threshold.")
parser.add_argument("--simple-physical-channels", action="store_true",
default=False,
help="""SimpleNetwork links uses a separate physical
channel for each virtual network""")
def create_network(options, ruby):
@@ -187,6 +191,9 @@ def init_network(options, network, InterfaceClass):
extLink.int_cred_bridge = int_cred_bridges
if options.network == "simple":
if options.simple_physical_channels:
network.physical_vnets_channels = \
[1] * int(network.number_of_virtual_networks)
network.setup_buffers()
if InterfaceClass != None:

22
configs/nvm/sweep.py
View File

@@ -106,14 +106,14 @@ MemConfig.config_mem(args, system)
# controller with an NVM interface, check to be sure
if not isinstance(system.mem_ctrls[0], m5.objects.MemCtrl):
fatal("This script assumes the controller is a MemCtrl subclass")
if not isinstance(system.mem_ctrls[0].nvm, m5.objects.NVMInterface):
if not isinstance(system.mem_ctrls[0].dram, m5.objects.NVMInterface):
fatal("This script assumes the memory is a NVMInterface class")
# there is no point slowing things down by saving any data
system.mem_ctrls[0].nvm.null = True
system.mem_ctrls[0].dram.null = True
# Set the address mapping based on input argument
system.mem_ctrls[0].nvm.addr_mapping = args.addr_map
system.mem_ctrls[0].dram.addr_mapping = args.addr_map
# stay in each state for 0.25 ms, long enough to warm things up, and
# short enough to avoid hitting a refresh
@@ -124,21 +124,21 @@ period = 250000000
# the DRAM maximum bandwidth to ensure that it is saturated
# get the number of regions
nbr_banks = system.mem_ctrls[0].nvm.banks_per_rank.value
nbr_banks = system.mem_ctrls[0].dram.banks_per_rank.value
# determine the burst length in bytes
burst_size = int((system.mem_ctrls[0].nvm.devices_per_rank.value *
system.mem_ctrls[0].nvm.device_bus_width.value *
system.mem_ctrls[0].nvm.burst_length.value) / 8)
burst_size = int((system.mem_ctrls[0].dram.devices_per_rank.value *
system.mem_ctrls[0].dram.device_bus_width.value *
system.mem_ctrls[0].dram.burst_length.value) / 8)
# next, get the page size in bytes
buffer_size = system.mem_ctrls[0].nvm.devices_per_rank.value * \
system.mem_ctrls[0].nvm.device_rowbuffer_size.value
buffer_size = system.mem_ctrls[0].dram.devices_per_rank.value * \
system.mem_ctrls[0].dram.device_rowbuffer_size.value
# match the maximum bandwidth of the memory, the parameter is in seconds
# and we need it in ticks (ps)
itt = system.mem_ctrls[0].nvm.tBURST.value * 1000000000000
itt = system.mem_ctrls[0].dram.tBURST.value * 1000000000000
# assume we start at 0
max_addr = mem_range.end
@@ -179,7 +179,7 @@ def trace():
0, max_addr, burst_size, int(itt), int(itt),
args.rd_perc, 0,
num_seq_pkts, buffer_size, nbr_banks, bank,
addr_map, args.nvm_ranks)
addr_map, args.dram_ranks)
yield system.tgen.createExit(0)
system.tgen.start(trace())

4
configs/nvm/sweep_hybrid.py
View File

@@ -117,8 +117,8 @@ MemConfig.config_mem(args, system)
# the following assumes that we are using the native controller
# with NVM and DRAM interfaces, check to be sure
if not isinstance(system.mem_ctrls[0], m5.objects.MemCtrl):
fatal("This script assumes the controller is a MemCtrl subclass")
if not isinstance(system.mem_ctrls[0], m5.objects.HeteroMemCtrl):
fatal("This script assumes the controller is a HeteroMemCtrl subclass")
if not isinstance(system.mem_ctrls[0].dram, m5.objects.DRAMInterface):
fatal("This script assumes the first memory is a DRAMInterface subclass")
if not isinstance(system.mem_ctrls[0].nvm, m5.objects.NVMInterface):

20
configs/ruby/CHI.py
View File

@@ -43,6 +43,7 @@ def define_options(parser):
default=None,
help="NoC config. parameters and bindings. "
"Required for CustomMesh topology")
parser.add_argument("--enable-dvm", default=False, action="store_true")
def read_config_file(file):
''' Read file as a module and return it '''
@@ -65,6 +66,13 @@ def create_system(options, full_system, system, dma_ports, bootmem,
if options.num_l3caches < 1:
m5.fatal('--num-l3caches must be at least 1')
if full_system and options.enable_dvm:
if len(cpus) <= 1:
m5.fatal("--enable-dvm can't be used with a single CPU")
for cpu in cpus:
for decoder in cpu.decoder:
decoder.dvm_enabled = True
# read specialized classes from config file if provided
if options.chi_config:
chi_defs = read_config_file(options.chi_config)
@@ -79,6 +87,7 @@ def create_system(options, full_system, system, dma_ports, bootmem,
# Node types
CHI_RNF = chi_defs.CHI_RNF
CHI_HNF = chi_defs.CHI_HNF
CHI_MN = chi_defs.CHI_MN
CHI_SNF_MainMem = chi_defs.CHI_SNF_MainMem
CHI_SNF_BootMem = chi_defs.CHI_SNF_BootMem
CHI_RNI_DMA = chi_defs.CHI_RNI_DMA
@@ -140,6 +149,14 @@ def create_system(options, full_system, system, dma_ports, bootmem,
network_nodes.append(rnf)
network_cntrls.extend(rnf.getNetworkSideControllers())
# Creates one Misc Node
ruby_system.mn = [ CHI_MN(ruby_system, [cpu.l1d for cpu in cpus]) ]
for mn in ruby_system.mn:
all_cntrls.extend(mn.getAllControllers())
network_nodes.append(mn)
network_cntrls.extend(mn.getNetworkSideControllers())
assert(mn.getAllControllers() == mn.getNetworkSideControllers())
# Look for other memories
other_memories = []
if bootmem:
@@ -156,8 +173,9 @@ def create_system(options, full_system, system, dma_ports, bootmem,
for m in other_memories:
sysranges.append(m.range)
hnf_list = [i for i in range(options.num_l3caches)]
CHI_HNF.createAddrRanges(sysranges, system.cache_line_size.value,
options.num_l3caches)
hnf_list)
ruby_system.hnf = [ CHI_HNF(i, ruby_system, HNFCache, None)
for i in range(options.num_l3caches) ]

88
configs/ruby/CHI_config.py
View File

@@ -230,6 +230,9 @@ class CHI_L1Controller(CHI_Cache_Controller):
self.number_of_TBEs = 16
self.number_of_repl_TBEs = 16
self.number_of_snoop_TBEs = 4
self.number_of_DVM_TBEs = 16
self.number_of_DVM_snoop_TBEs = 4
self.unify_repl_TBEs = False
class CHI_L2Controller(CHI_Cache_Controller):
@@ -262,6 +265,8 @@ class CHI_L2Controller(CHI_Cache_Controller):
self.number_of_TBEs = 32
self.number_of_repl_TBEs = 32
self.number_of_snoop_TBEs = 16
self.number_of_DVM_TBEs = 1 # should not receive any dvm
self.number_of_DVM_snoop_TBEs = 1 # should not receive any dvm
self.unify_repl_TBEs = False
class CHI_HNFController(CHI_Cache_Controller):
@@ -295,8 +300,41 @@ class CHI_HNFController(CHI_Cache_Controller):
self.number_of_TBEs = 32
self.number_of_repl_TBEs = 32
self.number_of_snoop_TBEs = 1 # should not receive any snoop
self.number_of_DVM_TBEs = 1 # should not receive any dvm
self.number_of_DVM_snoop_TBEs = 1 # should not receive any dvm
self.unify_repl_TBEs = False
class CHI_MNController(MiscNode_Controller):
'''
Default parameters for a Misc Node
'''
def __init__(self, ruby_system, addr_range, l1d_caches,
early_nonsync_comp):
super(CHI_MNController, self).__init__(
version = Versions.getVersion(MiscNode_Controller),
ruby_system = ruby_system,
mandatoryQueue = MessageBuffer(),
triggerQueue = TriggerMessageBuffer(),
retryTriggerQueue = TriggerMessageBuffer(),
schedRspTriggerQueue = TriggerMessageBuffer(),
reqRdy = TriggerMessageBuffer(),
snpRdy = TriggerMessageBuffer(),
)
# Set somewhat large number since we really a lot on internal
# triggers. To limit the controller performance, tweak other
# params such as: input port buffer size, cache banks, and output
# port latency
self.transitions_per_cycle = 1024
self.addr_ranges = [addr_range]
# 16 total transaction buffer entries, but 1 is reserved for DVMNonSync
self.number_of_DVM_TBEs = 16
self.number_of_non_sync_TBEs = 1
self.early_nonsync_comp = early_nonsync_comp
# "upstream_destinations" = targets for DVM snoops
self.upstream_destinations = l1d_caches
class CHI_DMAController(CHI_Cache_Controller):
'''
Default parameters for a DMA controller
@@ -333,6 +371,8 @@ class CHI_DMAController(CHI_Cache_Controller):
self.number_of_TBEs = 16
self.number_of_repl_TBEs = 1
self.number_of_snoop_TBEs = 1 # should not receive any snoop
self.number_of_DVM_TBEs = 1 # should not receive any dvm
self.number_of_DVM_snoop_TBEs = 1 # should not receive any dvm
self.unify_repl_TBEs = False
class CPUSequencerWrapper:
@@ -486,15 +526,14 @@ class CHI_HNF(CHI_Node):
'''HNFs may also define the 'pairing' parameter to allow pairing'''
pairing = None
_addr_ranges = []
_addr_ranges = {}
@classmethod
def createAddrRanges(cls, sys_mem_ranges, cache_line_size, num_hnfs):
def createAddrRanges(cls, sys_mem_ranges, cache_line_size, hnfs):
# Create the HNFs interleaved addr ranges
block_size_bits = int(math.log(cache_line_size, 2))
cls._addr_ranges = []
llc_bits = int(math.log(num_hnfs, 2))
llc_bits = int(math.log(len(hnfs), 2))
numa_bit = block_size_bits + llc_bits - 1
for i in range(num_hnfs):
for i, hnf in enumerate(hnfs):
ranges = []
for r in sys_mem_ranges:
addr_range = AddrRange(r.start, size = r.size(),
@@ -502,7 +541,7 @@ class CHI_HNF(CHI_Node):
intlvBits = llc_bits,
intlvMatch = i)
ranges.append(addr_range)
cls._addr_ranges.append((ranges, numa_bit, i))
cls._addr_ranges[hnf] = (ranges, numa_bit)
@classmethod
def getAddrRanges(cls, hnf_idx):
@@ -514,10 +553,9 @@ class CHI_HNF(CHI_Node):
def __init__(self, hnf_idx, ruby_system, llcache_type, parent):
super(CHI_HNF, self).__init__(ruby_system)
addr_ranges,intlvHighBit,intlvMatch = self.getAddrRanges(hnf_idx)
addr_ranges,intlvHighBit = self.getAddrRanges(hnf_idx)
# All ranges should have the same interleaving
assert(len(addr_ranges) >= 1)
assert(intlvMatch == hnf_idx)
ll_cache = llcache_type(start_index_bit = intlvHighBit + 1)
self._cntrl = CHI_HNFController(ruby_system, ll_cache, NULL,
@@ -537,6 +575,40 @@ class CHI_HNF(CHI_Node):
return [self._cntrl]
class CHI_MN(CHI_Node):
'''
Encapsulates a Misc Node controller.
'''
class NoC_Params(CHI_Node.NoC_Params):
'''HNFs may also define the 'pairing' parameter to allow pairing'''
pairing = None
# The CHI controller can be a child of this object or another if
# 'parent' if specified
def __init__(self, ruby_system, l1d_caches, early_nonsync_comp=False):
super(CHI_MN, self).__init__(ruby_system)
# MiscNode has internal address range starting at 0
addr_range = AddrRange(0, size = "1kB")
self._cntrl = CHI_MNController(ruby_system, addr_range, l1d_caches,
early_nonsync_comp)
self.cntrl = self._cntrl
self.connectController(self._cntrl)
def connectController(self, cntrl):
CHI_Node.connectController(self, cntrl)
def getAllControllers(self):
return [self._cntrl]
def getNetworkSideControllers(self):
return [self._cntrl]
class CHI_SNF_Base(CHI_Node):
'''
Creates CHI node controllers for the memory controllers

86
configs/ruby/GPU_VIPER.py
View File

@@ -34,6 +34,8 @@ from m5.defines import buildEnv
from m5.util import addToPath
from .Ruby import create_topology
from .Ruby import send_evicts
from common import ObjectList
from common import MemConfig
from common import FileSystemConfig
addToPath('../')
@@ -441,6 +443,66 @@ def construct_dirs(options, system, ruby_system, network):
dir_cntrl.responseToCore = MessageBuffer()
dir_cntrl.responseToCore.out_port = network.in_port
dir_cntrl.triggerQueue = MessageBuffer(ordered = True)
dir_cntrl.L3triggerQueue = MessageBuffer(ordered = True)
dir_cntrl.requestToMemory = MessageBuffer(ordered = True)
dir_cntrl.responseFromMemory = MessageBuffer(ordered = True)
dir_cntrl.requestFromDMA = MessageBuffer(ordered=True)
dir_cntrl.requestFromDMA.in_port = network.out_port
dir_cntrl.responseToDMA = MessageBuffer()
dir_cntrl.responseToDMA.out_port = network.in_port
exec("ruby_system.dir_cntrl%d = dir_cntrl" % i)
dir_cntrl_nodes.append(dir_cntrl)
return dir_cntrl_nodes
def construct_gpudirs(options, system, ruby_system, network):
dir_cntrl_nodes = []
mem_ctrls = []
xor_low_bit = 0
# For an odd number of CPUs, still create the right number of controllers
TCC_bits = int(math.log(options.num_tccs, 2))
dir_bits = int(math.log(options.dgpu_num_dirs, 2))
block_size_bits = int(math.log(options.cacheline_size, 2))
numa_bit = block_size_bits + dir_bits - 1
gpu_mem_range = AddrRange(0, size = options.dgpu_mem_size)
for i in range(options.dgpu_num_dirs):
addr_range = m5.objects.AddrRange(gpu_mem_range.start,
size = gpu_mem_range.size(),
intlvHighBit = numa_bit,
intlvBits = dir_bits,
intlvMatch = i,
xorHighBit = xor_low_bit)
dir_cntrl = DirCntrl(noTCCdir = True, TCC_select_num_bits = TCC_bits)
dir_cntrl.create(options, [addr_range], ruby_system, system)
dir_cntrl.number_of_TBEs = options.num_tbes
dir_cntrl.useL3OnWT = False
# Connect the Directory controller to the ruby network
dir_cntrl.requestFromCores = MessageBuffer(ordered = True)
dir_cntrl.requestFromCores.in_port = network.out_port
dir_cntrl.responseFromCores = MessageBuffer()
dir_cntrl.responseFromCores.in_port = network.out_port
dir_cntrl.unblockFromCores = MessageBuffer()
dir_cntrl.unblockFromCores.in_port = network.out_port
dir_cntrl.probeToCore = MessageBuffer()
dir_cntrl.probeToCore.out_port = network.in_port
dir_cntrl.responseToCore = MessageBuffer()
dir_cntrl.responseToCore.out_port = network.in_port
dir_cntrl.triggerQueue = MessageBuffer(ordered = True)
dir_cntrl.L3triggerQueue = MessageBuffer(ordered = True)
dir_cntrl.requestToMemory = MessageBuffer()
@@ -455,10 +517,28 @@ def construct_dirs(options, system, ruby_system, network):
dir_cntrl.requestToMemory = MessageBuffer()
dir_cntrl.responseFromMemory = MessageBuffer()
exec("ruby_system.dir_cntrl%d = dir_cntrl" % i)
dir_cntrl_nodes.append(dir_cntrl)
# Create memory controllers too
mem_type = ObjectList.mem_list.get(options.dgpu_mem_type)
dram_intf = MemConfig.create_mem_intf(mem_type, gpu_mem_range, i,
int(math.log(options.dgpu_num_dirs, 2)), options.cacheline_size,
xor_low_bit)
if issubclass(mem_type, DRAMInterface):
mem_ctrl = m5.objects.MemCtrl(dram = dram_intf)
else:
mem_ctrl = dram_intf
return dir_cntrl_nodes
mem_ctrl.port = dir_cntrl.memory_out_port
mem_ctrl.dram.enable_dram_powerdown = False
dir_cntrl.addr_ranges = dram_intf.range
# Append
exec("system.ruby.gpu_dir_cntrl%d = dir_cntrl" % i)
dir_cntrl_nodes.append(dir_cntrl)
mem_ctrls.append(mem_ctrl)
system.gpu_mem_ctrls = mem_ctrls
return dir_cntrl_nodes, mem_ctrls
def construct_corepairs(options, system, ruby_system, network):

4
configs/ruby/Ruby.py
View File

@@ -130,7 +130,7 @@ def setup_memory_controllers(system, ruby, dir_cntrls, options):
if len(system.mem_ranges) > 1:
crossbar = IOXBar()
crossbars.append(crossbar)
dir_cntrl.memory = crossbar.cpu_side_ports
dir_cntrl.memory_out_port = crossbar.cpu_side_ports
dir_ranges = []
for r in system.mem_ranges:
@@ -152,7 +152,7 @@ def setup_memory_controllers(system, ruby, dir_cntrls, options):
if crossbar != None:
mem_ctrl.port = crossbar.mem_side_ports
else:
mem_ctrl.port = dir_cntrl.memory
mem_ctrl.port = dir_cntrl.memory_out_port
# Enable low-power DRAM states if option is set
if issubclass(mem_type, DRAMInterface):

8
configs/topologies/CustomMesh.py
View File

@@ -239,6 +239,7 @@ class CustomMesh(SimpleTopology):
# classify nodes into different types
rnf_nodes = []
hnf_nodes = []
mn_nodes = []
mem_nodes = []
io_mem_nodes = []
rni_dma_nodes = []
@@ -248,6 +249,7 @@ class CustomMesh(SimpleTopology):
# the same base type.
rnf_params = None
hnf_params = None
mn_params = None
mem_params = None
io_mem_params = None
rni_dma_params = None
@@ -264,6 +266,9 @@ class CustomMesh(SimpleTopology):
elif isinstance(n, CHI.CHI_HNF):
hnf_nodes.append(n)
hnf_params = check_same(type(n).NoC_Params, hnf_params)
elif isinstance(n, CHI.CHI_MN):
mn_nodes.append(n)
mn_params = check_same(type(n).NoC_Params, mn_params)
elif isinstance(n, CHI.CHI_SNF_MainMem):
mem_nodes.append(n)
mem_params = check_same(type(n).NoC_Params, mem_params)
@@ -298,6 +303,9 @@ class CustomMesh(SimpleTopology):
# Place CHI_HNF on the mesh
self.distributeNodes(hnf_params, hnf_nodes)
# Place CHI_MN on the mesh
self.distributeNodes(options, mn_params, mn_nodes)
# Place CHI_SNF_MainMem on the mesh
self.distributeNodes(mem_params, mem_nodes)

4
ext/Kconfiglib/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
*.py[co]
build/
*.egg-info/
dist/

5
ext/Kconfiglib/LICENSE.txt Normal file
View File

@@ -0,0 +1,5 @@
Copyright (c) 2011-2019, Ulf Magnusson <ulfalizer@gmail.com>
Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

2
ext/Kconfiglib/MANIFEST.in Normal file
View File

@@ -0,0 +1,2 @@
# Include the license file in source distributions
include LICENSE.txt

841
ext/Kconfiglib/README.rst Normal file
View File

@@ -0,0 +1,841 @@
.. contents:: Table of contents
:backlinks: none
News
----
Dependency loop with recent linux-next kernels
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To fix issues with dependency loops on recent linux-next kernels, apply `this
patch <https://www.spinics.net/lists/linux-kbuild/msg23455.html>`_. Hopefully,
it will be in ``linux-next`` soon.
``windows-curses`` is no longer automatically installed on Windows
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Starting with Kconfiglib 13.0.0, the `windows-curses
<https://github.com/zephyrproject-rtos/windows-curses>`__ package is no longer
automatically installed on Windows, and needs to be installed manually for the
terminal ``menuconfig`` to work.
This fixes installation of Kconfiglib on MSYS2, which is not compatible with
``windows-curses``. See `this issue
<https://github.com/ulfalizer/Kconfiglib/issues/77>`__.
The ``menuconfig`` now shows a hint re. installing ``windows-curses`` when the
``curses`` module can't be imported on Windows.
Sorry if this change caused problems!
Overview
--------
Kconfiglib is a `Kconfig
<https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-language.rst>`__
implementation in Python 2/3. It started out as a helper library, but now has a
enough functionality to also work well as a standalone Kconfig implementation
(including `terminal and GUI menuconfig interfaces <Menuconfig interfaces_>`_
and `Kconfig extensions`_).
The entire library is contained in `kconfiglib.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_. The
bundled scripts are implemented on top of it. Implementing your own scripts
should be relatively easy, if needed.
Kconfiglib is used exclusively by e.g. the `Zephyr
<https://www.zephyrproject.org/>`__, `esp-idf
<https://github.com/espressif/esp-idf>`__, and `ACRN
<https://projectacrn.org/>`__ projects. It is also used for many small helper
scripts in various projects.
Since Kconfiglib is based around a library, it can be used e.g. to generate a
`Kconfig cross-reference
<https://docs.zephyrproject.org/latest/reference/kconfig/index.html>`_, using
the same robust Kconfig parser used for other Kconfig tools, instead of brittle
ad-hoc parsing. The documentation generation script can be found `here
<https://github.com/zephyrproject-rtos/zephyr/blob/master/doc/scripts/genrest.py>`__.
Kconfiglib implements the recently added `Kconfig preprocessor
<https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-macro-language.rst>`__.
For backwards compatibility, environment variables can be referenced both as
``$(FOO)`` (the new syntax) and as ``$FOO`` (the old syntax). The old syntax is
deprecated, but will probably be supported for a long time, as it's needed to
stay compatible with older Linux kernels. The major version will be increased
if support is ever dropped. Using the old syntax with an undefined environment
variable keeps the string as is.
Note: See `this issue <https://github.com/ulfalizer/Kconfiglib/issues/47>`__ if
you run into a "macro expanded to blank string" error with kernel 4.18+.
See `this page
<https://docs.zephyrproject.org/latest/guides/kconfig/tips.html>`__ for some
Kconfig tips and best practices.
Installation
------------
Installation with pip
~~~~~~~~~~~~~~~~~~~~~
Kconfiglib is available on `PyPI <https://pypi.python.org/pypi/kconfiglib/>`_ and can be
installed with e.g.
.. code::
$ pip(3) install kconfiglib
Microsoft Windows is supported.
The ``pip`` installation will give you both the base library and the following
executables. All but two (``genconfig`` and ``setconfig``) mirror functionality
available in the C tools.
- `menuconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_
- `guiconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/guiconfig.py>`_
- `oldconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/oldconfig.py>`_
- `olddefconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/olddefconfig.py>`_
- `savedefconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/savedefconfig.py>`_
- `defconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/defconfig.py>`_
- `alldefconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/alldefconfig.py>`_
- `allnoconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/allnoconfig.py>`_
- `allmodconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/allmodconfig.py>`_
- `allyesconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/allyesconfig.py>`_
- `listnewconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/listnewconfig.py>`_
- `genconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/genconfig.py>`_
- `setconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/setconfig.py>`_
``genconfig`` is intended to be run at build time. It generates a C header from
the configuration and (optionally) information that can be used to rebuild only
files that reference Kconfig symbols that have changed value.
Starting with Kconfiglib version 12.2.0, all utilities are compatible with both
Python 2 and Python 3. Previously, ``menuconfig.py`` only ran under Python 3
(i.e., it's now more backwards compatible than before).
**Note:** If you install Kconfiglib with ``pip``'s ``--user`` flag, make sure
that your ``PATH`` includes the directory where the executables end up. You can
list the installed files with ``pip(3) show -f kconfiglib``.
All releases have a corresponding tag in the git repository, e.g. ``v14.1.0``
(the latest version).
`Semantic versioning <http://semver.org/>`_ is used. There's been ten small
changes to the behavior of the API, a Windows packaging change, and a hashbang
change to use ``python3``
(`1 <https://github.com/ulfalizer/Kconfiglib/commit/e8b4ecb6ff6ccc1c7be0818314fbccda2ef2b2ee>`_,
`2 <https://github.com/ulfalizer/Kconfiglib/commit/db633015a4d7b0ba1e882f665e191f350932b2af>`_,
`3 <https://github.com/ulfalizer/Kconfiglib/commit/8983f7eb297dd614faf0beee3129559bc8ba338e>`_,
`4 <https://github.com/ulfalizer/Kconfiglib/commit/cbf32e29a130d22bc734b7778e6304ac9df2a3e8>`_,
`5 <https://github.com/ulfalizer/Kconfiglib/commit/eb6c21a9b33a2d6e2bed9882d4f930d0cab2f03b>`_,
`6 <https://github.com/ulfalizer/Kconfiglib/commit/c19fc11355b13d75d97286402c7a933fb23d3b70>`_,
`7 <https://github.com/ulfalizer/Kconfiglib/commit/7a428aa415606820a44291f475248b08e3952c4b>`_,
`8 <https://github.com/ulfalizer/Kconfiglib/commit/f247ddf618ad29718e5efd3e69f8baf75d4d347b>`_,
`9 <https://github.com/ulfalizer/Kconfiglib/commit/4fed39d9271ceb68be4157ab3f96a45b94f77dc0>`_,
`10 <https://github.com/ulfalizer/Kconfiglib/commit/55bc8c380869ea663092212e8fe388ad7abae596>`_,
`Windows packaging change <https://github.com/ulfalizer/Kconfiglib/commit/21b4c1e3b6e2867b9a0788d21a358f6b1f581d86>`_,
`Python 3 hashbang change <https://github.com/ulfalizer/Kconfiglib/commit/9e0a8d29fa76adcb3f27bb2e20f16fefc2a8591e>`_),
which is why the major version is at 14 rather than 2. I do major version bumps
for all behavior changes, even tiny ones, and most of these were fixes for baby
issues in the early days of the Kconfiglib 2 API.
Manual installation
~~~~~~~~~~~~~~~~~~~
Just drop ``kconfiglib.py`` and the scripts you want somewhere. There are no
third-party dependencies, but the terminal ``menuconfig`` won't work on Windows
unless a package like `windows-curses
<https://github.com/zephyrproject-rtos/windows-curses>`__ is installed.
Installation for the Linux kernel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
See the module docstring at the top of `kconfiglib.py <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_.
Python version compatibility (2.7/3.2+)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Kconfiglib and all utilities run under both Python 2.7 and Python 3.2 and
later. The code mostly uses basic Python features and has no third-party
dependencies, so keeping it backwards-compatible is pretty low effort.
The 3.2 requirement comes from ``argparse``. ``format()`` with unnumbered
``{}`` is used as well.
A recent Python 3 version is recommended if you have a choice, as it'll give
you better Unicode handling.
Getting started
---------------
1. `Install <Installation_>`_ the library and the utilities.
2. Write `Kconfig
<https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-language.rst>`__
files that describe the available configuration options. See `this page
<https://docs.zephyrproject.org/latest/guides/kconfig/tips.html>`__ for some
general Kconfig advice.
3. Generate an initial configuration with e.g. ``menuconfig``/``guiconfig`` or
``alldefconfig``. The configuration is saved as ``.config`` by default.
For more advanced projects, the ``defconfig`` utility can be used to
generate the initial configuration from an existing configuration file.
Usually, this existing configuration file would be a minimal configuration
file, as generated by e.g. ``savedefconfig``.
4. Run ``genconfig`` to generate a header file. By default, it is saved as
``config.h``.
Normally, ``genconfig`` would be run automatically as part of the build.
Before writing a header file or other configuration output, Kconfiglib
compares the old contents of the file against the new contents. If there's
no change, the write is skipped. This avoids updating file metadata like the
modification time, and might save work depending on your build setup.
Adding new configuration output formats should be relatively straightforward.
See the implementation of ``write_config()`` in `kconfiglib.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_.
The documentation for the ``Symbol.config_string`` property has some tips as
well.
5. To update an old ``.config`` file after the Kconfig files have changed (e.g.
to add new options), run ``oldconfig`` (prompts for values for new options)
or ``olddefconfig`` (gives new options their default value). Entering the
``menuconfig`` or ``guiconfig`` interface and saving the configuration will
also update it (the configuration interfaces always prompt for saving
on exit if it would modify the contents of the ``.config`` file).
Due to Kconfig semantics, simply loading an old ``.config`` file performs an
implicit ``olddefconfig``, so building will normally not be affected by
having an outdated configuration.
Whenever ``.config`` is overwritten, the previous version of the file is saved
to ``.config.old`` (or, more generally, to ``$KCONFIG_CONFIG.old``).
Using ``.config`` files as Make input
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``.config`` files use Make syntax and can be included directly in Makefiles to
read configuration values from there. This is why ``n``-valued
``bool``/``tristate`` values are written out as ``# CONFIG_FOO is not set`` (a
Make comment) in ``.config``, allowing them to be tested with ``ifdef`` in
Make.
If you make use of this, you might want to pass ``--config-out <filename>`` to
``genconfig`` and include the configuration file it generates instead of
including ``.config`` directly. This has the advantage that the generated
configuration file will always be a "full" configuration file, even if
``.config`` is outdated. Otherwise, it might be necessary to run
``old(def)config`` or ``menuconfig``/``guiconfig`` before rebuilding with an
outdated ``.config``.
If you use ``--sync-deps`` to generate incremental build information, you can
include ``deps/auto.conf`` instead, which is also a full configuration file.
Useful helper macros
~~~~~~~~~~~~~~~~~~~~
The `include/linux/kconfig.h
<https://github.com/torvalds/linux/blob/master/include/linux/kconfig.h>`_
header in the Linux kernel defines some useful helper macros for testing
Kconfig configuration values.
``IS_ENABLED()`` is generally useful, allowing configuration values to be
tested in ``if`` statements with no runtime overhead.
Incremental building
~~~~~~~~~~~~~~~~~~~~
See the docstring for ``Kconfig.sync_deps()`` in `kconfiglib.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_ for hints
on implementing incremental builds (rebuilding just source files that reference
changed configuration values).
Running the ``scripts/basic/fixdep.c`` tool from the kernel on the output of
``gcc -MD <source file>`` might give you an idea of how it all fits together.
Library documentation
---------------------
Kconfiglib comes with extensive documentation in the form of docstrings. To view it, run e.g.
the following command:
.. code:: sh
$ pydoc(3) kconfiglib
For HTML output, add ``-w``:
.. code:: sh
$ pydoc(3) -w kconfiglib
This will also work after installing Kconfiglib with ``pip(3)``.
Documentation for other modules can be viewed in the same way (though a plain
``--help`` will work when they're run as executables):
.. code:: sh
$ pydoc(3) menuconfig/guiconfig/...
A good starting point for learning the library is to read the module docstring
(which you could also just read directly at the beginning of `kconfiglib.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_). It
gives an introduction to symbol values, the menu tree, and expressions.
After reading the module docstring, a good next step is to read the ``Kconfig``
class documentation, and then the documentation for the ``Symbol``, ``Choice``,
and ``MenuNode`` classes.
Please tell me if something is unclear or can be explained better.
Library features
----------------
Kconfiglib can do the following, among other things:
- **Programmatically get and set symbol values**
See `allnoconfig.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/allnoconfig.py>`_ and
`allyesconfig.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/allyesconfig.py>`_,
which are automatically verified to produce identical output to the standard
``make allnoconfig`` and ``make allyesconfig``.
- **Read and write .config and defconfig files**
The generated ``.config`` and ``defconfig`` (minimal configuration) files are
character-for-character identical to what the C implementation would generate
(except for the header comment). The test suite relies on this, as it
compares the generated files.
- **Write C headers**
The generated headers use the same format as ``include/generated/autoconf.h``
from the Linux kernel. Output for symbols appears in the order that they're
defined, unlike in the C tools (where the order depends on the hash table
implementation).
- **Implement incremental builds**
This uses the same scheme as the ``include/config`` directory in the kernel:
Symbols are translated into files that are touched when the symbol's value
changes between builds, which can be used to avoid having to do a full
rebuild whenever the configuration is changed.
See the ``sync_deps()`` function for more information.
- **Inspect symbols**
Printing a symbol or other item (which calls ``__str__()``) returns its
definition in Kconfig format. This also works for symbols defined in multiple
locations.
A helpful ``__repr__()`` is on all objects too.
All ``__str__()`` and ``__repr__()`` methods are deliberately implemented
with just public APIs, so all symbol information can be fetched separately as
well.
- **Inspect expressions**
Expressions use a simple tuple-based format that can be processed manually
if needed. Expression printing and evaluation functions are provided,
implemented with public APIs.
- **Inspect the menu tree**
The underlying menu tree is exposed, including submenus created implicitly
from symbols depending on preceding symbols. This can be used e.g. to
implement menuconfig-like functionality.
See `menuconfig.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_/`guiconfig.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/guiconfig.py>`_ and the
minimalistic `menuconfig_example.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/examples/menuconfig_example.py>`_
example.
Kconfig extensions
~~~~~~~~~~~~~~~~~~
The following Kconfig extensions are available:
- ``source`` supports glob patterns and includes each matching file. A pattern
is required to match at least one file.
A separate ``osource`` statement is available for cases where it's okay for
the pattern to match no files (in which case ``osource`` turns into a no-op).
- A relative ``source`` statement (``rsource``) is available, where file paths
are specified relative to the directory of the current Kconfig file. An
``orsource`` statement is available as well, analogous to ``osource``.
- Preprocessor user functions can be defined in Python, which makes it simple
to integrate information from existing Python tools into Kconfig (e.g. to
have Kconfig symbols depend on hardware information stored in some other
format).
See the *Kconfig extensions* section in the
`kconfiglib.py <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_
module docstring for more information.
- ``def_int``, ``def_hex``, and ``def_string`` are available in addition to
``def_bool`` and ``def_tristate``, allowing ``int``, ``hex``, and ``string``
symbols to be given a type and a default at the same time.
These can be useful in projects that make use of symbols defined in multiple
locations, and remove some Kconfig inconsistency.
- Environment variables are expanded directly in e.g. ``source`` and
``mainmenu`` statements, meaning ``option env`` symbols are redundant.
This is the standard behavior with the new `Kconfig preprocessor
<https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-macro-language.rst>`__,
which Kconfiglib implements.
``option env`` symbols are accepted but ignored, which leads the caveat that
they must have the same name as the environment variables they reference
(Kconfiglib warns if the names differ). This keeps Kconfiglib compatible with
older Linux kernels, where the name of the ``option env`` symbol always
matched the environment variable. Compatibility with older Linux kernels is
the main reason ``option env`` is still supported.
The C tools have dropped support for ``option env``.
- Two extra optional warnings can be enabled by setting environment variables,
covering cases that are easily missed when making changes to Kconfig files:
* ``KCONFIG_WARN_UNDEF``: If set to ``y``, warnings will be generated for all
references to undefined symbols within Kconfig files. The only gotcha is
that all hex literals must be prefixed with ``0x`` or ``0X``, to make it
possible to distinguish them from symbol references.
Some projects (e.g. the Linux kernel) use multiple Kconfig trees with many
shared Kconfig files, leading to some safe undefined symbol references.
``KCONFIG_WARN_UNDEF`` is useful in projects that only have a single
Kconfig tree though.
``KCONFIG_STRICT`` is an older alias for this environment variable,
supported for backwards compatibility.
* ``KCONFIG_WARN_UNDEF_ASSIGN``: If set to ``y``, warnings will be generated
for all assignments to undefined symbols within ``.config`` files. By
default, no such warnings are generated.
This warning can also be enabled/disabled by setting
``Kconfig.warn_assign_undef`` to ``True``/``False``.
Other features
--------------
- **Single-file implementation**
The entire library is contained in `kconfiglib.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_.
The tools implemented on top of it are one file each.
- **Robust and highly compatible with the C Kconfig tools**
 The `test suite <https://github.com/ulfalizer/Kconfiglib/blob/master/testsuite.py>`_
automatically compares output from Kconfiglib and the C tools
by diffing the generated ``.config`` files for the real kernel Kconfig and
defconfig files, for all ARCHes.
This currently involves comparing the output for 36 ARCHes and 498 defconfig
files (or over 18000 ARCH/defconfig combinations in "obsessive" test suite
mode). All tests are expected to pass.
A comprehensive suite of selftests is included as well.
- **Not horribly slow despite being a pure Python implementation**
The `allyesconfig.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/allyesconfig.py>`_
script currently runs in about 1.3 seconds on the Linux kernel on a Core i7
2600K (with a warm file cache), including the ``make`` overhead from ``make
scriptconfig``. Note that the Linux kernel Kconfigs are absolutely massive
(over 14k symbols for x86) compared to most projects, and also have overhead
from running shell commands via the Kconfig preprocessor.
Kconfiglib is especially speedy in cases where multiple ``.config`` files
need to be processed, because the ``Kconfig`` files will only need to be parsed
once.
For long-running jobs, `PyPy <https://pypy.org/>`_ gives a big performance
boost. CPython is faster for short-running jobs as PyPy needs some time to
warm up.
Kconfiglib also works well with the
`multiprocessing <https://docs.python.org/3/library/multiprocessing.html>`_
module. No global state is kept.
- **Generates more warnings than the C implementation**
Generates the same warnings as the C implementation, plus additional ones.
Also detects dependency and ``source`` loops.
All warnings point out the location(s) in the ``Kconfig`` files where a
symbol is defined, where applicable.
- **Unicode support**
Unicode characters in string literals in ``Kconfig`` and ``.config`` files are
correctly handled. This support mostly comes for free from Python.
- **Windows support**
Nothing Linux-specific is used. Universal newlines mode is used for both
Python 2 and Python 3.
The `Zephyr <https://www.zephyrproject.org/>`_ project uses Kconfiglib to
generate ``.config`` files and C headers on Linux as well as Windows.
- **Internals that (mostly) mirror the C implementation**
While being simpler to understand and tweak.
Menuconfig interfaces
---------------------
Three configuration interfaces are currently available:
- `menuconfig.py <https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_
is a terminal-based configuration interface implemented using the standard
Python ``curses`` module. ``xconfig`` features like showing invisible symbols and
showing symbol names are included, and it's possible to jump directly to a symbol
in the menu tree (even if it's currently invisible).
.. image:: https://raw.githubusercontent.com/ulfalizer/Kconfiglib/screenshots/screenshots/menuconfig.gif
*There is now also a show-help mode that shows the help text of the currently
selected symbol in the help window at the bottom.*
Starting with Kconfiglib 12.2.0, ``menuconfig.py`` runs under both Python 2
and Python 3 (previously, it only ran under Python 3, so this was a
backport). Running it under Python 3 provides better support for Unicode text
entry (``get_wch()`` is not available in the ``curses`` module on Python 2).
There are no third-party dependencies on \*nix. On Windows,
the ``curses`` modules is not available by default, but support
can be added by installing the ``windows-curses`` package:
.. code-block:: shell
$ pip install windows-curses
This uses wheels built from `this repository
<https://github.com/zephyrproject-rtos/windows-curses>`_, which is in turn
based on Christoph Gohlke's `Python Extension Packages for Windows
<https://www.lfd.uci.edu/~gohlke/pythonlibs/#curses>`_.
See the docstring at the top of `menuconfig.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_ for
more information about the terminal menuconfig implementation.
- `guiconfig.py
<https://github.com/ulfalizer/Kconfiglib/blob/master/guiconfig.py>`_ is a
graphical configuration interface written in `Tkinter
<https://docs.python.org/3/library/tkinter.html>`_. Like ``menuconfig.py``,
it supports showing all symbols (with invisible symbols in red) and jumping
directly to symbols. Symbol values can also be changed directly from the
jump-to dialog.
When single-menu mode is enabled, a single menu is shown at a time, like in
the terminal menuconfig. Only this mode distinguishes between symbols defined
with ``config`` and symbols defined with ``menuconfig``.
``guiconfig.py`` has been tested on X11, Windows, and macOS, and is
compatible with both Python 2 and Python 3.
Despite being part of the Python standard library, ``tkinter`` often isn't
included by default in Python installations on Linux. These commands will
install it on a few different distributions:
- Ubuntu: ``sudo apt install python-tk``/``sudo apt install python3-tk``
- Fedora: ``dnf install python2-tkinter``/``dnf install python3-tkinter``
- Arch: ``sudo pacman -S tk``
- Clear Linux: ``sudo swupd bundle-add python3-tcl``
Screenshot below, with show-all mode enabled and the jump-to dialog open:
.. image:: https://raw.githubusercontent.com/ulfalizer/Kconfiglib/screenshots/screenshots/guiconfig.png
To avoid having to carry around a bunch of GIFs, the image data is embedded
in ``guiconfig.py``. To use separate GIF files instead, change
``_USE_EMBEDDED_IMAGES`` to ``False`` in ``guiconfig.py``. The image files
can be found in the `screenshots
<https://github.com/ulfalizer/Kconfiglib/tree/screenshots/guiconfig>`_
branch.
I did my best with the images, but some are definitely only art adjacent.
Touch-ups are welcome. :)
- `pymenuconfig <https://github.com/RomaVis/pymenuconfig>`_, built by `RomaVis
<https://github.com/RomaVis>`_, is an older portable Python 2/3 TkInter
menuconfig implementation.
Screenshot below:
.. image:: https://raw.githubusercontent.com/RomaVis/pymenuconfig/master/screenshot.PNG
While working on the terminal menuconfig implementation, I added a few APIs
to Kconfiglib that turned out to be handy. ``pymenuconfig`` predates
``menuconfig.py`` and ``guiconfig.py``, and so didn't have them available.
Blame me for any workarounds.
Examples
--------
Example scripts
~~~~~~~~~~~~~~~
The `examples/ <https://github.com/ulfalizer/Kconfiglib/blob/master/examples>`_ directory contains some simple example scripts. Among these are the following ones. Make sure you run them with the latest version of Kconfiglib, as they might make use of newly added features.
- `eval_expr.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/eval_expr.py>`_ evaluates an expression in the context of a configuration.
- `find_symbol.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/find_symbol.py>`_ searches through expressions to find references to a symbol, also printing a "backtrace" with parents for each reference found.
- `help_grep.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/help_grep.py>`_ searches for a string in all help texts.
- `print_tree.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/print_tree.py>`_ prints a tree of all configuration items.
- `print_config_tree.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/print_config_tree.py>`_ is similar to ``print_tree.py``, but dumps the tree as it would appear in ``menuconfig``, including values. This can be handy for visually diffing between ``.config`` files and different versions of ``Kconfig`` files.
- `list_undefined.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/list_undefined.py>`_ finds references to symbols that are not defined by any architecture in the Linux kernel.
- `merge_config.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/merge_config.py>`_ merges configuration fragments to produce a complete .config, similarly to ``scripts/kconfig/merge_config.sh`` from the kernel.
- `menuconfig_example.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/menuconfig_example.py>`_ implements a configuration interface that uses notation similar to ``make menuconfig``. It's deliberately kept as simple as possible to demonstrate just the core concepts.
Real-world examples
~~~~~~~~~~~~~~~~~~~
- `kconfig.py
<https://github.com/zephyrproject-rtos/zephyr/blob/master/scripts/kconfig/kconfig.py>`_
from the `Zephyr <https://www.zephyrproject.org/>`_ project handles
``.config`` and header file generation, also doing configuration fragment
merging
- `genrest.py
<https://github.com/zephyrproject-rtos/zephyr/blob/master/doc/scripts/genrest.py>`_
generates a Kconfig symbol cross-reference, which can be viewed `here
<http://docs.zephyrproject.org/reference/kconfig/index.html>`__
- `CMake and IDE integration
<https://github.com/espressif/esp-idf/tree/master/tools/kconfig_new>`_ from
the ESP-IDF project, via a configuration server program.
- `A script for turning on USB-related options
<https://github.com/google/syzkaller/blob/master/dashboard/config/kconfiglib-merge-usb-configs.py>`_,
from the `syzkaller <https://github.com/google/syzkaller>`_ project.
- `Various automated checks
<https://github.com/zephyrproject-rtos/ci-tools/blob/master/scripts/check_compliance.py>`_,
including a check for references to undefined Kconfig symbols in source code.
See the ``KconfigCheck`` class.
- `Various utilities
<https://github.com/projectacrn/acrn-hypervisor/tree/master/scripts/kconfig>`_
from the `ACRN <https://projectacrn.org/>`_ project
These use the older Kconfiglib 1 API, which was clunkier and not as general
(functions instead of properties, no direct access to the menu structure or
properties, uglier ``__str__()`` output):
- `genboardscfg.py <http://git.denx.de/?p=u-boot.git;a=blob;f=tools/genboardscfg.py;hb=HEAD>`_ from `Das U-Boot <http://www.denx.de/wiki/U-Boot>`_ generates some sort of legacy board database by pulling information from a newly added Kconfig-based configuration system (as far as I understand it :).
- `gen-manual-lists.py <https://git.busybox.net/buildroot/tree/support/scripts/gen-manual-lists.py?id=5676a2deea896f38123b99781da0a612865adeb0>`_ generated listings for an appendix in the `Buildroot <https://buildroot.org>`_ manual. (The listing has since been removed.)
- `gen_kconfig_doc.py <https://github.com/espressif/esp-idf/blob/master/docs/gen-kconfig-doc.py>`_ from the `esp-idf <https://github.com/espressif/esp-idf>`_ project generates documentation from Kconfig files.
- `SConf <https://github.com/CoryXie/SConf>`_ builds an interactive configuration interface (like ``menuconfig``) on top of Kconfiglib, for use e.g. with `SCons <scons.org>`_.
- `kconfig-diff.py <https://gist.github.com/dubiousjim/5638961>`_ -- a script by `dubiousjim <https://github.com/dubiousjim>`_ that compares kernel configurations.
- Originally, Kconfiglib was used in chapter 4 of my `master's thesis <http://liu.diva-portal.org/smash/get/diva2:473038/FULLTEXT01.pdf>`_ to automatically generate a "minimal" kernel for a given system. Parts of it bother me a bit now, but that's how it goes with old work.
Sample ``make iscriptconfig`` session
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The following log should give some idea of the functionality available in the API:
.. code-block::
$ make iscriptconfig
A Kconfig instance 'kconf' for the architecture x86 has been created.
>>> kconf # Calls Kconfig.__repr__()
<configuration with 13711 symbols, main menu prompt "Linux/x86 4.14.0-rc7 Kernel Configuration", srctree ".", config symbol prefix "CONFIG_", warnings enabled, undef. symbol assignment warnings disabled>
>>> kconf.mainmenu_text # Expanded main menu text
'Linux/x86 4.14.0-rc7 Kernel Configuration'
>>> kconf.top_node # The implicit top-level menu
<menu node for menu, prompt "Linux/x86 4.14.0-rc7 Kernel Configuration" (visibility y), deps y, 'visible if' deps y, has child, Kconfig:5>
>>> kconf.top_node.list # First child menu node
<menu node for symbol SRCARCH, deps y, has next, Kconfig:7>
>>> print(kconf.top_node.list) # Calls MenuNode.__str__()
config SRCARCH
string
option env="SRCARCH"
default "x86"
>>> sym = kconf.top_node.list.next.item # Item contained in next menu node
>>> print(sym) # Calls Symbol.__str__()
config 64BIT
bool "64-bit kernel" if ARCH = "x86"
default ARCH != "i386"
help
Say yes to build a 64-bit kernel - formerly known as x86_64
Say no to build a 32-bit kernel - formerly known as i386
>>> sym # Calls Symbol.__repr__()
<symbol 64BIT, bool, "64-bit kernel", value y, visibility y, direct deps y, arch/x86/Kconfig:2>
>>> sym.assignable # Currently assignable values (0, 1, 2 = n, m, y)
(0, 2)
>>> sym.set_value(0) # Set it to n
True
>>> sym.tri_value # Check the new value
0
>>> sym = kconf.syms["X86_MPPARSE"] # Look up symbol by name
>>> print(sym)
config X86_MPPARSE
bool "Enable MPS table" if (ACPI || SFI) && X86_LOCAL_APIC
default y if X86_LOCAL_APIC
help
For old smp systems that do not have proper acpi support. Newer systems
(esp with 64bit cpus) with acpi support, MADT and DSDT will override it
>>> default = sym.defaults[0] # Fetch its first default
>>> sym = default[1] # Fetch the default's condition (just a Symbol here)
>>> print(sym)
config X86_LOCAL_APIC
bool
default y
select IRQ_DOMAIN_HIERARCHY
select PCI_MSI_IRQ_DOMAIN if PCI_MSI
depends on X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI
>>> sym.nodes # Show the MenuNode(s) associated with it
[<menu node for symbol X86_LOCAL_APIC, deps n, has next, arch/x86/Kconfig:1015>]
>>> kconfiglib.expr_str(sym.defaults[0][1]) # Print the default's condition
'X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI'
>>> kconfiglib.expr_value(sym.defaults[0][1]) # Evaluate it (0 = n)
0
>>> kconf.syms["64BIT"].set_value(2)
True
>>> kconfiglib.expr_value(sym.defaults[0][1]) # Evaluate it again (2 = y)
2
>>> kconf.write_config("myconfig") # Save a .config
>>> ^D
$ cat myconfig
# Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib)
CONFIG_64BIT=y
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_MMU=y
...
Test suite
----------
The test suite is run with
.. code::
$ python(3) Kconfiglib/testsuite.py
`pypy <https://pypy.org/>`_ works too, and is much speedier for everything except ``allnoconfig.py``/``allnoconfig_simpler.py``/``allyesconfig.py``, where it doesn't have time to warm up since
the scripts are run via ``make scriptconfig``.
The test suite must be run from the top-level kernel directory. It requires that the
Kconfiglib git repository has been cloned into it and that the makefile patch has been applied.
To get rid of warnings generated for the kernel ``Kconfig`` files, add ``2>/dev/null`` to the command to
discard ``stderr``.
**NOTE: Forgetting to apply the Makefile patch will cause some tests that compare generated configurations to fail**
**NOTE: The test suite overwrites .config in the kernel root, so make sure to back it up.**
The test suite consists of a set of selftests and a set of compatibility tests that
compare configurations generated by Kconfiglib with
configurations generated by the C tools, for a number of cases. See
`testsuite.py <https://github.com/ulfalizer/Kconfiglib/blob/master/testsuite.py>`_
for the available options.
The `tests/reltest <https://github.com/ulfalizer/Kconfiglib/blob/master/tests/reltest>`_ script runs the test suite
and all the example scripts for both Python 2 and Python 3, verifying that everything works.
Rarely, the output from the C tools is changed slightly (most recently due to a
`change <https://www.spinics.net/lists/linux-kbuild/msg17074.html>`_ I added).
If you get test suite failures, try running the test suite again against the
`linux-next tree <https://www.kernel.org/doc/man-pages/linux-next.html>`_,
which has all the latest changes. I will make it clear if any
non-backwards-compatible changes appear.
A lot of time is spent waiting around for ``make`` and the C utilities (which need to reparse all the
Kconfig files for each defconfig test). Adding some multiprocessing to the test suite would make sense
too.
Notes
-----
* This is version 2 of Kconfiglib, which is not backwards-compatible with
Kconfiglib 1. A summary of changes between Kconfiglib 1 and Kconfiglib
2 can be found `here
<https://github.com/ulfalizer/Kconfiglib/blob/screenshots/kconfiglib-2-changes.txt>`__.
* I sometimes see people add custom output formats, which is pretty
straightforward to do (see the implementations of ``write_autoconf()`` and
``write_config()`` for a template, and also the documentation of the
``Symbol.config_string`` property). If you come up with something you think
might be useful to other people, I'm happy to take it in upstream. Batteries
included and all that.
* Kconfiglib assumes the modules symbol is ``MODULES``, which is backwards-compatible.
A warning is printed by default if ``option modules`` is set on some other symbol.
Let me know if you need proper ``option modules`` support. It wouldn't be that
hard to add.
Thanks
------
- To `RomaVis <https://github.com/RomaVis>`_, for making
`pymenuconfig <https://github.com/RomaVis/pymenuconfig>`_ and suggesting
the ``rsource`` keyword.
- To `Mitja Horvat <https://github.com/pinkfluid>`_, for adding support
for user-defined styles to the terminal menuconfig.
- To `Philip Craig <https://github.com/philipc>`_ for adding
support for the ``allnoconfig_y`` option and fixing an obscure issue
with ``comment``\s inside ``choice``\s (that didn't affect correctness but
made outputs differ). ``allnoconfig_y`` is used to force certain symbols
to ``y`` during ``make allnoconfig`` to improve coverage.
License
-------
See `LICENSE.txt <https://github.com/ulfalizer/Kconfiglib/blob/master/LICENSE.txt>`_. SPDX license identifiers are used in the
source code.

27
ext/Kconfiglib/alldefconfig.py Executable file
View File

@@ -0,0 +1,27 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2019, Ulf Magnusson
# SPDX-License-Identifier: ISC
"""
Writes a configuration file where all symbols are set to their their default
values.
The default output filename is '.config'. A different filename can be passed in
the KCONFIG_CONFIG environment variable.
Usage for the Linux kernel:
$ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/alldefconfig.py
"""
import kconfiglib
def main():
kconf = kconfiglib.standard_kconfig(__doc__)
kconf.load_allconfig("alldef.config")
print(kconf.write_config())
if __name__ == "__main__":
main()

46
ext/Kconfiglib/allmodconfig.py Executable file
View File

@@ -0,0 +1,46 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2019, Ulf Magnusson
# SPDX-License-Identifier: ISC
"""
Writes a configuration file where as many symbols as possible are set to 'm'.
The default output filename is '.config'. A different filename can be passed
in the KCONFIG_CONFIG environment variable.
Usage for the Linux kernel:
$ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/allmodconfig.py
"""
import kconfiglib
def main():
kconf = kconfiglib.standard_kconfig(__doc__)
# See allnoconfig.py
kconf.warn = False
for sym in kconf.unique_defined_syms:
if sym.orig_type == kconfiglib.BOOL:
# 'bool' choice symbols get their default value, as determined by
# e.g. 'default's on the choice
if not sym.choice:
# All other bool symbols get set to 'y', like for allyesconfig
sym.set_value(2)
elif sym.orig_type == kconfiglib.TRISTATE:
sym.set_value(1)
for choice in kconf.unique_choices:
choice.set_value(2 if choice.orig_type == kconfiglib.BOOL else 1)
kconf.warn = True
kconf.load_allconfig("allmod.config")
print(kconf.write_config())
if __name__ == "__main__":
main()

45
ext/Kconfiglib/allnoconfig.py Executable file
View File

@@ -0,0 +1,45 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2019, Ulf Magnusson
# SPDX-License-Identifier: ISC
"""
Writes a configuration file where as many symbols as possible are set to 'n'.
The default output filename is '.config'. A different filename can be passed
in the KCONFIG_CONFIG environment variable.
Usage for the Linux kernel:
$ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/allnoconfig.py
"""
# See examples/allnoconfig_walk.py for another way to implement this script
import kconfiglib
def main():
kconf = kconfiglib.standard_kconfig(__doc__)
# Avoid warnings that would otherwise get printed by Kconfiglib for the
# following:
#
# 1. Assigning a value to a symbol without a prompt, which never has any
# effect
#
# 2. Assigning values invalid for the type (only bool/tristate symbols
# accept 0/1/2, for n/m/y). The assignments will be ignored for other
# symbol types, which is what we want.
kconf.warn = False
for sym in kconf.unique_defined_syms:
sym.set_value(2 if sym.is_allnoconfig_y else 0)
kconf.warn = True
kconf.load_allconfig("allno.config")
print(kconf.write_config())
if __name__ == "__main__":
main()

56
ext/Kconfiglib/allyesconfig.py Executable file
View File

@@ -0,0 +1,56 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2019, Ulf Magnusson
# SPDX-License-Identifier: ISC
"""
Writes a configuration file where as many symbols as possible are set to 'y'.
The default output filename is '.config'. A different filename can be passed
in the KCONFIG_CONFIG environment variable.
Usage for the Linux kernel:
$ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/allyesconfig.py
"""
import kconfiglib
def main():
kconf = kconfiglib.standard_kconfig(__doc__)
# See allnoconfig.py
kconf.warn = False
# Try to set all symbols to 'y'. Dependencies might truncate the value down
# later, but this will at least give the highest possible value.
#
# Assigning 0/1/2 to non-bool/tristate symbols has no effect (int/hex
# symbols still take a string, because they preserve formatting).
for sym in kconf.unique_defined_syms:
# Set choice symbols to 'm'. This value will be ignored for choices in
# 'y' mode (the "normal" mode), which will instead just get their
# default selection, but will set all symbols in m-mode choices to 'm',
# which is as high as they can go.
#
# Here's a convoluted example of how you might get an m-mode choice
# even during allyesconfig:
#
# choice
# tristate "weird choice"
# depends on m
sym.set_value(1 if sym.choice else 2)
# Set all choices to the highest possible mode
for choice in kconf.unique_choices:
choice.set_value(2)
kconf.warn = True
kconf.load_allconfig("allyes.config")
print(kconf.write_config())
if __name__ == "__main__":
main()

43
ext/Kconfiglib/defconfig.py Executable file
View File

@@ -0,0 +1,43 @@
#!/usr/bin/env python3
# Copyright (c) 2019, Ulf Magnusson
# SPDX-License-Identifier: ISC
"""
Reads a specified configuration file, then writes a new configuration file.
This can be used to initialize the configuration from e.g. an arch-specific
configuration file. This input configuration file would usually be a minimal
configuration file, as generated by e.g. savedefconfig.
The default output filename is '.config'. A different filename can be passed in
the KCONFIG_CONFIG environment variable.
"""
import argparse
import kconfiglib
def main():
parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter,
description=__doc__)
parser.add_argument(
"--kconfig",
default="Kconfig",
help="Top-level Kconfig file (default: Kconfig)")
parser.add_argument(
"config",
metavar="CONFIGURATION",
help="Input configuration file")
args = parser.parse_args()
kconf = kconfiglib.Kconfig(args.kconfig, suppress_traceback=True)
print(kconf.load_config(args.config))
print(kconf.write_config())
if __name__ == "__main__":
main()

102
ext/Kconfiglib/examples/Kmenuconfig Normal file
View File

@@ -0,0 +1,102 @@
mainmenu "Example Kconfig configuration"
config MODULES
bool "Enable loadable module support"
option modules
default y
menu "Bool and tristate symbols"
config BOOL
bool "Bool symbol"
default y
config BOOL_DEP
bool "Dependent bool symbol"
depends on BOOL
# Mix it up a bit with an 'if' instead of a 'depends on'
if BOOL
config TRI_DEP
tristate "Dependent tristate symbol"
select SELECTED_BY_TRI_DEP
imply IMPLIED_BY_TRI_DEP
endif
config TWO_MENU_NODES
bool "First prompt"
depends on BOOL
config TRI
tristate "Tristate symbol"
config TWO_MENU_NODES
bool "Second prompt"
comment "These are selected by TRI_DEP"
config SELECTED_BY_TRI_DEP
tristate "Tristate selected by TRI_DEP"
config IMPLIED_BY_TRI_DEP
tristate "Tristate implied by TRI_DEP"
endmenu
menu "String, int, and hex symbols"
config STRING
string "String symbol"
default "foo"
config INT
int "Int symbol"
default 747
config HEX
hex "Hex symbol"
default 0xABC
endmenu
menu "Various choices"
choice BOOL_CHOICE
bool "Bool choice"
config BOOL_CHOICE_SYM_1
bool "Bool choice sym 1"
config BOOL_CHOICE_SYM_2
bool "Bool choice sym 2"
endchoice
choice TRI_CHOICE
tristate "Tristate choice"
config TRI_CHOICE_SYM_1
tristate "Tristate choice sym 1"
config TRI_CHOICE_SYM_2
tristate "Tristate choice sym 2"
endchoice
choice OPT_BOOL_CHOICE
bool "Optional bool choice"
optional
config OPT_BOOL_CHOICE_SYM_1
bool "Optional bool choice sym 1"
config OPT_BOOL_CHOICE_SYM_2
bool "Optional bool choice sym 2"
endchoice
endmenu

66
ext/Kconfiglib/examples/allnoconfig_walk.py Normal file
View File

@@ -0,0 +1,66 @@
# This is tree-walking version of allnoconfig.py, for demonstration purposes.
# Verified by the test suite to generate identical output to 'make allnoconfig'
# for all ARCHes.
#
# Note: A more practical version would use Kconfig.node_iter(). The manual tree
# walking is for demonstration purposes.
#
# Usage for the Linux kernel:
#
# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/allnoconfig_walk.py
import sys
from kconfiglib import Kconfig, Symbol
def do_allnoconfig(node):
global changed
# Walk the tree of menu nodes. You can imagine this as going down/into menu
# entries in the menuconfig interface, setting each to n (or the lowest
# assignable value).
while node:
if isinstance(node.item, Symbol):
sym = node.item
# Is the symbol a non-allnoconfig_y symbol that can be set to a
# lower value than its current value?
if (not sym.is_allnoconfig_y and
sym.assignable and
sym.assignable[0] < sym.tri_value):
# Yup, lower it
sym.set_value(sym.assignable[0])
changed = True
# Recursively lower children
if node.list:
do_allnoconfig(node.list)
node = node.next
# Parse the Kconfig files
kconf = Kconfig(sys.argv[1])
# Do an initial pass to set 'option allnoconfig_y' symbols to y
for sym in kconf.unique_defined_syms:
if sym.is_allnoconfig_y:
sym.set_value(2)
while True:
# Changing later symbols in the configuration can sometimes allow earlier
# symbols to be lowered, e.g. if a later symbol 'select's an earlier
# symbol. To handle such situations, we do additional passes over the tree
# until we're no longer able to change the value of any symbol in a pass.
changed = False
do_allnoconfig(kconf.top_node)
# Did the pass change any symbols?
if not changed:
break
print(kconf.write_config())

39
ext/Kconfiglib/examples/defconfig_oldconfig.py Normal file
View File

@@ -0,0 +1,39 @@
# Produces exactly the same output as the following script:
#
# make defconfig
# echo CONFIG_ETHERNET=n >> .config
# make oldconfig
# echo CONFIG_ETHERNET=y >> .config
# yes n | make oldconfig
#
# This came up in https://github.com/ulfalizer/Kconfiglib/issues/15.
#
# Usage:
#
# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/defconfig_oldconfig.py
import sys
import kconfiglib
kconf = kconfiglib.Kconfig(sys.argv[1])
# Mirrors defconfig
kconf.load_config("arch/x86/configs/x86_64_defconfig")
kconf.write_config()
# Mirrors the first oldconfig
kconf.load_config()
kconf.syms["ETHERNET"].set_value(0)
kconf.write_config()
# Mirrors the second oldconfig
kconf.load_config()
kconf.syms["ETHERNET"].set_value(2)
for s in kconf.unique_defined_syms:
if s.user_value is None and 0 in s.assignable:
s.set_value(0)
# Write the final configuration
print(kconf.write_config())

15
ext/Kconfiglib/examples/dumpvars.py Normal file
View File

@@ -0,0 +1,15 @@
# Prints all (set) environment variables referenced in the Kconfig files
# together with their values, as a list of assignments.
#
# Note: This only works for environment variables referenced via the $(FOO)
# preprocessor syntax. The older $FOO syntax is maintained for backwards
# compatibility.
import os
import sys
import kconfiglib
print(" ".join("{}='{}'".format(var, os.environ[var])
for var in kconfiglib.Kconfig(sys.argv[1]).env_vars))

24
ext/Kconfiglib/examples/eval_expr.py Normal file
View File

@@ -0,0 +1,24 @@
# Evaluates an expression (e.g. "X86_64 || (X86_32 && X86_LOCAL_APIC)") in the
# context of a configuration. Note that this always yields a tristate value (n,
# m, or y).
#
# Usage:
#
# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/eval_expr.py SCRIPT_ARG=<expr>
import sys
import kconfiglib
if len(sys.argv) < 3:
sys.exit("Pass the expression to evaluate with SCRIPT_ARG=<expression>")
kconf = kconfiglib.Kconfig(sys.argv[1])
expr = sys.argv[2]
# Enable modules so that m doesn't get demoted to n
kconf.modules.set_value(2)
print("the expression '{}' evaluates to {}"
.format(expr, kconf.eval_string(expr)))

112
ext/Kconfiglib/examples/find_symbol.py Normal file
View File

@@ -0,0 +1,112 @@
# Prints all menu nodes that reference a given symbol any of their properties
# or property conditions, along with their parent menu nodes.
#
# Usage:
#
# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/find_symbol.py SCRIPT_ARG=<name>
#
# Example output for SCRIPT_ARG=X86:
#
# Found 470 locations that reference X86:
#
# ========== Location 1 (init/Kconfig:1108) ==========
#
# config SGETMASK_SYSCALL
# bool
# prompt "sgetmask/ssetmask syscalls support" if EXPERT
# default PARISC || M68K || PPC || MIPS || X86 || SPARC || MICROBLAZE || SUPERH
# help
# sys_sgetmask and sys_ssetmask are obsolete system calls
# no longer supported in libc but still enabled by default in some
# architectures.
#
# If unsure, leave the default option here.
#
# ---------- Parent 1 (init/Kconfig:1077) ----------
#
# menuconfig EXPERT
# bool
# prompt "Configure standard kernel features (expert users)"
# select DEBUG_KERNEL
# help
# This option allows certain base kernel options and settings
# to be disabled or tweaked. This is for specialized
# environments which can tolerate a "non-standard" kernel.
# Only use this if you really know what you are doing.
#
# ---------- Parent 2 (init/Kconfig:39) ----------
#
# menu "General setup"
#
# ========== Location 2 (arch/Kconfig:29) ==========
#
# config OPROFILE_EVENT_MULTIPLEX
# bool
# prompt "OProfile multiplexing support (EXPERIMENTAL)"
# default "n"
# depends on OPROFILE && X86
# help
# The number of hardware counters is limited. The multiplexing
# feature enables OProfile to gather more events than counters
# are provided by the hardware. This is realized by switching
# between events at a user specified time interval.
#
# If unsure, say N.
#
# ---------- Parent 1 (arch/Kconfig:16) ----------
#
# config OPROFILE
# tristate
# prompt "OProfile system profiling"
# select RING_BUFFER
# select RING_BUFFER_ALLOW_SWAP
# depends on PROFILING && HAVE_OPROFILE
# help
# OProfile is a profiling system capable of profiling the
# whole system, include the kernel, kernel modules, libraries,
# and applications.
#
# If unsure, say N.
#
# ---------- Parent 2 (init/Kconfig:39) ----------
#
# menu "General setup"
#
# ... (tons more)
import sys
import kconfiglib
if len(sys.argv) < 3:
sys.exit('Pass symbol name (without "CONFIG_" prefix) with SCRIPT_ARG=<name>')
kconf = kconfiglib.Kconfig(sys.argv[1])
sym_name = sys.argv[2]
if sym_name not in kconf.syms:
print("No symbol {} exists in the configuration".format(sym_name))
sys.exit(0)
referencing = [node for node in kconf.node_iter()
if kconf.syms[sym_name] in node.referenced]
if not referencing:
print("No references to {} found".format(sym_name))
sys.exit(0)
print("Found {} locations that reference {}:\n"
.format(len(referencing), sym_name))
for i, node in enumerate(referencing, 1):
print("========== Location {} ({}:{}) ==========\n\n{}"
.format(i, node.filename, node.linenr, node))
# Print the parents of the menu node too
node = node.parent
parent_i = 1
while node is not kconf.top_node:
print("---------- Parent {} ({}:{}) ----------\n\n{}"
.format(parent_i, node.filename, node.linenr, node))
node = node.parent
parent_i += 1

Some files were not shown because too many files have changed in this diff Show More