In gem5, there are many equally valid and equally useful top level
targets which the user might want. It no longer makes sense to
arbitrarily pick one to be the default target. It makes sense to force
the user to actually specify what they want, instead of assuming it
must be the ARM debug binary.
There is currently an M5_DEFAULT_BINARY environment variable which
will change what the default binary is, if set. This change leaves
that in place, but removes the default-default, or in other words the
default that is used if M5_DEFAULT_BINARY is not set.
This way if the user knows what default they want, they can specify it
locally in their environment and avoid having to type it over and over
again, but we're not making an arbitrary choice at a more global level
without the context to know what actually makes sense.
Change-Id: I886adb1289b9879d53387250f950909a4809ed8b
These two utilities help update an old config to add settings for new
config options. The difference between them is that oldconfig asks what
new settings you want to use, while olddefconfig automatically picks the
defaults.
Change-Id: Icd3e57f834684e620705beb884faa5b6e2cc7baa
This helper utility lets you save the defconfig which would give rise to
a given config. For instance, you could use menuconfig to set up a
config how you want it with the options you cared about configured, and
then use savedefconfig to save a defconfig of that somewhere to the
side, in the gem5 defconfig directory, etc. Then later, you could use
that defconfig to set up a new build directory with that same config,
even if the kconfig options have changed a little bit since then.
A saved defconfig like that can also be a good way to visually see what
options have been set to something interesting, and an easier way to
pass a config to someone else to use, to put in bug reports, etc.
Change-Id: Ifd344278638c59b48c261b36058832034c009c78
This helper lists config options which are new in the Kconfig and which
are not currently set in the config file.
Change-Id: I0c426d85c0cf0d2bdbac599845669165285a82a0
This little utility lets you set particular values in an existing config
without having to open up the whole menuconfig interface.
Also reorganize things in kconfig.py a little to help share code between
wrappers.
Change-Id: I7cba0c0ef8d318d6c39e49c779ebb2bbdc3d94c8
This will let you specify *any* defconfig file, instead of implicitly
selecting one from the defconfig directory based on the variant name.
Change-Id: I74c981b206849f08e60c2df702c06534c670cc7c
If you call scons with the fist argument set to menuconfig, that means
to run menuconfig on the path following it. Or in other words, if
you ran this command:
scons menuconfig build/foo/bar
That would tell SCons to set up a build directory at the path
build/foo/bar, and then invoke menuconfig so you can set up its
configuration.
In addition to using this mechanism to set up a new build directory, you
can also use it to reconfigure an existing directory.
This supplements and does not replace the existing mechanism of using
"build/${VARIANT}" to select a config with defconfig.
Change-Id: Ief8e8c2ee6477799455c2004bef06c64be5cc1db
These targets are not necessarily obvious, and tell SCons to do useful
things, like build a particular version of the gem5 binary with a
particular configuration, or run the unit tests.
Add descriptions of these targets to the help so that they are much
more discoverable.
Change-Id: If84399be1a7155ff5f66f511efe1f1c241089c84
These are not yet consumed by anything, but convert all the settings
from SCons variables to Kconfig variables.
If you have existing SConsopts files which need to be converted, you
should take a look at KCONFIG.md to learn about how kconfig is used in
gem5. You should decide if any variables need to be available to C++ or
kconfig itself, and whether those are options which should be detected
automatically, or should be up to the user. Options which should be
measured automatically should still be in SConsopts files, while user
facing options should be added to new or existing Kconfig files.
Generally, make sure you're storing c++/kconfig visible options in
env['CONF'][...]. Also remove references to sticky_vars since persistent
options should now be handled with kconfig, and export_vars since
everything in env['CONF'] is now exported automatically.
Switch SCons/gem5 to use Kconfig for configuration, except EXTRAS which
is still a sticky SCons variable. This is necessary because EXTRAS also
controls what config options exist. If it came from Kconfig itself, then
there would be a circular dependency. This dependency could
theoretically be handled by reparsing the Kconfig when EXTRAS
directories were added or removed, but that would be complicated, and
isn't supported by kconfiglib. It wouldn't be worth the significant
effort it would take to add it, just to use Kconfig more purely.
Change-Id: I29ab1940b2d7b0e6635a490452d05befe5b4a2c9
If gem5.build already exists within a directory, then that build
directory can be used without having to worry about variants.
If it doesn't exist and we find a build/${VARIANT} style path, then we
use that as the anchor.
In either case, the variant name is the final component of the build
path. The parse_build_path function had been separating that out, but it
was just put back onto the path again anyway by the only caller, and
then split out again when that path was consumed. We save a step by not
splitting it out in parse_build_path.
Change-Id: I8705b3dbb7664748f5525869cb188df70319d403
The scons function Detect will return the program name if the program
is exists in the system. However, the HAVE_PKG_CONFIG is used to
check the pkg-config program is exists and it should be the boolean
type.
Change-Id: I18c4813d36eea68b8851a41db41777bdb2a80b7b
To use mold linker with gcc of version older than 12.1.0, the user
has to pass the -B option to specify where the linker is. [1]
Currently in gem5, scons only looks for the mold binary at
conventional places, such as /usr/libexec/mold and
/usr/local/libexec/mold. There's no option to manually specify
the path to the linker.
gcc-12 and mold are not widely available on older systems. Having
an option to manually input the mold linker path allows users to use
built mold instances anywhere on the system, not just the default
locations in /usr where they may not have permission to install mold
(i.e., no sudo permissions).
[1] https://github.com/rui314/mold#how-to-use
Change-Id: Ifb2366a0c2342bf4e7207df8db6196e14184a9d4
Signed-off-by: Hoa Nguyen <hn@hnpl.org>
* scons: Fix sanitizer lib link for clang
Change-Id: I2441466c5c9343afd938185b8ec5047d4e95ac70
* scons: Statically link libubsan when using sanitizers with gcc
Change-Id: I362a1fb87771454ad94e439847a85d19108f375a
---------
Co-authored-by: Gabriel Busnot <gabriel.busnot@arteris.com>
The code generation in gem5's build system requires the use of Regular
Expression flags when defining the regular expressions used for
tokenization. However, the Python Lex-Yacc (PLY) [1] library used by
gem5 does not allow the user sufficient control of the flags for RE
compilation.
Previously, gem5 used inline RE flags to control RE compilation.
However, from Python 3.11, inline RE flags must be at the start of the
RE string. Because PLY wraps the user supplied RE strings before
compilation, there is no way for the user to supply a RE string with
the inline flag at the start. This makes gem5 incompatible with Python
3.11 when using PLY.
This change modifies gem5's build files to patch `re.compile` with a
wrapped version that can handle embedded flags anywhere in the RE
string, for all current versions of Python. The patched version
re-formats the user supplied RE string to convert inline RE flags to
explicit RE flags.
This patch is intended as a temporary stop-gap until PLY can be fixed
upstream.
See the gem5 Issue Tracker [2] for more details.
[1] https://github.com/dabeaz/ply
[2] https://gem5.atlassian.net/browse/GEM5-1321
Change-Id: I3ab371f2e5cf267c0a89caaf8a2bacfed78545ef
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/70237
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Boris Shingarov <shingarov@labware.com>
Tested-by: kokoro <noreply+kokoro@google.com>
A flag, introduced in this patch:
https://gem5-review.googlesource.com/c/public/gem5/+/68518
allowed users to pass "no-duplicate-sources" to a gem5 compilation to
not symlink sources in the build directory.
In this patch "src" was added as a shared top-level header directory.
This means that the header files are not copied to the "build" directory
whether or not "no-duplicate-sources" is set.
This patch ensures the "src" directory is only added as a shared
top-level headers directory in the case where "no-duplicate-sources" is
set.
In addition, the "duplicate_sources" parameter (the destination for the
"no-duplicate-sources") was "None" by default, and only set to False
when the flag was used. `default=True` has been added so
"duplicate_sources" can be used as a boolean.
This bug was a cause of a Nightly build error:
https://jenkins.gem5.org/job/nightly/570
In this error, building ext/sst resulted in an error as the Makefile
depends on adding "build/RISCV" to the include path. Without the header
files in the "build" directory, building SST failed. Though, ext/stt
should probably not be using header files in the "build/RISCV"
directory. This will be fixed in another change.
Change-Id: I786486a177fe17a67f3b939c539eecdcbfcaeaf2
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/69717
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
This commit is in response to a failure in the compiler tests:
https://jenkins.gem5.org/job/compiler-checks/570/
GCC versions <9 failed to compile gem5 with the following error:
```
socket.cc:(.text+0x32d5): undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
socket.cc:(.text+0x32e5): undefined reference to `std::filesystem::create_directory(std::filesystem::__cxx11::path const&, std::error_code&)'
socket.cc:(.text+0x3370): undefined reference to `std::filesystem::current_path[abi:cxx11](std::error_code&)'
socket.cc:(.text+0x33cc): undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
socket.cc:(.text+0x33d7): undefined reference to `std::filesystem::current_path(std::filesystem::__cxx11::path const&, std::error_code&)'
socket.cc:(.text+0x3468): undefined reference to `std::filesystem::current_path(std::filesystem::__cxx11::path const&, std::error_code&)'
```
This was due to this patch:
https://gem5-review.googlesource.com/c/public/gem5/+/69164 which
utilized `std::filesystem`. Prior to to GCC v9, explicit linking with
the stdc++fs library is required due to `std::filesystem` being an
experimental implementation prior to GGC v9.
Change-Id: I584e29f100cb59b40bd155c212e1814f6d8fbb99
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/69597
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
This adds a new scons flag --no-duplicate-sources to build without
linking source files to the build directory.
I find this very helpful when using CLion, since I can now generate a
compilation database using
`bear scons build/ALL/gem5.debug --no-duplicate-sources` and CLion will
now correctly semantically analyze all the files inside src/.
It also ensures that clicking on a build warning/error now opens the
real source file rather than a symlink.
This is not enabled by default since it's possible that certain use
cases are not working correctly, but the basic testing I've done so
far appears to work just fine.
It appears that with this change the `<root>/src` directory is no longer
added to `PYTHONPATH` when running `tests/main.py`, so this change
depends on https://gem5-review.git.corp.google.com/c/public/gem5/+/68757
Change-Id: Iddc9bf9c8211e68e5432c0a07f5c95f427c1ca16
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/68518
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
The asan(enable with --with-asan) sanitizer interpret calls to dlopen().
That replaces the RUNPATH of an executable with RUNPATH of libasan.so
after libasan.so is loaded by loader. Then it may cause some shared
libraries missing, i.e. the error messages is like
"cannot open shared object file: No such file or directory" since the
RUNPATH is no longer correct. Force the libasan to static linking
can avoid this issue since libasan.a does not have a RUNPATH, thus
the replacement will never happen.
Change-Id: I8e5ff4d1fbe4644a258054be6e9f6d4db9062e56
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67298
Reviewed-by: Earl Ou <shunhsingou@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Sanitizers can enlarge binary size drammatically, north of 2GB. This
can prevent successful linkage due to symbol relocation outside from
the 2GB region allocated by the small x86_64 code model that is
enabled by default (32-bit relative offset limitation). Switching to
the medium model in x86_64 enables 64-bit relative offset for large
objects (>64KB by default) while sticking to 32-bit relative
addressing for code and smaller objects. Note this comes at a
potential performance cost so it should not be enabled in all cases.
This should still be a very happy medium for non-perf-critical
sanitized builds.
Jira issue: https://gem5.atlassian.net/browse/GEM5-1313
Change-Id: I9aceacfcda99cc29c8fb24b7c69aaab019ce97fd
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67451
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
-Wno-free-nonheap-object can happen at compile or link time depending on
the versions. To better disable this false alarm, we move the memory
management part into .cc file, so the check is always done at link time.
This change also removes the global flags so other code is still checked
with the flags.
Change-Id: I8f1e20197b25c90b5f439e2ecc474bd99e4f82ed
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67237
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Yu-hsin Wang <yuhsingw@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
This makes what are configuration and what are internal SCons variables
explicit and separate, and makes it unnecessary to call out what
variables to export to C++.
These variables will also be plumbed into and out of kconfiglib in later
changes.
Change-Id: Iaf5e098d7404af06285c421dbdf8ef4171b3f001
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56892
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The SConsopts files that need to be processed is affected by the EXTRAS
setting, and if all settings are going to be local to the variants, that
means each variant can have a different set of SConsopts files to use.
This does result in Configure type output showing up repeatedly when
building multiple variants, but that's uncommon for users, and will
hopefully become unnecessary for developers over time.
Change-Id: I7df287c37fc32e91576dced80d0a9e56806c4581
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56751
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
These are no longer split out and shared in the root build/ directory.
This does result in a small amount of overhead from building redundant
copies of these files, although the overhead is not significant. When
building 7 different variants of gem5, all the ISAs and NULL, the
difference on my machine was:
Before:
real 41m25.372s
user 914m22.266s
sys 41m51.816s
After:
real 42m38.074s
user 921m36.852s
sys 43m2.949s
This is about a 2-3% difference, which is a worse than typical case,
since the overhead scales with the number of variants being built.
The benefit of pulling ext/ into the variant directory is that there can
now be a single config which applies to all files used to build gem5,
and that config is represented by the variant of gem5 being built.
Change-Id: I6f0db97c63a7f3e252e7e351aa862340978e701b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56750
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gabe Black <gabe.black@gmail.com>
This new and improved linker approximately halves link time for
build/X86/gem5.opt for me compared to lld, the previously fastest
linker. The -fuse-ld=mold option is supported by current versions of
clang, and will be supported in gcc 12.
To use the mold linker with gcc prior to version 12, you need to set
the LINKFLAGS_EXTRA scons variable to pass in a -B option as described
on this page:
https://github.com/rui314/mold
Change-Id: Ic5ad0e532fac078d384a0aebb3e04a4b04ce4880
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/57173
Maintainer: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Earl Ou <shunhsingou@google.com>
Whether to honor the USE_KVM setting can be handled in a more targeted
way down at individual consumers. They will know what ISA the host needs
to be for their particular model to make sense, where the SConstruct has
to enable or disable it for the entire build of gem5.
Change-Id: I9e82e232203834f6fe56d0ce5cf696809a0970bc
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56188
Maintainer: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
These variables will always be imported from the environment, and never
from previous builds. For SCons to actually use these values, they need
to not only be in the environment variables external commands SCons runs
sees, they also have to be promoted to actual SCons construction
variables.
Change-Id: I54035442d70972396f1788bd0773877222d7a7c5
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56386
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
There is currently a "--gold-linker" option, which tells SCons to use
the gold linker. This puts the gold linker in a special position where
it has its own command line option, and makes it impossible to
explicitly select any other linker.
This change adds a --linker option which can be set to "bfd" (the
default gnu linker), "gold" (the gnu gold linker), or "lld" which is the
clang linker. If none is specified, SCons will let the compiler fall
back to whatever its default is.
The --gold-linker option is modified so that it just sets the --linker
option for you. Because of how SCons handles adding command line
options, there isn't a good way to complain if both it and --linker is
used, nor is there a good way to warn that the user is using a
deprecated command line option. A warning is in the help text for that
option now, though.
If the gold linker is selected, this change also sets the --threads
option, and sets the --thread-count option. This tells the gold linker
to run multi-threaded to speed up linking. Apparently the gold linker
must have been built with an option that enables threading for these
options to have any effect. If threads and LTO are both enabled, the
gold linker segfaults. To avoid that problem, we don't add --threads if
LTO has been enabled.
This change also enables a new configuration, where lld, the clang
linker, can be used with gcc. Unfortunately the format of LTO
information gcc/g++ generates is not compatible with lld, and so if you
set the linker to lld and enable LTO, you'll probably get errors like
not being able to find the symbol "main".
This change also allows you to select the default gnu linker ("bfd")
when using clang, although it's not clear why you would want to do that.
Change-Id: Ib37030431527f5d0bc626e9594b2e1142df889be
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/49003
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
