To use mold linker with gcc of version older than 12.1.0, the user
has to pass the -B option to specify where the linker is. [1]
Currently in gem5, scons only looks for the mold binary at
conventional places, such as /usr/libexec/mold and
/usr/local/libexec/mold. There's no option to manually specify
the path to the linker.
gcc-12 and mold are not widely available on older systems. Having
an option to manually input the mold linker path allows users to use
built mold instances anywhere on the system, not just the default
locations in /usr where they may not have permission to install mold
(i.e., no sudo permissions).
[1] https://github.com/rui314/mold#how-to-use
Change-Id: Ifb2366a0c2342bf4e7207df8db6196e14184a9d4
Signed-off-by: Hoa Nguyen <hn@hnpl.org>
* scons: Fix sanitizer lib link for clang
Change-Id: I2441466c5c9343afd938185b8ec5047d4e95ac70
* scons: Statically link libubsan when using sanitizers with gcc
Change-Id: I362a1fb87771454ad94e439847a85d19108f375a
---------
Co-authored-by: Gabriel Busnot <gabriel.busnot@arteris.com>
The code generation in gem5's build system requires the use of Regular
Expression flags when defining the regular expressions used for
tokenization. However, the Python Lex-Yacc (PLY) [1] library used by
gem5 does not allow the user sufficient control of the flags for RE
compilation.
Previously, gem5 used inline RE flags to control RE compilation.
However, from Python 3.11, inline RE flags must be at the start of the
RE string. Because PLY wraps the user supplied RE strings before
compilation, there is no way for the user to supply a RE string with
the inline flag at the start. This makes gem5 incompatible with Python
3.11 when using PLY.
This change modifies gem5's build files to patch `re.compile` with a
wrapped version that can handle embedded flags anywhere in the RE
string, for all current versions of Python. The patched version
re-formats the user supplied RE string to convert inline RE flags to
explicit RE flags.
This patch is intended as a temporary stop-gap until PLY can be fixed
upstream.
See the gem5 Issue Tracker [2] for more details.
[1] https://github.com/dabeaz/ply
[2] https://gem5.atlassian.net/browse/GEM5-1321
Change-Id: I3ab371f2e5cf267c0a89caaf8a2bacfed78545ef
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/70237
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Boris Shingarov <shingarov@labware.com>
Tested-by: kokoro <noreply+kokoro@google.com>
A flag, introduced in this patch:
https://gem5-review.googlesource.com/c/public/gem5/+/68518
allowed users to pass "no-duplicate-sources" to a gem5 compilation to
not symlink sources in the build directory.
In this patch "src" was added as a shared top-level header directory.
This means that the header files are not copied to the "build" directory
whether or not "no-duplicate-sources" is set.
This patch ensures the "src" directory is only added as a shared
top-level headers directory in the case where "no-duplicate-sources" is
set.
In addition, the "duplicate_sources" parameter (the destination for the
"no-duplicate-sources") was "None" by default, and only set to False
when the flag was used. `default=True` has been added so
"duplicate_sources" can be used as a boolean.
This bug was a cause of a Nightly build error:
https://jenkins.gem5.org/job/nightly/570
In this error, building ext/sst resulted in an error as the Makefile
depends on adding "build/RISCV" to the include path. Without the header
files in the "build" directory, building SST failed. Though, ext/stt
should probably not be using header files in the "build/RISCV"
directory. This will be fixed in another change.
Change-Id: I786486a177fe17a67f3b939c539eecdcbfcaeaf2
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/69717
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
This commit is in response to a failure in the compiler tests:
https://jenkins.gem5.org/job/compiler-checks/570/
GCC versions <9 failed to compile gem5 with the following error:
```
socket.cc:(.text+0x32d5): undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
socket.cc:(.text+0x32e5): undefined reference to `std::filesystem::create_directory(std::filesystem::__cxx11::path const&, std::error_code&)'
socket.cc:(.text+0x3370): undefined reference to `std::filesystem::current_path[abi:cxx11](std::error_code&)'
socket.cc:(.text+0x33cc): undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
socket.cc:(.text+0x33d7): undefined reference to `std::filesystem::current_path(std::filesystem::__cxx11::path const&, std::error_code&)'
socket.cc:(.text+0x3468): undefined reference to `std::filesystem::current_path(std::filesystem::__cxx11::path const&, std::error_code&)'
```
This was due to this patch:
https://gem5-review.googlesource.com/c/public/gem5/+/69164 which
utilized `std::filesystem`. Prior to to GCC v9, explicit linking with
the stdc++fs library is required due to `std::filesystem` being an
experimental implementation prior to GGC v9.
Change-Id: I584e29f100cb59b40bd155c212e1814f6d8fbb99
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/69597
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
This adds a new scons flag --no-duplicate-sources to build without
linking source files to the build directory.
I find this very helpful when using CLion, since I can now generate a
compilation database using
`bear scons build/ALL/gem5.debug --no-duplicate-sources` and CLion will
now correctly semantically analyze all the files inside src/.
It also ensures that clicking on a build warning/error now opens the
real source file rather than a symlink.
This is not enabled by default since it's possible that certain use
cases are not working correctly, but the basic testing I've done so
far appears to work just fine.
It appears that with this change the `<root>/src` directory is no longer
added to `PYTHONPATH` when running `tests/main.py`, so this change
depends on https://gem5-review.git.corp.google.com/c/public/gem5/+/68757
Change-Id: Iddc9bf9c8211e68e5432c0a07f5c95f427c1ca16
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/68518
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
The asan(enable with --with-asan) sanitizer interpret calls to dlopen().
That replaces the RUNPATH of an executable with RUNPATH of libasan.so
after libasan.so is loaded by loader. Then it may cause some shared
libraries missing, i.e. the error messages is like
"cannot open shared object file: No such file or directory" since the
RUNPATH is no longer correct. Force the libasan to static linking
can avoid this issue since libasan.a does not have a RUNPATH, thus
the replacement will never happen.
Change-Id: I8e5ff4d1fbe4644a258054be6e9f6d4db9062e56
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67298
Reviewed-by: Earl Ou <shunhsingou@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Sanitizers can enlarge binary size drammatically, north of 2GB. This
can prevent successful linkage due to symbol relocation outside from
the 2GB region allocated by the small x86_64 code model that is
enabled by default (32-bit relative offset limitation). Switching to
the medium model in x86_64 enables 64-bit relative offset for large
objects (>64KB by default) while sticking to 32-bit relative
addressing for code and smaller objects. Note this comes at a
potential performance cost so it should not be enabled in all cases.
This should still be a very happy medium for non-perf-critical
sanitized builds.
Jira issue: https://gem5.atlassian.net/browse/GEM5-1313
Change-Id: I9aceacfcda99cc29c8fb24b7c69aaab019ce97fd
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67451
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
-Wno-free-nonheap-object can happen at compile or link time depending on
the versions. To better disable this false alarm, we move the memory
management part into .cc file, so the check is always done at link time.
This change also removes the global flags so other code is still checked
with the flags.
Change-Id: I8f1e20197b25c90b5f439e2ecc474bd99e4f82ed
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/67237
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Yu-hsin Wang <yuhsingw@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
This makes what are configuration and what are internal SCons variables
explicit and separate, and makes it unnecessary to call out what
variables to export to C++.
These variables will also be plumbed into and out of kconfiglib in later
changes.
Change-Id: Iaf5e098d7404af06285c421dbdf8ef4171b3f001
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56892
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The SConsopts files that need to be processed is affected by the EXTRAS
setting, and if all settings are going to be local to the variants, that
means each variant can have a different set of SConsopts files to use.
This does result in Configure type output showing up repeatedly when
building multiple variants, but that's uncommon for users, and will
hopefully become unnecessary for developers over time.
Change-Id: I7df287c37fc32e91576dced80d0a9e56806c4581
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56751
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
These are no longer split out and shared in the root build/ directory.
This does result in a small amount of overhead from building redundant
copies of these files, although the overhead is not significant. When
building 7 different variants of gem5, all the ISAs and NULL, the
difference on my machine was:
Before:
real 41m25.372s
user 914m22.266s
sys 41m51.816s
After:
real 42m38.074s
user 921m36.852s
sys 43m2.949s
This is about a 2-3% difference, which is a worse than typical case,
since the overhead scales with the number of variants being built.
The benefit of pulling ext/ into the variant directory is that there can
now be a single config which applies to all files used to build gem5,
and that config is represented by the variant of gem5 being built.
Change-Id: I6f0db97c63a7f3e252e7e351aa862340978e701b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56750
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gabe Black <gabe.black@gmail.com>
This new and improved linker approximately halves link time for
build/X86/gem5.opt for me compared to lld, the previously fastest
linker. The -fuse-ld=mold option is supported by current versions of
clang, and will be supported in gcc 12.
To use the mold linker with gcc prior to version 12, you need to set
the LINKFLAGS_EXTRA scons variable to pass in a -B option as described
on this page:
https://github.com/rui314/mold
Change-Id: Ic5ad0e532fac078d384a0aebb3e04a4b04ce4880
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/57173
Maintainer: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Earl Ou <shunhsingou@google.com>
Whether to honor the USE_KVM setting can be handled in a more targeted
way down at individual consumers. They will know what ISA the host needs
to be for their particular model to make sense, where the SConstruct has
to enable or disable it for the entire build of gem5.
Change-Id: I9e82e232203834f6fe56d0ce5cf696809a0970bc
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56188
Maintainer: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
These variables will always be imported from the environment, and never
from previous builds. For SCons to actually use these values, they need
to not only be in the environment variables external commands SCons runs
sees, they also have to be promoted to actual SCons construction
variables.
Change-Id: I54035442d70972396f1788bd0773877222d7a7c5
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56386
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
There is currently a "--gold-linker" option, which tells SCons to use
the gold linker. This puts the gold linker in a special position where
it has its own command line option, and makes it impossible to
explicitly select any other linker.
This change adds a --linker option which can be set to "bfd" (the
default gnu linker), "gold" (the gnu gold linker), or "lld" which is the
clang linker. If none is specified, SCons will let the compiler fall
back to whatever its default is.
The --gold-linker option is modified so that it just sets the --linker
option for you. Because of how SCons handles adding command line
options, there isn't a good way to complain if both it and --linker is
used, nor is there a good way to warn that the user is using a
deprecated command line option. A warning is in the help text for that
option now, though.
If the gold linker is selected, this change also sets the --threads
option, and sets the --thread-count option. This tells the gold linker
to run multi-threaded to speed up linking. Apparently the gold linker
must have been built with an option that enables threading for these
options to have any effect. If threads and LTO are both enabled, the
gold linker segfaults. To avoid that problem, we don't add --threads if
LTO has been enabled.
This change also enables a new configuration, where lld, the clang
linker, can be used with gcc. Unfortunately the format of LTO
information gcc/g++ generates is not compatible with lld, and so if you
set the linker to lld and enable LTO, you'll probably get errors like
not being able to find the symbol "main".
This change also allows you to select the default gnu linker ("bfd")
when using clang, although it's not clear why you would want to do that.
Change-Id: Ib37030431527f5d0bc626e9594b2e1142df889be
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/49003
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
SCons does not use a variable called LDFLAGS, it uses one called
LINKFLAGS. Switch some errant uses to the correct name.
Also, adjust all the other variable names to use LINK, for consistency
and to avoid confusion and avoid mistakes in the future.
Change-Id: I38d40f5231afdf62bcfba04478d403d65e9b1e26
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/51987
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The new c++ wrapper is called gem5py, and will run any python script
using gem5's embedded python interpreter. The "marshal" functionality is
split out into a separate python script gem5py can run.
The command line for gem5py should look like this:
gem5py ${SCRIPT TO RUN} ${ARGS TO THE SCRIPT}
So, for instance, to marshal a file called foo.py, the command line
might look like this:
gem5py python/marshal.py foo.py
Also, this change reorders the sources for the python embedding action
and limits the max_sources for Transform() to 1, so that it just shows
the python file being embedded and not gem5py or the marshal.py script.
Those are still sources so dependency tracking works correctly, but they
are also implied and just add visual noise to the build output.
Change-Id: I7ae6bd114973ae44c3b634884b6dafc6577e0788
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/49392
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
On a clean build, the git tool will wait for input() before installing
git hooks. Allow bypassing this via a command-line flag, making it
possible to perform a clean build non-interactively.
Bug: 199780674
Test: build_gem5 --install-hooks
Change-Id: I48be2c1a7c2335a2f4f6359adf582ca8b0ae5939
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/50410
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Disable warnings when targets can be built with multiple environments
but with the same actions. This can happen intentionally if, for
instance, a generated source file is used to build object files in
different ways in different environments, but generating the source
file itself is exactly the same. This can be re-enabled from the
command line if desired.
Change-Id: I426ca3331267f026553c9a2bccc79494cace8109
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/48133
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
g++ 11.1 has a false positive in the systemc kernel for
free-nonheap-object which breaks the build. The check sees that an
EventWrapper is part of the systemc scheduler object, and doesn't
realize that it was *not* set to autodelete, and so would *not* delete
itself (as part of the scheduler) when it is descheduled.
Change-Id: Ia9f48f2113e8c165923e55d7db1442b11bb6a2e8
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/48646
Reviewed-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
