This makes what are configuration and what are internal SCons variables
explicit and separate, and makes it unnecessary to call out what
variables to export to C++.
These variables will also be plumbed into and out of kconfiglib in later
changes.
Change-Id: Iaf5e098d7404af06285c421dbdf8ef4171b3f001
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56892
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The SConsopts files that need to be processed is affected by the EXTRAS
setting, and if all settings are going to be local to the variants, that
means each variant can have a different set of SConsopts files to use.
This does result in Configure type output showing up repeatedly when
building multiple variants, but that's uncommon for users, and will
hopefully become unnecessary for developers over time.
Change-Id: I7df287c37fc32e91576dced80d0a9e56806c4581
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56751
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
These are no longer split out and shared in the root build/ directory.
This does result in a small amount of overhead from building redundant
copies of these files, although the overhead is not significant. When
building 7 different variants of gem5, all the ISAs and NULL, the
difference on my machine was:
Before:
real 41m25.372s
user 914m22.266s
sys 41m51.816s
After:
real 42m38.074s
user 921m36.852s
sys 43m2.949s
This is about a 2-3% difference, which is a worse than typical case,
since the overhead scales with the number of variants being built.
The benefit of pulling ext/ into the variant directory is that there can
now be a single config which applies to all files used to build gem5,
and that config is represented by the variant of gem5 being built.
Change-Id: I6f0db97c63a7f3e252e7e351aa862340978e701b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56750
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gabe Black <gabe.black@gmail.com>
This new and improved linker approximately halves link time for
build/X86/gem5.opt for me compared to lld, the previously fastest
linker. The -fuse-ld=mold option is supported by current versions of
clang, and will be supported in gcc 12.
To use the mold linker with gcc prior to version 12, you need to set
the LINKFLAGS_EXTRA scons variable to pass in a -B option as described
on this page:
https://github.com/rui314/mold
Change-Id: Ic5ad0e532fac078d384a0aebb3e04a4b04ce4880
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/57173
Maintainer: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Earl Ou <shunhsingou@google.com>
Whether to honor the USE_KVM setting can be handled in a more targeted
way down at individual consumers. They will know what ISA the host needs
to be for their particular model to make sense, where the SConstruct has
to enable or disable it for the entire build of gem5.
Change-Id: I9e82e232203834f6fe56d0ce5cf696809a0970bc
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56188
Maintainer: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
These variables will always be imported from the environment, and never
from previous builds. For SCons to actually use these values, they need
to not only be in the environment variables external commands SCons runs
sees, they also have to be promoted to actual SCons construction
variables.
Change-Id: I54035442d70972396f1788bd0773877222d7a7c5
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/56386
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
There is currently a "--gold-linker" option, which tells SCons to use
the gold linker. This puts the gold linker in a special position where
it has its own command line option, and makes it impossible to
explicitly select any other linker.
This change adds a --linker option which can be set to "bfd" (the
default gnu linker), "gold" (the gnu gold linker), or "lld" which is the
clang linker. If none is specified, SCons will let the compiler fall
back to whatever its default is.
The --gold-linker option is modified so that it just sets the --linker
option for you. Because of how SCons handles adding command line
options, there isn't a good way to complain if both it and --linker is
used, nor is there a good way to warn that the user is using a
deprecated command line option. A warning is in the help text for that
option now, though.
If the gold linker is selected, this change also sets the --threads
option, and sets the --thread-count option. This tells the gold linker
to run multi-threaded to speed up linking. Apparently the gold linker
must have been built with an option that enables threading for these
options to have any effect. If threads and LTO are both enabled, the
gold linker segfaults. To avoid that problem, we don't add --threads if
LTO has been enabled.
This change also enables a new configuration, where lld, the clang
linker, can be used with gcc. Unfortunately the format of LTO
information gcc/g++ generates is not compatible with lld, and so if you
set the linker to lld and enable LTO, you'll probably get errors like
not being able to find the symbol "main".
This change also allows you to select the default gnu linker ("bfd")
when using clang, although it's not clear why you would want to do that.
Change-Id: Ib37030431527f5d0bc626e9594b2e1142df889be
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/49003
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
SCons does not use a variable called LDFLAGS, it uses one called
LINKFLAGS. Switch some errant uses to the correct name.
Also, adjust all the other variable names to use LINK, for consistency
and to avoid confusion and avoid mistakes in the future.
Change-Id: I38d40f5231afdf62bcfba04478d403d65e9b1e26
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/51987
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The new c++ wrapper is called gem5py, and will run any python script
using gem5's embedded python interpreter. The "marshal" functionality is
split out into a separate python script gem5py can run.
The command line for gem5py should look like this:
gem5py ${SCRIPT TO RUN} ${ARGS TO THE SCRIPT}
So, for instance, to marshal a file called foo.py, the command line
might look like this:
gem5py python/marshal.py foo.py
Also, this change reorders the sources for the python embedding action
and limits the max_sources for Transform() to 1, so that it just shows
the python file being embedded and not gem5py or the marshal.py script.
Those are still sources so dependency tracking works correctly, but they
are also implied and just add visual noise to the build output.
Change-Id: I7ae6bd114973ae44c3b634884b6dafc6577e0788
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/49392
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
On a clean build, the git tool will wait for input() before installing
git hooks. Allow bypassing this via a command-line flag, making it
possible to perform a clean build non-interactively.
Bug: 199780674
Test: build_gem5 --install-hooks
Change-Id: I48be2c1a7c2335a2f4f6359adf582ca8b0ae5939
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/50410
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Disable warnings when targets can be built with multiple environments
but with the same actions. This can happen intentionally if, for
instance, a generated source file is used to build object files in
different ways in different environments, but generating the source
file itself is exactly the same. This can be re-enabled from the
command line if desired.
Change-Id: I426ca3331267f026553c9a2bccc79494cace8109
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/48133
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
g++ 11.1 has a false positive in the systemc kernel for
free-nonheap-object which breaks the build. The check sees that an
EventWrapper is part of the systemc scheduler object, and doesn't
realize that it was *not* set to autodelete, and so would *not* delete
itself (as part of the scheduler) when it is descheduled.
Change-Id: Ia9f48f2113e8c165923e55d7db1442b11bb6a2e8
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/48646
Reviewed-by: Hoa Nguyen <hoanguyen@ucdavis.edu>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
A comment in that section of SConstruct talked about how ParseConfig
couldn't understand -Xlinker and -export-dynamic, and how that meant
they needed to parse the output of python-config manually. Some
searching around turned up this:
https://bugs.python.org/issue36508
which seems to suggest that exactly those two options weren't actually
supposed to be returned by python-config in the first place, and are
really for python's own internal build.
This change switches over to using ParseConfig, but to avoid bad options
sneaking through, it uses an option argument to ParseConfig to intervene
before the options are consumed and filter out everything except
required libraries, and library and header search directories.
Change-Id: I1a50eb030019a447b919381c32168fb3bb27b89b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40971
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Instead, create a new tool called EnvDefaults. This tool now needs
to be explicitly listed in the set of tools available through an
Environment, but that's easy to do in on place (all environments should
stem from main). Hijacking default like we were doing was (as far as I
can tell) not how that tool was intended to be used, and doing things
this way is a bit less hacky.
Also, we can split more Builders, etc, out of SConstruct, and these will
need to attach to main in the same way. We're going to need a list of
tools when main is constructed one way or the other, so we might as well
follow the rules.
Change-Id: I392e1639cb69d373c64970dccf45258000498cc3
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40965
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
This has two purposes. First, SCons assumes that once you call
Configure, you won't set up the environment the Configure is based on
until after you get the environment back from it again with
conf.Finish(). We get away with this when the cache mode for config
tests is not "force", since Configure just reuses the environment we
pass in, and any changes we make are immediately communicated between
the two.
If the cache mode *is* "force" though, SCons modifies the decider so
that everything the conf environment goes to build looks like it's out
of date. It does that by cloning the original environment, and then
using that clone to do its tests. That causes a problem because we have
a long lived "conf" object and make further changes to main, and since
the two environments are now separate the one in conf doesn't see those
updates.
Second, and more subtly, we export our "main" and "env" environments so
that other SConsopts and SConscript files can use them and define things
in them. The way Configure is designed, if the config caching mode is
"force", then it will create a new environment, and then that
environment will replace what the, for instance, "main" variable points
to when "main = conf.Finish()" is executed.
Unfortunately, if we've already Export()-ed main, we've exported what
the "main" variable pointed to at that time. Our view of "main" will
track with the value that conf.Finish() returned, but since that
construction environment is mearly derived from the main we Exported and
not actually the same thing, they have diverged at that point and will
behave independently.
To solve both of these problems, this change modifies the
gem5_scons.Configure() method so that it's a context manager instead of
a regular function. As before, it will call Configure for us and create
a configuration context, which it will yield as the "with" value. When
the context exits, all the variables in the context Finish() returns
will be shoved back into the original context with Replace(). This isn't
perfect since variables which were deleted in the environment (probably
very rare in practice) will not exist and so will not overwrite the
still existent variable in the original dict.
This has several advantages. The environment never splits into two
copies which continue on independently. It makes the lifetime of a
configuration context short, which is good because behavior during that
time is tricky and unintuitive. It also makes the scope of the context
very clear, so that you won't miss the fact that you're in a special
setting and need to pay attention to what environment you're modifying.
Also, this keeps the conceptual overhead of configuration localized to
where the configuration is happening. In parts of the SConscripts which
are not doing anything with conf, etc, they don't have to modify their
behavior since no configuration context is active.
This change is based on this change from Hanhwi Jang who identified this
problem and proposed an initial solution:
https://gem5-review.googlesource.com/c/public/gem5/+/44265
Change-Id: Iae0a292d6b375c5da98619f31392ca1de6216fcd
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/44389
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Hanhwi Jang <jang.hanhwi@gmail.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
This option would set the -msse2 compiler flag to avoid using x87
floating point, but every 64 bit x86 CPU supports sse2, and so sse2 is
enabled by default on compilers targeting it. 32 bit x86 CPUs are
obscure enough that it's not worth preserving this marginally meaningful
option which would only matter for them.
Change-Id: Ib425c261b27f1f74300e714619787d04970065ce
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40879
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This flag was necessary because of self assignments in the ISA parser
where self assignments are often hints to the parser itself, and in one
case because a pybind-ism used to attach the -= operator looked like a
self assignment.
This change narrows the scope of the flag that disables this warning to
only files generated by the ISA parser, and the single file in the
systemc code which uses that operator overload.
Change-Id: Ib64fc72e46f894cba9064afcdbdcc5859c30e745
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40952
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
