If these tools aren't enabled and CXX isn't set, scons will look for
generically named compiler aliases like cc and c++. These will generally
work, but if scons knows that the compiler is specifically clang, it
will set the CXXCONFIG variable we can use to do compiler version
checking.
Because scons blindly forces a tool into use if you specify it, we need
to use the FindTool method which will check a list of tools and add the
first one that it actually finds.
Change-Id: Ie6bebb8eab531989575c878bee07189541756d2c
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/41673
Reviewed-by: Earl Ou <shunhsingou@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
gcc uses "make" to parallelize LTO. If we're using gcc and make isn't
found, we have to use single threaded LTO instead. A warning will let
the user know what's happening and that they might want to correct the
situation.
Technically gcc can use the MAKE environment variable to override the
program it uses, although I assume it still has to be "make" compatible.
Given the fairly low likelihood that someone will need that override and
the fact that scons won't pipe that variable through unless we plumb it
up, we'll just ignore that for now.
Change-Id: I891b213ece2a75bd8a915ee91f4130458dab397b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/41773
Reviewed-by: Earl Ou <shunhsingou@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The mismatches were from places where Params structs had been declared
as classes instead of structs, and ruby's MachineID struct.
A comment describing why the warning had been disabled said that it was
because of libstdc++ version 4.8. As far as I can tell, that version is
old enough to be outside the window we support, and so that should no
longer be a problem. It looks like the oldest version of gcc we
support, 5.0, corresponds with approximately libstdc++ version 6.0.21.
https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html#abi.versioning
Change-Id: I75ad92f3723a1883bd47e3919c5572a353344047
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40953
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The name of the build is opt, so it should be fully optomized. Also, the
fast build, the only one with LTO historically, is dangerous to use
since it disables many error checks. I personally run gem5 many times
while developing, iterating and trying to fix bugs, and so want it to
run quickly then too, not just the final time when collecting results.
Also, since they mirror the opt build, the perf and prof builds also
have LTO options added.
This has the nice side effect of speeding up the build time of build/X86
significantly (6:20 -> 4:27) due to parallelization of the link, and
reduces the size of the build/X86 directory (with debug compression
enabled) from 3.4GB to 2.8GB.
The size of build/X86/python/_m5 is still 1.6GB, so still more than half
of the total size of build/X86.
Change-Id: I8feabf99454693fdd100d9e1a64fdeae53362f75
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40815
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Earl Ou <shunhsingou@google.com>
In the background, scons invokes commands using what is essentially
'sh -c ${CMD}'. It's a little more sophisticated than that, but the net
effect is that all of the arguments of a given command are collected
into a single argument for sh. While Linux has a very generous limit
on the number of command line arguments you can have, and a relatively
generous limit on the total size of the arguments and environment
variables which can be read with "getconf ARG_MAX" (about 2MB only my
system), the limit on an individual argument, defined in the little
documented constant MAX_ARG_STRLEN, is much smaller, 32 pages or
131072 bytes from what I've read.
Unfortunately, when building gem5, especially without partial linking,
the command line for the final linking step can be very long, and with
some extra files brought in with EXTRAS can relatively easily exceed
this limit.
To work around this problem, this change adds a new scons "tool" which
replaces the SPAWN and PSPAWN construction environment variables (not
the shell environment, scon's env, aka main) with wrappes. These
wrappers create a temporary file, write the command into it, and then
tell the original SPAWN/PSPAWN implementation to run that file as the
actual command.
This works, but it has two potential drawbacks. First, it creates
extra temporary files which may add some overhead to the build. The
overhead is likely small, but is non-zero.
Second, while this restores the original breaks between command line
arguments, it doesn't completely avoid the total argument/environment
size limit which would be harder to hit, but does still exist.
Another option would be to use the same sort of technique, but to use
gcc's (and I assume clang's) ability to read options from a file using
an @file argument, where file is replaced with the name of the
arguments.
The upside of this approach is that it avoids all of these limits
since gcc is just reading a file, and the only limits are ones it self
imposes. Also, this would only apply to invocations of gcc, and so
would be worth plumbing in to only affect, say, the linking step.
There would be no hypothetical overhead from creating temp files.
One downside though, is that this is not a universal approach and
would break anything which doesn't understand @file style arguments.
Also, when intercepting calls to the shell, the arguments we're
getting have been made appropriate for that environment (escaping,
quoting, etc). The rules for escaping arguments in an @file may not be
quite the same as for the shell, and so there may be weird cases where
a command line gets garbled this way.
Given the tradeoffs, I think always putting the commands into temp
files and running them as scripts is the way to go.
Change-Id: I0a5288aed745a432ed72ffd990ceded2b9422585
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/41273
Reviewed-by: Earl Ou <shunhsingou@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The original implementation had a function we'd call back into later
which checked if a given ISA has KVM support on the current host.
Instead, this change reverse that and statically figures out which
single target ISA could possibly run under KVM on this host. Then later,
we can just check if the ISA we're trying to build is that one.
Change-Id: I3e7e06180983dfcc4611181718eb5d6210844d36
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40870
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Collapse nesting, and constrain the try/except to only the line in that
section which might actually be expected to throw an exception when
things are working normally.
This makes the code easier to read, and won't absorb and discard
exceptions which are thrown for unexpected reasons.
Change-Id: Id3ac417ab5545bef35ed8a87d05211c8ef2f5a50
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40867
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Collapse the check_hdf5 to the only place it was called, to set a simple
flag variable. Pull the invariant have_pkg_config check out of
check_hdf5_pkg, since if we don't have pkg-config there's no reason to
even try to use to set up hdf5. Turn the one-off, non-standard "Warning"
print into an actual warning().
Change-Id: I04793ae862aeaf1605467d205837b9dd744f93cb
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40865
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
There are really only two options current, glibc or none. If there's a
working implementation there's no real reason to select none, and if
there isn't there's no other option but to select none.
Instead of building up a list, having a default, and making what option
on the list is selected configurable, boil it down to either using glibc
if that implementation is detected, or warning and using none. Also
merge the "normal" and *BSD versions of the checks to reduce redundancy.
The complexity can be added back in if/when there are other
implementations to choose from.
Change-Id: I27c77996a00018302f4daea40924cf059d5a4323
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40864
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
This check used uname to determine if scons was running on macos, and
then a fairly elaborate check to see if the version was above 9, and if
the hardware supported 64 bit. I think at this point it's safe to assume
both that we're at least at macos 10 which is 19 years old, and that Mac
hardware supports 64 bit.
Change-Id: Ice66df2530bbcc929d3a37e7679634b75ba7b860
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40857
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
This avoids boilerplate where we check to see if flag X is supported,
and if so then set flag X. Since there are shared and static versions of
the linker flags but we only explicitly check the static ones, this
change also adds a parameter to CheckLinkFlag to set both flavors. This
defaults to true since I assume most of the time linking flags will
apply to both.
Change-Id: I983222169e9835aeb98570362f7004e2ef0240d0
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40855
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
If supported this will compress the debug information in object files,
libraries, and binaries. This decreases the size of the build/ARM
directory from 11GB to 7.2GB.
Because the benefit of this mechanism depends on the performance and
capacity of the build machine's storage, it can be disabled with the
--no-compress-debug scons flag. For instance if your storage is very
fast, writing out a larger binary may take less time than compressing
that same number of bytes, plus the time to write out the smaller
file.
This feature seems to make our presubmit test machine run out of memory
when trying to build gem5, and so is disabled in the testing scripts.
Change-Id: I71919062d23742b7658918b0fa9c4d91d0521fbf
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40715
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
SCons has a system of "tools", which basically detect versions of build
tools (compilers, linkers, etc) and set up an environment with the
appropriate build variable substitutions for that tool to be used.
For instance, there would be a "tool" for gcc, and it would detect if
gcc is present on the system, and if so would set the "CC" variable to
"gcc". An actually tool as defined by SCons would be a lot more
sophisticated than that and set more variables, but that's the basic
idea.
To help modularize the gem5 SConstruct file, I moved code which would
set up git commit hooks into a "tool" which helped modularize it and
reduce the size of SConstruct.
This isn't quite right since, while the code does detect if git was
used to check out the source (if there is a .git file at the root), it
doesn't really modify the environment at all. It will also be invoked
every time any environment is set up, although right now that will only
be the DefaultEnvironment, what's used when loose functions like
Builder or Command are called with, and the "main" environment which
all the others are Clone-d from.
Normally, when SCons sets up a new environment, either implicitly or
when Environment() is called, it sets up a bunch of built in tools
which are fixed within SCons itself. If you want, you can add a "tools"
argument to Environment (or to the DefaultEnvironment() function) which
will replace that list of tools. That can be used to make an
environment use the new "git" tool, but it isn't automatic.
SCons also lets you override default tools by creating your own with
the same name as the default. To make loading the git tool automatic,
I added an override "default" tool which, in addition to setting some
defaults which should apply to all environments, also pulled in other
tools, at that time "git" and "mercurial" (RIP).
Unfortunately, that meant that today, apparently particularly with
SCons version 4, *any* Environment would pull in "git", and all of
"git"'s dependencies, even if SCons wasn't set up enough for those to
work.
To break that dependency, this change stops the default tool from
automatically loading the git tool, although it does continue to set
other defaults which have very minimal external dependencies. When
creating the "main" Environment in the SConstruct, the "git" tool is
now added in explicitly. Since the list of tools replaces the original
and doesn't extend it, we have to add in "default" explicitly as well.
Really, the "git" tool should be converted from the tool interface into
something more appropriate, like perhaps a small module under
site_scons which site_init.py can import and call. When that happens,
main can be declared like normal again.
While making this change, I also got rid of a few nonstandard additions
to the main environment that were little used and not really necessary.
When reading the SConstruct, it wasn't very obvious where those extra
values were coming from, and they didn't really add any value.
Change-Id: I574db42fc2196bf62fc13d6754357c753ceb9117
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/38616
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Yu-hsin Wang <yuhsingw@google.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Add some more stringent Python tests that ensure that we can link with
and run applications that embed Python. This is implemented by running
building a small c++ program that embeds Python using PyBind11. The
program is run by the build system and prints the version of the
Python interpreter. The version information is then used by the build
system to ensure that the installed version is supported.
Change-Id: I727e0832f171362f5506247c022bea365068a0f6
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/36383
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This matches what's documented elsewhere. We *need* version 3.4 to
support c++14, but we support only as far back as 3.9. Also, the
argument to set c++14 as the standard is different in 3.4 and earlier
(-std=c++1y), so it makes life slightly easier to move past it to 3.9.
Change-Id: I66fa578dd3222c62907496a888f8068ed0918c7b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/34819
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The "append" option of the Help() scons method can be used to avoid
clobbering the built in and local option help.
This has the nice side effect of making it easier to add options in
other files since you now only need the built in AddOption provided by
scons itself, not the custom AddLocalOption version.
Change-Id: Ifa566087797d578df0c90f8f4fca70c8152fbf63
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32115
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Maintainer: Gabe Black <gabeblack@google.com>
This code was at least a little Alpha specific, and now that Alpha is
gone it can no longer be compiled. We could either fix it up to work
with other/all ISAs or delete it, and the consensus was to delete it. It
could potentially be revived in the future by retrieving it from version
control.
Change-Id: Ied073f2b9b166951ecba3442cd762eb19bc690b3
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32954
Reviewed-by: Steve Reinhardt <stever@gmail.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This option was for use with the old scons based regression tests.
Those had been deleted for a while, but some other bits and pieces
related to them were still lying around, depending on this option. Now
that those have been cleaned up, this option can go away.
Change-Id: I95b41dd0a14c51e74d1e527eb40e09f49aaeaf7e
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32122
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Like the optparse module's add_option method, most keyword arguments
have sensible default values. We should avoid setting those explicitly
when calling AddOption, since it usually doesn't make anything clearer
and just adds more text to wade through.
Change-Id: I70e425d9f1a0da1cafcc3d7dd24bfde18c0b3f35
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32116
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
It appears that scons bugs are not on tigris.org any more and are now
on github, although fortunately old bugs seem to have been ported over
and have the same numbering.
This CL updates URLs which were in comments in the gem5 source,
specifically in scons scripts, to point to the corresponding github
version.
I also checked to see if these bugs were still open, or if we could
remove our workarounds for them.
1. 2356 is still open, and has been fairly recently assigned.
2. 2611 is marked as fixed. We might be able to implement the
workaround in its last comment from August of 2019.
3. 2811 has been marked fixed, and as best I can tell the fix first
appeared in around version 3.0 of scons. If/when that is our
minimum version, we can remove the workaround in
site_scons/site_tools/default.py. That is mostly fixing an annoying
spurious rebuild by scons which does not affect correctness, so even
if we remove that workaround we shouldn't break earlier versions,
although it would be obnoxious for people that are affected by it
and best avoided.
Change-Id: I0d74820f399044c6f80148bf3022d07d7bf6f4e5
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32114
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
We already provide to the user the CCFLAGS_EXTRA, LDFLAGS_EXTRA
variables to pass flags to scons when compiling/linking gem5.
Those variables are not passed to the marshal object.
We add an extra pair:
MARSHAL_CCFLAGS_EXTRA, MARSHAL_LDFLAGS_EXTRA
to add flag injection capabilities to the marshal object.
The patch is also renaming base_py_env to marshal_env.
This happens for 2 reasons:
1) At the moment the marshal compilation is the only task
making use of the base python environment.
2) Consistency with the EXTRA variable names added with this patch.
I could have named them as BASE_XXFLAGS_EXTRA, but it seems too much
generic and users might be confused by that, as they might think
the BASE_XXFLAGS_EXTRA is a subset of the XXFLAGS_EXTRA so that
setting it will affect gem5 compilation as well.
Change-Id: I3e420caa897059455ff8f35462db2b38da050e93
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/30016
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
