Add some more stringent Python tests that ensure that we can link with
and run applications that embed Python. This is implemented by running
building a small c++ program that embeds Python using PyBind11. The
program is run by the build system and prints the version of the
Python interpreter. The version information is then used by the build
system to ensure that the installed version is supported.
Change-Id: I727e0832f171362f5506247c022bea365068a0f6
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/36383
Reviewed-by: Gabe Black <gabe.black@gmail.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This matches what's documented elsewhere. We *need* version 3.4 to
support c++14, but we support only as far back as 3.9. Also, the
argument to set c++14 as the standard is different in 3.4 and earlier
(-std=c++1y), so it makes life slightly easier to move past it to 3.9.
Change-Id: I66fa578dd3222c62907496a888f8068ed0918c7b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/34819
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The "append" option of the Help() scons method can be used to avoid
clobbering the built in and local option help.
This has the nice side effect of making it easier to add options in
other files since you now only need the built in AddOption provided by
scons itself, not the custom AddLocalOption version.
Change-Id: Ifa566087797d578df0c90f8f4fca70c8152fbf63
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32115
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Maintainer: Gabe Black <gabeblack@google.com>
This code was at least a little Alpha specific, and now that Alpha is
gone it can no longer be compiled. We could either fix it up to work
with other/all ISAs or delete it, and the consensus was to delete it. It
could potentially be revived in the future by retrieving it from version
control.
Change-Id: Ied073f2b9b166951ecba3442cd762eb19bc690b3
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32954
Reviewed-by: Steve Reinhardt <stever@gmail.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This option was for use with the old scons based regression tests.
Those had been deleted for a while, but some other bits and pieces
related to them were still lying around, depending on this option. Now
that those have been cleaned up, this option can go away.
Change-Id: I95b41dd0a14c51e74d1e527eb40e09f49aaeaf7e
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32122
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Like the optparse module's add_option method, most keyword arguments
have sensible default values. We should avoid setting those explicitly
when calling AddOption, since it usually doesn't make anything clearer
and just adds more text to wade through.
Change-Id: I70e425d9f1a0da1cafcc3d7dd24bfde18c0b3f35
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32116
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
It appears that scons bugs are not on tigris.org any more and are now
on github, although fortunately old bugs seem to have been ported over
and have the same numbering.
This CL updates URLs which were in comments in the gem5 source,
specifically in scons scripts, to point to the corresponding github
version.
I also checked to see if these bugs were still open, or if we could
remove our workarounds for them.
1. 2356 is still open, and has been fairly recently assigned.
2. 2611 is marked as fixed. We might be able to implement the
workaround in its last comment from August of 2019.
3. 2811 has been marked fixed, and as best I can tell the fix first
appeared in around version 3.0 of scons. If/when that is our
minimum version, we can remove the workaround in
site_scons/site_tools/default.py. That is mostly fixing an annoying
spurious rebuild by scons which does not affect correctness, so even
if we remove that workaround we shouldn't break earlier versions,
although it would be obnoxious for people that are affected by it
and best avoided.
Change-Id: I0d74820f399044c6f80148bf3022d07d7bf6f4e5
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32114
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
We already provide to the user the CCFLAGS_EXTRA, LDFLAGS_EXTRA
variables to pass flags to scons when compiling/linking gem5.
Those variables are not passed to the marshal object.
We add an extra pair:
MARSHAL_CCFLAGS_EXTRA, MARSHAL_LDFLAGS_EXTRA
to add flag injection capabilities to the marshal object.
The patch is also renaming base_py_env to marshal_env.
This happens for 2 reasons:
1) At the moment the marshal compilation is the only task
making use of the base python environment.
2) Consistency with the EXTRA variable names added with this patch.
I could have named them as BASE_XXFLAGS_EXTRA, but it seems too much
generic and users might be confused by that, as they might think
the BASE_XXFLAGS_EXTRA is a subset of the XXFLAGS_EXTRA so that
setting it will affect gem5 compilation as well.
Change-Id: I3e420caa897059455ff8f35462db2b38da050e93
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/30016
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The defaulted-function-deleted warning triggers in generated code which
would be very tricky to address.
The c99-designator refers to using an array index to specify which
element of an array is being initialized. This makes the code more
clear and is supported by both g++ and clang++. Designated initializers
for structures are being introduced in C++20, but there is no word I
could find on arrays. This warning option seems to only exist in clang
versions 10 and up, so we can only use it on those versions.
Change-Id: I8fb858e643814638c552a49336db2672be8e43c8
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/29396
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
This reverts commit f41abbdb5c,
"scons: Enable LTO and partial linking with gcc >= 8.1."
LTO and partial linking does not work on GCC 9.3 on Ubuntu 20.04 when
compiling gem5.fast. This error was exposed via the following command:
```
docker run -u $UID:$GID --volume $(pwd):/gem5 -w /gem5 --rm \
gcr.io/gem5-test/ubuntu-20.04_all-dependencies:latest scons \
build/MIPS/gem5.fast
```
The following error was received:
```
usr/bin/ld: cannot find lib.fo.partial.lto.o: No such file or directory
/usr/bin/ld: error: could not unlink output file
collect2: error: ld returned 1 exit status
scons: *** [build/MIPS/mem/ruby/system/lib.fo.partial] Error 1
```
Issue-on: https://gem5.atlassian.net/browse/GEM5-555https://gem5.atlassian.net/browse/GEM5-557
Change-Id: Id9e7fc81aec9f94524acc92c05aabdf96bd284cd
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/29272
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The bug(s) which prevented LTO and partial linking from working with gcc
have been fixed in my local version (9.3), and according to one of the
original bug reports:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69866
A fix was committed in gcc version 8.1.
The original code left in the SConstruct describing the problem with
versions greated than 6.0 also enabled an -flinker-output option and set
it to "rel". That option doesn't show up in the gcc 8.4 documentation
even though it was added in 6.0, but in the 9.3 documentation it
describes it and says that it defaults to "rel" when the -r (partial
linking) option is used.
This *should* mean that LTO and partial linking can be used together
with no issues after version 8.1, and at most by version 9.3. If someone
finds that that isn't true, then the range of bad versions can be
expanded.
Change-Id: Ie0529d077a0042ef55e2af995d01430d1695c031
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/27131
Tested-by: Gem5 Cloud Project GCB service account <345032938727@cloudbuild.gserviceaccount.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Python 3 uses the concepts of two different types:
text and binary strings.
Those cannot be implicilty combined (as it was happening in python2) and
in order to be used together one of them must be converted to the other
type:
* Text can be encoded into Bytes via the encode() method
* Bytes can be decoded to Text using the decode() method
By default encode/decode will assume UTF-8 format
JIRA: https://gem5.atlassian.net/browse/GEM5-345
Change-Id: I1bdf7db17b49cc109239fd5f44791769370853f8
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/26250
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
* dict methods dict.keys(), dict.items() and dict.values()
return "views" instead of lists
* The dict.iterkeys(), dict.iteritems() and dict.itervalues()
methods are no longer supported.
* map() and filter() return iterators.
* range() now behaves like xrange() used to behave, except it works with
values of arbitrary size. The latter no longer exists.
* zip() now returns an iterator.
Change-Id: Id480018239db88d7f5d60588c93719056de4a0c0
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/26248
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Not running the systemc test SConscript reduces the scons startup time
(before any file is compiled) from about 10s to 4s on my machine.
The performance investigation was done at:
https://gem5.atlassian.net/browse/GEM5-256
As before, the systemc tests are still automatically built when
they are run with:
src/systemc/tests/verify.py --update-json build/ARM -j `nproc` \
--filter-file src/systemc/tests/working.filt
Change-Id: I33b7a53c0a7d70386ab17d7bb4886c84a97a2eb3
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/25385
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
I've arbitrarily chosen to make ARM the default ISA for now, since I
think it's the best supported ISA with X86 somewhere a little behind
it. As a compromise, I change all mention of ALPHA (or even ALPHA_SE!)
in comments to be X86 instead, so it gets some attention too.
Change-Id: I1d8edc7925ca2d94f11b26e2c0b9314216e9b97d
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/25463
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
This was primarily in Alpha where disassmbly output could be compatible
(default off, probably not usd in a long time), and floating point
could be compatible (default on). A small bit had crept into x86 from
long ago which is also removed.
Change-Id: Ibb68b63787f370259bd1613b393e0b057c007704
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/25012
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The build was failing with:
/usr/bin/ld: unrecognized option '--as-needed -fuse-ld=gold'
and --verbose confirms that a single quoted CLI parameter was being
executed:
"-Wl,--as-needed -fuse-ld=gold"
This happened because at Ifb001786a66b0dd9b29865e39a5740313002f250
--as-needed was added, and because it is the second option to happen before
the following main.subst, it exposed the fact that the existing main.subst
was wrong, because it returns a string instead of the expected array.
Change-Id: I619d242d60fe9d27438638ac11c2b92512881f26
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/24624
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commands that blindly use PROTOC will try to execute "False" which is
very confusing for someone looking at the console output and error
messages. Instead, create a new environment setting HAVE_PROTOC which
is either true or false depending on if the protoc command exists and
passes muster.
Also, if there's an error running protoc, catch that and use it to
mark protoc as unavailable. The previous behavior was to supress errors
and just return an empty string instead, I assume with the expectation
that that would be an invalid version and fail later checks.
Change-Id: I1251b4e7e0e9894cdd3343e59498cc653b648b26
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/22883
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This changeset add support for stat dumps in the HDF5 file
format. HDF5 is a binary data format that represents data in a
file-system-like balanced tree. It has native support for
N-dimensional arrays and binary data (e.g., frame buffers).
It has the following benefits over traditional text stat files:
* Efficient storage of time series (multiple stat dumps)
* Fast lookup of stats
* Plenty of existing tooling (e.g., Python libraries and graphical
viewers)
* File format can be used to store frame buffers together with
normal stats.
Drawbacks:
* Large startup cost (single stat dump larger than text equivalent)
* Stat dumps are slower than text
Known limitations:
* Distributions and histograms aren't supported.
HDF5 stat output can be enabled using the 'h5' URL scheme when
overriding the stat file name on gem5's command line. The following
parameters are supported:
* chunking (unsigned): Number of time steps to pre-allocate
(default: 10)
* desc (bool): Output stat descriptions (default: True)
* formulas (bool): Output derived stats (default: True)
Example gem5 command line:
./build/ARM/gem5.opt \
--stats-file="h5://stats.h5?desc=False;formulas=False" \
configs/example/fs.py
Example Python stat consumer that computes IPC:
import h5py
f = h5py.File('stats.h5', 'r')
group = f['/system/cpu']
for i, c in zip(group['committedInsts'], group['numCycles']):
print i, c, i / c
Change-Id: I351c6cbff2fb7bef9012f47876ba227ed288975b
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/8121
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Ciro Santilli <ciro.santilli@arm.com>
The flags may be passed as:
scons CCFLAGS_EXTRA='-Wno-error -pedantic' \
LDFLAGS_EXTRA='-g -g' build/<arch>/gem5.opt
The initial motivation for this commit is to help disable warning that
have become errors while bisecting.
scons orders the flags by Append call order, and ideally these flags
should be added last to override the others, since the last GCC flags
take precedence. However I haven't found a simple way to put them at
the very end.
Change-Id: Ida24dfb9604d88b99f113392ab5e47d578ba7259
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19048
Reviewed-by: Juha Jäykkä <juha.jaykka@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>
