Hi, we are security researchers from the Advanced Research Center at Trellix.
We have began a campaign to patch a widespread bug named CVE-2007-4559.
CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using
extract() or extractall() on a tarfile object without sanitizing input,
a maliciously crafted .tar file could perform a directory path traversal
attack. We found at least one unsantized extractall() in your codebase
and are providing a patch for you via pull request. The patch essentially
checks to see if all tarfile members will be extracted safely and throws
an exception otherwise. We encourage you to use this patch or your own
solution to secure against CVE-2007-4559.
If you have further questions you may contact us through this
projects lead researcher Kasimir Schulz.
Change-Id: I891ac6652cfbd479aed51d64ef6d4e0fe740e06d
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65271
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Previously, shared memory server remove old socket *before* filling the
target path into API's data structure. However, the target path might
get truncated hence the path we check against might not be the one we
will be using in the end.
In a case where the path specified by user is free while the truncated
path is in used, gem5 will get a mysterious EADDRINUSE.
We swap the two steps in the CL, so we'll be checking against the actual
path we use, instead of the path user request to use.
Change-Id: Ib34f8b00ea1d2f15dcd4e7b6d2d4a6d6ddc4e411
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65153
Reviewed-by: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
This changeset replicates some of the multiprocessing module
implementation from the python standard library in gem5. The goal of
this and following changesets is to enable users to use a *single* set
of python scripts to run and analyze a suite of gem5 simulations.
We must reimplement some of the multiprocessing module becaue it is not
flexible enough to allow for customized command line parameter to the
"python" executable (gem5 in our case). To get around this, I extended
the Process and context objects to be gem5 specific.
The next steps is to wrap the Process and Pool types with gem5-specific
versions that will improve their usability for our needs. With this
changeset, these objects are usable, but it will require significant
user effort to reach the goal of running/analyzing many different gem5
simulations.
There are some limitation:
- The pool will only work if the max tasks per child is 1
- The functions that are executed must come from another module
As an example, the following code should work after applying this
change.
test.py:
```python
from gem5.utils.multiprocessing import Process, Pool
from sim import info, run_sim
if __name__ == '__m5_main__' or __name__ == '__main__':
info('main line')
p1 = Process(target=run_sim, args=('bob',))
p2 = Process(target=run_sim, args=('jane',))
p1.start()
p2.start()
p2.join()
p1.join()
with Pool(processes=4, maxtasksperchild=1) as pool:
pool.map(run_sim, range(10))
```
sim.py:
```
import os
def info(title):
print(title)
print('module name:', __name__)
print('parent process:', os.getppid())
print('process id:', os.getpid())
def run_sim(name):
info('function g')
from gem5.prebuilt.demo.x86_demo_board import X86DemoBoard
from gem5.resources.resource import Resource
from gem5.simulate.simulator import Simulator
board = X86DemoBoard()
board.set_kernel_disk_workload(
kernel=Resource("x86-linux-kernel-5.4.49"),
disk_image=Resource("x86-ubuntu-18.04-img"),
)
simulator = Simulator(board=board)
simulator.run(max_ticks=10000000)
```
Change-Id: I4348ebaa75d006949ec96e732f5dc2a5173c6048
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63432
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Richard Cooper <richard.cooper@arm.com>
Since moving `_connect_things` to a pre-init step, the ArmBoard can now
be refactored to set up things in a more logical manner. In particular,
this patch moves activity out of the `_add_disk_to_board` function and
into the `_pre_initialization` function.
Change-Id: I5d40267f28ae87cd483a0396739c09b8b2b46383
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65052
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Through working with the gem5 stdlib there have been instances where
connecting the memory, processor, and cache hierarchy to the board (via
the AbstractBoard's `_connect_things` function) at the point of the
AbstractBoard's construction is problematic as the memory, processor,
and cache hierarchy may require information to connect correctly that is
only known to the AbstractBoard after construction. In particular this
can occur when a Workload contains information needed to configure
correctly.
To resolve this problem the `_connect_things` function has been moved to
run as a pre-initialization step. That is, run immediately before
`m5.instantiate`. This is done in the Simulator module.
This will break cases where a user utilizes the stdlib AbstractBoard but
does not use the stdlib Simulator module. As such, an Exception is
raised in these cases explaining the fix to the user. This is done via a
hack where the boards' `createCCObject` function (inheritted
from SimObject) is overriden with a check to ensure `_connect_things`
has been run. To fix the `_pre_instantiate` function must be executed
prior to `m5.instantiate` in the Python configuration script. Test and
config scripts in the gem5 repo have been updated accordingly.
Change-Id: Ibaef36eb7433ce104b861b1da80fc600f08f715a
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65051
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
It is possible that the board has more than just a "main" memory. For
instance, the ArmBoard has a boot memory which is separate from the
`get_memory` function.
This moves the `get_mem_ports` function to the board so that the board
can optionally override it.
Change-Id: I05e388cc93e691e9a4fa674023f158af447349f9
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64631
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
The PM4 release_mem packet is used as a DMA fence in the driver. It
specifies which queue the interrupt came from by encoding the me, pipe,
and queue fields from the map_queue packet into the interrupt ring ID.
Currently these fields are incorrect because (1) the order in the
bitfield is backwards, (2) the queue constructor assigns a pointer to
the PM4MapQueue packet containing this data to the dmaBuffer which gets
deleted in short order, and (3) the order of the encoding of ring ID is
incorrect.
This change fixes these issues by (1) placing the struct vales in
correct order, (2) creating a const copy of the dmaBuffer on
construction, and (3) using the ring ID encoding expected by the driver:
https://github.com/RadeonOpenCompute/ROCK-Kernel-Driver/blob/roc-4.3.x/
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c#L5989
Change-Id: I72c382980e57573f8a8a6879912c4139c7e2f505
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65095
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
The PM4 NOP header is used to insert spaces in the PM4 ring and can
therefore be any size. This includes zero. A size of zero is denoted by
a value of 0x3fff in the NOP packet header. Currently we assume this
means the remainder of the PM4 queue up to the wptr is empty/NOPs. This
is not always true.
This changeset reworks the PM4 NOP packet to handle the value of 0x3fff
as a special value and advances the rptr by 0 bytes. This fixes issues
where there were additional packets in the queue which were being
skipped over by fast forwarding. Since those packets could be anything,
that leads to undefined behavior afterwards.
Change-Id: I3f5c3f4b7dd50f93ba503fea97454a9d41771e30
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65094
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
SDMA traps are used in the driver as a DMA fence. To pass a fence, the
SDMA sends the driver the interrupt context from a trap packet and the
ring ID which specifies which queue in the SDMA engine is passing a
fence. Currently the interrupt context is using the wrong value in the
packet and the ring ID is hard-coded to always be the gfx queue.
This changeset uses the correct interrupt context from the SDMA packet
and sets the ring ID to either 0 if the gfx queue is currently being
processed or 3 if the page queue is being processed.
The relevant interrupt service routine in the driver can be found at:
https://github.com/RadeonOpenCompute/ROCK-Kernel-Driver/blob/roc-4.3.x/
drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c#L2129
Change-Id: Ie4a4a9d6ab1d3bf83bf76bb57a02a91100217b51
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65093
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The interrupt handler's base address is sent via MMIO and must be
shifted by 8 bits to convert to a byte address. The current code is
shifting the MMIO dword first then assigning, resulting in the top 8
bits being shifted out.
This changeset fixes the issue by assigning the dword to the 64-bit
address first then shifting after. Similarly, the upper dword is cast to
a 64-bit value first before shifting.
This fixes some "fence fallback timeout" errors in the m5term output.
These timeouts become a problem because the driver will reset after a
few hundred of them, killing any running GPU applications as part of the
process.
Change-Id: I0beec313f533765c94063bcf4de8c65aacf2986b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65092
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
The GART table is a legacy 1-level page table primarily used for
supervisor mode accesses to GPUs. The PTE size is 64-bits, not 32-bit.
This causes memory sizes >3GB (in X86) to fail loading amdgpu driver.
This changeset fixes the issue by setting the GART mappings to the
correct data type.
Change-Id: Ibfba2443675fe28316d26afa5f1a14885fdce40c
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/65091
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
The current implementation of SDMA copy calls the GPU memory manager's
read/write method one time passing a physical address as the
source/destination. This implicitly assumes the physical addresses are
contiguous which is generally not true for large allocations. This
results in reading from/writing to the wrong address.
This changeset fixes the problem by copying large copies in chunks of
the minimum possible page size on the GPU (4kB). Each page is translated
seperately to ensure the correct physical address. The final copy "done"
callback is only used for the last transfer. The transfers should
complete in order so the copy command will not complete until all chunks
have been copied. Tested and verified on an application with a large
allocation (~5GB).
Change-Id: I27018a963da7133f5e49dec13b0475c3637c8765
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64752
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Vega allows for any integer multiple of 4kB pages. However, the current
implementation is designed for 4kB page primarily. In order to support
variable page sizes, the physical address calculation needs to be
updated to add the virtual page offset to the base physical address
rather than bitwise-OR. Bitwise-OR assumes physical pages are at
aligned to the page size which is generally not the case for very
large pages (1GB+).
This changeset changes all of the physical address computations to add
the virtual offset to the physical page address. This fixes many GPUFS
applications which use larger pages. The support was tested by
hipMalloc'ing ~5GB to induce a large page being created. The test
application now passes verification with this change.
Change-Id: Ic8d1475e001def443f3e4ab609449bca0c40b638
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64751
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
It never made much sense to set checkpoint via the Simulator module as
Checkpoints are very tightly coupled with the Workload being run. This
change therefore moves the checkpoint to the set_workload functions.
Setting checkpoints via the Simulator is deprecated and will be removed
in a future release.
Change-Id: I24d2133b38a86423d3553ec888c917c5fe47b93d
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64571
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Some CPU wrappers like the Fastmodel one do extend the
ThreadContext interface in order to retrieve system register
state... By bypassing the TC interface and by using the ISA
instead, we are basically forcing users to extend the ISA
as well to intercept these calls.
So with this patch we are making sure every system register is accessed
(like HCR_EL2 or SCR_EL3) through the thread context. This of course
does not apply to the CPU interface registers as we still use the ISA
storage for them. In the future we should probably move that storage
from the ISA class to the Gicv3CPUInterface class itself
This is also simplifying Gicv3CPUInterface::isEL3OrMon:
currEL already covers the AArch32 case so no need to
differentiate between AArch32 and AArch64
Change-Id: I446a14a6e12b77e1a62040b3422f79ae52cc9eec
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64913
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
This patch is adding an outfile parameter to the TarmacTracer
This has 3 options:
1) stdoutput = dump to standard output (default behaviour)
2) stderror = dump to standard error
3) file = dump to a file. As there is one tracer per CPU,
this means every CPU will dump its trace to a different file,
named after the tracer name (e.g. cpu0.tracer, cpu1.tracer)
It is still possible to redirect to a file with option 1 and 2
thanks to common bash redirection. What the third option is
really buying us is the capability to dump CPU traces on
separate files, and to separate the trace output from the debug-flag
output
Change-Id: Icd2bcc721f8598d494c9efabdf5e092666ebdece
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63892
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Richard Cooper <richard.cooper@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
The previous SimPoint warmup length was limited by the gaps between
the starting instruction of one SimPoint and the ending instruction of
the SimPoint before it. This was to prevent duplicate SimPoints, but it
can significantly limit the warmup length.
In this commit, the warmup length limitation will be extended to the
starting instruction of one SimPoint regardless of the gap between
SimPoints.
A SimPoint checkpoint generator is created to help taking checkpoints
for SimPoints and make sure multiple SimPoint checkpoints are taken
when there are multiple SimPoints sharing the same starting instruction
Change-Id: If95f6813e8cbf5c01e41135c1b1bb91ed2e950ad
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64351
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
For SVE it is possible to override the run-time vector length (VL) per
exception level by setting the value in the appropriate ZCR_ELx
registers. In general instructions query the appropriate registers
during execute() to determine the actual vector length. The exception
to this rule are the SVE Macromem instructions which use the VL to
determine the number of micro-ops to crack into during
decode. However, as there is no available ExecContext during the
decode stage these instructions rely on a cached value stored in the
decoder.
Previously we were updating the VL in the decoder using potentially
stale values of ZCR_ELx by calling the update before actually setting
the registers themselves. We now set the registers before updating the
decoder.
Change-Id: I0167095699f7f950ee99fc42c7c8292fe8938d28
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64331
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Maintainer: Giacomo Travaglini <giacomo.travaglini@arm.com>
This change makes the default vendor string AuthenticAMD.
GLIBC now is much more strict about checking for the current system's
supported features. In Ubuntu 22.04, when trying to load a dynamically
linked file, the CPUID is checked for the required features. If they are
not there, an error saying ISA level too low is returned and the program
crashes.
The underlying issue is that GLIBC does not check and populate the
cpu_feature data structure if it does not detect a *known* CPU model.
The options are hardcoded. See the following file for the glibc code.
glibc/sysdeps/x86/cpu-features.c
Note that the cpu_features is not populated with the
COMMON_CPUID_INDEX_1 unless there is a known family, which is only set
if the vendor string matches a known vendor.
This change uses AuthenticAMD instead of the alternatives because the
checks in glibc are most simple (no special cases) for AuthenticAMD in
the init_cpu_features functions.
GLIBC has been unable to populate the cpu_features datastructure
correctly with gem5 for a long time. However, this has just now become a
problem for us because the library now is more strict on not allowing
code to execute unless the processor meets certain minimum requirements.
I believe the commit for GLIBC which caused this breakage is
ecce11aa0752735c4fd730da6e7c9e0b98e12fb8
See https://sourceware.org/pipermail/binutils/2020-October/113593.html
for more details on that commit.
Change-Id: I8eedb46f577361e749ad8d0adda4fd0753e99960
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64831
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>
The current MESI_Two_Level protocol's L1 caches updates the MRU information twice per request on misses -- once when the request reaches Ruby and once when the miss is returned from another level of the memory hierarchy.
Although this approach does not cause any correctness bugs for replacement policies like LRU since this request is the LRU in both cases, it does not work correctly for other policies like SecondChance and LFU, where updating the information twice (for misses) causes them to devolve to LRU.
Note that this was not directly a problem with Ruby previously, because it only supported LRU-based policies that were unaffected by this. However, with the integration of 20879 Ruby now uses the same replacement policies as Classic (which has additional, non-LRU based replacement policies).
This patch resolves this problem by not updating the MRU information a second time for the misses. It has been tested and validated with the replacement policy tests.
Change-Id: I9e7e96a9d6c09f3d6b7daae7115ef091ac3bdc08
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64371
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Reviewed-by: Matt Sinclair <mattdsinclair@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Matt Sinclair <mattdsinclair@gmail.com>
In python, when you use -c it consumes all subsequent parameters and
appends them to argv. Now, gem5 and python behave the same with -c.
Python:
> python -c "import sys; print(sys.argv)" --hello -j
['-c', '--hello', '-j']
gem5:
> gem5.opt -c "import sys; print(sys.argv)" --hello -j
gem5 Simulator System. https://www.gem5.org
gem5 is copyrighted software; use the --copyright option for details.
gem5 version [DEVELOP-FOR-22.1]
gem5 compiled Oct 17 2022 15:47:46
gem5 started Oct 17 2022 15:53:45
gem5 executing on challenger, pid 4021103
command line: build/ALL/gem5.opt -c 'import sys; print(sys.argv)' --hello -j
['-c', '--hello', '-j']
Change-Id: I53e87712be9523e0583149235c9787c92618f884
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/63151
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
The conversion logic between gem5 register id and iris resouce id is
duplicated in read and write function. Some of them also doesn't handle
the invalid id correctly. This change wraps the logic, fixes them, and
improves the debug messages by printing the register names.
Change-Id: I093d05f5f06d804d5f01988c2a7ffa60244c5516
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64651
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gabe Black <gabeblack@google.com>
The current protoc_action generates declarations that impose the requirement
for .proto files to import from the BUILDDIR. This prevents .proto files to
import themselves relative to their own directory.
For example, if there are two files build/pkg/a.proto and build/pkg/b.proto,
and b.proto imports a.proto, it must do so as "import pkg/a.proto", as opposed
to "import a.proto"; otherwise, the generated declaration will give rise to a
compilation error.
This is a problem for EXTRAS where .proto files are gem5-independent, so
they cannot import from gem5's build directory. An example is
https://github.com/ARM-software/ATP-Engine
This commit changes the protoc_action so that declarations are generated
relative to the SOURCE directory. This enables .proto files to import
other .proto files within the same directory.
Note it is not possible to import other .proto files in different
directories, but support for it is not currently necessary.
More details on the proto path:
https://developers.google.com/protocol-buffers/docs/reference/cpp-generated#invocation
Related:
https://gem5-review.googlesource.com/c/public/gem5/+/55903
Change-Id: Ib3e67ae817f8ad0b6803c90d23469267eff16178
Signed-off-by: Adrián Herrera Arcila <adrian.herrera@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/64491
Reviewed-by: Jason Lowe-Power <power.jg@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Maintainer: Jason Lowe-Power <power.jg@gmail.com>
