Files

Matthias Jung 412630122c Added Information about Windows Compiling

2018-06-08 14:30:53 +02:00

46 KiB

Raw Blame History

de.uni-kl.ems.dram.vp.system

DRAMSys [1] is a flexible DRAM subsystem design space exploration framework that consists of models reflecting the DRAM functionality, power consumption, temperature behaviour and retention time errors.

Basic Setup

Open a terminal window, go to your home directory, create a directory for your projects and change to it.

$ cd
$ mkdir projects
$ cd projects

Configure git on your machine. Some basic configurations follow. Replace rhrkuser with your own RHRK user when configuring your email.

$ git config --global user.name "FirstName OtherNames LastName"
$ git config --global user.email rhrkuser@rhrk.uni-kl.de
$ git config --global credential.helper 'cache --timeout=3600'
$ git config --global color.ui auto

Login using your git.rhrk account. Fork the repository by clicking in Fork in the upper-right corner of this page.

Now clone your fork of the official repository into a local folder on your computer. Replace the occurrences of the word user with your own RHRK user name. Note that it appears twice in the line below.

$ git clone --recursive https://user@git.rhrk.uni-kl.de/user/dram.vp.system.git

The --recursive flag tells git to initialize all submodules within the repository. DRAMPower [2] and tinyxml are examples third party repositories that were embedded within the source tree as submodules.

Now you can implement, test, commit and push features into your fork of the official repository.

When you consider your work stable enough to be merged into the official repository it is time to open a pull request using the web interface of git.rhrk.uni-kl.de.

Your changes will be reviewed and finally integrated to the official repository.

After cloning go to the project directory.

$ cd dram.vp.system

When working with a fork, the official repository must be added as a remote for your fork. Replace the word user with your own RHRK user name.

$ git remote add upstream https://user@git.rhrk.uni-kl.de/EIT-Wehn/dram.vp.system.git

In general, from time to time you should update your fork in order to keep it synchronized with the official repository. It is important to keep track of the evolution of the official repository and avoid a huge divergence.

To get the latest changes from the official repository merged into your fork you can use the commands that follow:

$ git fetch upstream
$ git checkout master
$ git merge upstream/master
$ git push origin HEAD
$ git submodule update --init --recursive

Dependencies

Make sure you have properly installed in your system the required libraries and programs.

General dependencies

You can use utils/install.sh in order to install dependencies. First read and understand the script then execute it. Type your password if required.

$ bash install.sh

SystemC 2.3.1 and TLM 2.0

You can use utils/getsysc.sh to download and install SystemC 2.3.1 and TLM 2.0. First read and understand the script then execute it.

$ bash getsysc.sh

Alternatively, the sources can be downloaded from here. For installation instructions see the installation notes file contained in the release package.

qwt-6.1

You can use utils/getqwt.sh in order to install qwt-6.1. First read and understand the script then execute it.

$ bash getqwt.sh
$ cd ~/qwt-6.1
$ sudo make install

Further information about Qwt can be found here.

To grant flexibility to the user the paths where to find some essential libraries and headers can be specified with environment variables. Make sure you have the environment variables below in your ~/.bashrc file.

Note that some of the variables are automatically generated by the scripts. If you install the libraries in custom paths in your system you have to adapt the environment variables accordingly.

# SystemC home and target architecture
export SYSTEMC_HOME=$HOME/systemc-2.3.1a
export SYSTEMC_TARGET_ARCH=linux64

# DRAMSys libraries and headers
export PYTHON_HOME=/usr/lib/python3.5
export PYTHON_HEADERS=/usr/include/python3.5m

export LIBQWT_HOME=/usr/local/qwt-6.1.4-svn/lib
export LIBQWT_HEADERS=/usr/local/qwt-6.1.4-svn/include

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+${LD_LIBRARY_PATH}:}$LIBQWT_HOME
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${SYSTEMC_HOME}/lib-$SYSTEMC_TARGET_ARCH

Coding Style

Please read the coding-style document before starting to code.

A script is provided to apply the coding style.

$ cd util
$ ./make_pretty.sh

Buiding with QTCreator

Execute the QTCreator.

$ qtcreator &

Use the menu bar and open the DRAMSys project.

File -> Open Project -> dram.vp.system/DRAMSys/DRAMSys.pro

When you open the project for the first time a configuration window pops-up. Then click in Configure Project and after that Build the project.

To speedup the building process one can use the additional make option -j[jobs]. The command line below returns a good number to be passed to make as the number of jobs that can run simultaneously to improve the building time.

$ cat /proc/cpuinfo | grep processor | wc -l

In the left bar go to Projects -> Build & Run -> Build Steps -> Make. Click in Details then Make arguments and add -j followed by the number you got.

In case you face a problem related to the Qt version double check the Qt version configuration shown in the image below.

Building without QTCreator

In case you prefer a command line interface to the QTCreator GUI you can also use qmake to generate a Makefile and then compile the project.

$ mkdir build
$ cd build
$ qmake ../DRAMSys/DRAMSys.pro
$ make -j4

The compilation generates executable binary files DRAMSys and traceAnalyzer that can be found inside sub-directories.

From the build directory use the commands below to execute DRAMSys.

$ cd simulator
$ ./DRAMSys

From the build directory use the commands below to execute the traceAnalyzer.

$ cd traceAnalyzer
$ export QT_QPA_PLATFORMTHEME=qgnomeplatform
$ ./traceAnalyzer

Building on MacOS (Formerly OSX)

Install XCode
Install SystemC manually in /opt:

$ ./configure --prefix=/opt/systemc
$ make
$ sudo make install

Or you can install via homebrew:

$ brew install systemc

in this case, systemc will be installed, e.g. in:

/usr/local/Cellar/systemc/2.3.1

and make a simlink of the lib directory:

ln -s lib/ lib-macosx64

Install the required python 3 over homebrew:

$ brew install python3

Python3 (via homebrew) will be installed in

/usr/local/Cellar/python3/3.5.2_2/Frameworks/Python.framework

or you can install manually using official package provided in link

Note: Official Python Package will be installed in

/Library/Frameworks/Python.framework

Install the QtCreator using offical setup file from link

Note: You have later setup PATH for Qt5 and its tool if you install QtCreator manually, e.g:

# Setting PATH for Qt5 and its tools
PATH="/Users/<username>/Qt5.7.0/5.7/clang_64/bin:${PATH}"
export PATH

Install the QWT manually to /opt/qwt, then do:

$ cd /Library/Frameworks
$ sudo ln -s /opt/qwt-6.1.2/lib/qwt.framework/ .

Export correct Environment Variables in your terminal's profile, e.g:

# Setting for DRAMSys
# SystemC via official source
export SYSTEMC_HOME=/opt/systemc
export SYSTEMC_TARGET_ARCH=macosx64
# SystemC via Homebrew
#export SYSTEMC_HOME=/usr/local/Cellar/systemc/2.3.1
#export SYSTEMC_TARGET_ARCH=macosx64

# Python via official pkg
export PYTHON_HOME=/Library/Frameworks/Python.framework/Versions/3.5/lib
export PYTHON_HEADERS=/Library/Frameworks/Python.framework/Versions/3.5/Headers

# Python3 via Homebrew
#export PYTHON_HOME=/usr/local/Cellar/python3/3.5.2_2/Frameworks/Python.framework/Versions/3.5/lib
#export PYTHON_HEADERS=/usr/local/Cellar/python3/3.5.2_2/Frameworks/Python.framework/Versions/3.5/Headers

export LIBQWT_HOME=/opt/qwt-6.1.4/lib
export LIBQWT_HEADERS=/opt/qwt-6.1.4/lib/qwt.framework/Headers

For the trace analyzer the file:

/opt/local/Library/Frameworks/Python.framework/Versions/3.5/include/python3.4m/pyport.h

has to be changed like this

Install package xerces if your system does not have.
Type following command inside your dram.vp.system folder:

$ mkdir build
$ qmake ../DRAMSys/DRAMSys.pro
$ make -j<number_jobs>

Now you can try to run DRAMSys and traceAnalyzer App inside folder simulator and traceAnalyzer, respectively

Building on Windows 10

DRAMSys can also run on Windows 10 with the Windows Subsystem for Linux (WSL) feature. You can install a linux distribution like Debian over the windows app store. We refer to the following example website.

Then DRAMSys can be installed as described above for Linux. However it might be advisable to install Qt, SystemC and qwt by hand. Qt should be downloaded from the Qt website. The install systemc.sh script can be used to install SystemC and for QWT the version 6.1.3 should be used.

Also the build configuration for SystemC should be configured as static with pthreads in the simulator.pro file:

    #LIBS += -L$${systemc_home}/lib-$${systemc_target_arch} -lsystemc
    LIBS += -L$${systemc_home}/lib-$${systemc_target_arch} -Wl,-Bstatic -lsystemc -Wl,-Bdynamic -pthread

DRAMSys Configuration

The DRAMSys executable supports one argument which is a XML file that contains configurable aspects of the desired simulation. If no argument is passed through the command line a default configuration file will be loaded.

The XML code below shows a typic configuration:

<simulation>
    <!-- Simulation file identifier -->
    <simulationid id="ddr3-example"></simulationid>
    <!-- Configuration for the DRAMSys Simulator -->
    <simconfig src="ddr3.xml" />
    <!-- Temperature Simulator Configuration -->
    <thermalconfig src="config.xml" />
    <!-- Memory Device Specification: Which Device is on the DDR3 DIMM -->
    <memspec src="MICRON_1Gb_DDR3-1600_8bit_G.xml"></memspec>
    <!-- Addressmapping Configuration of the Memory Controller -->
    <addressmapping src="am_ddr3_8x1Gbx8_dimm_p1KB_brc.xml"></addressmapping>
    <!-- Memory Controller Configuration: -->
    <mcconfig src="fifoStrict.xml"/>
    <!--
    The following trace setup is only used in standalone mode.
    In library mode e.g. in Platform Architect the trace setup is ignored.
    -->
    <tracesetup>
        <!--
        Specify here a trace file for each of the trace players. Trace
        players without a file will not generate transactions.
        -->
        <device clkMhz="200">ddr3_example.stl</device>
        <device clkMhz="100">ddr3_SAMSUNG_M471B5674QH0_DIMM_example.stl</device>
    </tracesetup>
</simulation>

Some configuration fields reference other XML files which contain more specialized chunks of the configuration like memory specification, address mapping and memory configurations.

The XML configuration files are parsed by the program and the configuration details extracted are assigned to the correspondent attributes of the internal configuration structure.

The device configuration consists of two parameters - clkMhz (operation frequency for this device) - and a trace file.

A trace file is a pre-recorded file containing memory transactions. Each memory transaction has a timestamp that tells the simulator when it shall happen, a transaction type (read or write) and a memory address given in hexadecimal.

Here is an example syntax:

# Comment lines begin with #
# [clock-cyle]: [write|read] [hex-address]
31:	read	0x400140
33:	read	0x400160
56:	write	0x7fff8000
81:	read	0x400180

The timestamp corresponds to the time the request is to be issued and it is given in cycles of the bus master device. Example: the device is a FPGA with frequency 200 MHz (clock period of 5 ns). If the timestamp is 10 it means that the request is to be issued when time is 50 ns.

A trace player is equivalent to a bus master device (processor, FPGA, etc.). It reads an input trace file and translates each line into a new memory request. By adding a new device element into the trace setup section one can specify a new trace player, its operating frequency and the trace file for that trace player.

Configuration File Sections

The main configuration file is divided into self-contained sections. Each of these sections refers to sub-configuration files.

Below, the sub-configurations are listed and explained.

Simulator Configuration

The content of ddr3.xml is presented below as an example.
```
<simconfig>
    <SimulationName value="ddr3" />
    <Debug value="0" />
    <DatabaseRecording value="1" />
    <PowerAnalysis value="1" />
    <EnableWindowing value = "1" />
    <WindowSize value="100" />
    <ThermalSimulation value="0"/>
    <SimulationProgressBar value="1"/>
    <NumberOfMemChannels value="1"/>
    <NumberOfDevicesOnDIMM value = "8" />
    <ECCControllerMode value = "Disabled" />
</simconfig>
```
- SimulationName (boolean)
  - Give the name of the simulation for distingushing from other simulations.
- Debug (boolean)
  - "1": enables debug output on console
  - "0": disables debug output
- DatabaseRecording (boolean)
  - "1": enables trace file recording for the trace analyser tool
  - "0": disables trace file recording
- PowerAnalysis (boolean)
  - "1": enables live power analysis with the DRAMPower tool
  - "0": disables power analysis
- EnableWindowing (boolean)
  - "1": enables temporal windowing
  - "0": disables temporal windowing
- WindowSize (unisgned int)
  - Size of the window in clock cycles used to evaluate average bandwidth and average power consumption
- NumberOfMemChannels (unsigned int)
  - Number of memory channels
- ThermalSimulation (boolean)
  - "1": enables thermal simulation
  - "0": static temperature during simulation
- SimulationProgressBar (boolean)
  - "1": enables the simulation progress bar
  - "0": disables the simulation progress bar
- NumberOfDevicesOnDIMM (unsigned int)
  - Number of devices on dual inline memory module
- CheckTLM2Protocol (boolean)
  - "1": enables the TLM 2.0 Protocol Checking
  - "0": disables the TLM 2.0 Protocol Checking
- ECCControllerMode (string)
  - "Disabled": No ECC Controller is used
  - "Hamming": Enables an ECC Controller with classic SECDED implementation using Hamming Code
Temperature Simulator Configuration

The content of config.xml is presented below as an example.
```
<thermalsimconfig>
    <TemperatureScale value="Celsius" />
    <StaticTemperatureDefaultValue value="89" />
    <ThermalSimPeriod value="100" />
    <ThermalSimUnit value="us" />
    <PowerInfoFile value="../../DRAMSys/library/resources/configs/thermalsim/powerInfo.xml"/>
    <IceServerIp value="127.0.0.1" />
    <IceServerPort value="11880" />
    <SimPeriodAdjustFactor value="10" />
    <NPowStableCyclesToIncreasePeriod value="5" />
    <GenerateTemperatureMap value="1" />
    <GeneratePowerMap value="1" />
</thermalsimconfig>
```
- TemperatureScale (string)
  - "Celsius"
  - "Fahrenheit"
  - "Kelvin"
- StaticTemperatureDefaultValue (int)
  - Temperature value for simulations with static temperature
- ThermalSimPeriod (double)
  - Period of the thermal simulation
- ThermalSimUnit (string)
  - "s": seconds
  - "ms": millisecond
  - "us": microseconds
  - "ns": nanoseconds
  - "ps": picoseconds
  - "fs": femtoseconds
- PowerInfoFile (string)
  - File containing power related information: devices identifiers, initial power values and power thresholds.
- IceServerIp (string)
  - 3D-Ice server IP address
- IceServerPort (unsigned int)
  - 3D-Ice server port
- SimPeriodAdjustFactor (unsigned int)
  - When substantial changes in power occur (i.e., changes that exceed the thresholds), then the simulation period will be divided by this number causing the thermal simulation to be executed more often.
- NPowStableCyclesToIncreasePeriod (unsigned int)
  - Wait this number of thermal simulation cycles with power stability (i.e., changes that do not exceed the thresholds) to start increasing the simulation period back to its configured value.
- GenerateTemperatureMap (boolean)
  - "1": generate temperature map files during thermal simulation
  - "0": do not generate temperature map files during thermal simulation
- GeneratePowerMap (boolean)
  - "1": generate power map files during thermal simulation
  - "0": do not generate power map files during thermal simulation
Memory Specification

A file with memory specifications. This information comes from datasheets and measurements, and usually does not change.
Address Mapping

There are currently two different file formats to describe the address mapping. This software automatically chooses the correct interpreter using the file extension as selection criterion. So please make sure that all files have the correct extension.

XML file format

XML files describe the address mapping to be used in the simulation.

Example for 1GB x64 DIMM with: 8 x 1 Gbit x8 Devices (Micron MT41J128M8) with Page Size: 1KB

am_ddr3_8x1Gbx8_dimm_p1KB_brc.xml

<!--
DDR3 Example:
1GB x64 DIMM with: 8 * 1 Gb x8 Devices (e.g. Micron MT41J128M8) with Page Size: 1KB

Device Characteristics:

Rows: 16 K [13:0]  -> 14 bit
Bank: 8    [2:0]   -> 3  bit
Cols: 1 K  [9:0]   -> 10 bit

Due to the DIMM we have a Byte Offset Y

2 2 2 | 2 2 2 2 2 2 2 1 1 1 1 1 1 1 | 1 1 1
9 8 7 | 6 5 4 3 2 1 0 9 8 7 6 5 4 3 | 2 1 0 9 8 7 6 5 4 3 | 2 1 0
B B B | R R R R R R R R R R R R R R | C C C C C C C C C C | Y Y Y
-->

<addressmapping>
    <channel from="128"  to="128"  /> <!-- only one channel -->
    <bank    from="27"   to="29"   />
    <row     from="13"   to="26"   />
    <column  from="3"    to="12"   />
    <bytes   from="0"    to="2"    />
</addressmapping>

Some more examples with graphical representation follow:

am_wideio.xml

<!-- Row Bank Column -->

<addressmapping>
    <channel from="27" to="28" />
    <row   from="14" to="26" />		
    <bank  from="11" to="13" />
    <column from="4" to="10" />
    <bytes from="0" to="3" />
</addressmapping>

<!-- Bank Row Column -->

<addressmapping>
    <channel from="27" to="28" />
    <bank  from="24" to="26" />
    <row   from="11" to="23" />
    <column from="4" to="10" />
    <bytes from="0" to="3" />
</addressmapping>

JSON file format

This file format is generated by ConGen. It does not have an unambiguous assignment of the address lines.

The format delivers more information than needed for an address mapping. Unused data:

Block "Config": Gives you information about the ConGen configuration
Key "Name": Name of the trace file which was used by ConGen
All items of the array "Solutions" but the first one: Alternative solution with same result.
Key "costs": Number of row misses which this configuration produces while playing the trace.

Used data:

First item of array "Solution":
- "XOR": Array of row and bank bits which are connected with an xor. Order of the bit: bank1, bank2, ..., row1, row2, ...
- "bank_bits": Number of the addres bits which are connected to a bank bit
- "row_bits": Number of the addres bits which are connected to a row bit

{
  "Config": {
    "numberOfBankBits": 3,
    "numberOfRowBits": 14,
    "numberOfColumnBits": 10,
    "numberOfByteBits": 3,
    "numberOfBLBits": 3
  },
  "Name": "trace_name",
  "Solutions": [
    {
      "XOR": [
      ],
      "Banks Rows": [
        {
          "bank_bits": [
            27,
            28,
            29
          ],
          "rows": {
            "row_bits": [
              13,
              14,
              15,
              16,
              17,
              18,
              19,
              20,
              21,
              22,
              23,
              24,
              25,
              26
            ],
            "costs": 477468
          },
          "costs": 477468
        }
      ]
    }
  ]
}


- **Memory Configuration**

  The content of [fifo.xml](DRAMSys/library/resources/configs/memconfigs/fifo.xml) is
  presented below as an example.

  ``` xml
  <mcconfig>
      <BankwiseLogic value="0"/>
      <OpenPagePolicy value="1" />
      <MaxNrOfTransactions value="8" />
      <Scheduler value="FIFO" />
      <Capsize value="5" />
      <!-- 4 Modes: NoPowerDown, Staggered, TimeoutPDN, TimeoutSREF -->
      <PowerDownMode value="NoPowerDown" />
      <PowerDownTimeout value="100" />
      <!-- Error Modelling -->
      <ErrorChipSeed value="42" />
      <ErrorCSVFile value="../../DRAMSys/library/src/error/error.csv" />
      <!-- Modes:
          - NoStorage,
          - Store (store data without errormodel),
          - ErrorModel (store data with errormodel)
      -->
      <StoreMode value="NoStorage" />
      <ControllerCoreDisableRefresh value="0"/>
  </mcconfig>

BankwiseLogic (boolean)
- "1": perform bankwise-refresh [3] and bankwise-powerdown [4]
- "0": do not perform bankwise operations
OpenPagePolicy (boolean)
- "1": use open page precharge policy
- "0": do not use open page precharge policy
MaxNrOfTransactions (unsigned int)
- Maximum number of transactions.
Scheduler (string)
- "FIFO": first in, first out
- "FIFO_STRICT": out-of-order treatment of queue elements not allowed
- "FR_FCFS": first-come, first-served
Capsize (unsigned int)
- Capacitor cell size.
PowerDownMode (enum EPowerDownMode)
- "NoPowerDown": no power down mode (active idle)
- "Staggered": staggered power down policy [5]
- "TimeoutPDN": precharge idle
- "TimeoutSREF": self refresh
ReadWriteGrouping (boolean)
- "1": enable read writing grouping
- "0": disable read writing grouping
ReorderBuffer (boolean)
- "1": use reordering buffer
- "0": do not use reordering buffer
ErrorChipSeed (unsigned int)
- Seed to initialize the random error generator.
ErrorCSVFile (string)
- CSV file with error injection information.
StoreMode (enum StorageMode)
- "NoStorage": no storage
- "Store": store data without error model
- "ErrorModel": store data with error model [6]
ControllerCoreDisableRefresh (boolean)
- "1": disables refreshes
- "0": normal operation (refreshes enabled)
ControllerCoreForceMaxRefBurst (boolean)
- "1": always postpone, resulting in a ControllerCoreMaxPostponedARCmd burst
- "0": normal operation
ControllerCoreEnableRefPostpone (boolean)
- "1": enables the postpone refresh feature
- "0": normal operation
ControllerCoreEnableRefPullIn (boolean)
- "1": enables the pull-in refresh feature
- "0": normal operation
ControllerCoreMaxPostponedARCmd (unsigned int)
- Max AR commands to be postponed.
ControllerCoreMaxPulledInARCmd (unsigned int)
- Max AR commands to be pulled-in.

For further details on the Flexible Refresh feature, please refer to this document.

Trace Setups
- clkMhz (unsigned int)
  - Speed of the trace player
- trace file
  - A pre-recorded file containing memory transactions to be executed by a trace player.

Some attributes are self-explanatory while others require some previous knowhow of memory technologies.

Resources of the simulator are available inside of the resources directory and its sub-directories.

$ cd DRAMSys/library/resources

A description of the content each directory follows.

resources
- configs: XML files that specify details of the simulation.
  - amconfigs: address mapping configs.
  - mcconfigs: memory controller configs.
  - memspecs: memory specification files (technology dependent).
  - simulator: simulator configs.
- scripts: useful tools.
- simulations: main configuration files.
- traces: pre-recorded trace files that may be used as stimuli in simulations.

Log Collector Script

Users can profit of running multiple simulations automatically with DRAMSylva.

Every time you run the script you get a new folder with the name containing the execution time: dram.vp.system_YYYY_MM_DD-HH.MM.SS.

Example on how to run the script:

$ cd DRAMSys/library/resources/scripts/DRAMSylva
$ ./DRAMSylva.sh

To see the generated plots and CSV files:

$ nautilus dram.vp.system_YYYY_MM_DD-HH.MM.SS/build/simulator

In that folder you will find plots as PDF files and CSV files with the output data used to generate the plots. The CSV files are:

output.csv (energy, average power, bandwidth, etc.)
metrics.csv (DRAMSys metrics like average response latency, memory utilization and many others)

The generated CSV files can be open in a spreadsheet program for further manipulation.

Additionally, the database files (*.tdb) generated will be available and can be open with the traceAnalyzer tool.

A DRAMSys simulation is defined by the main configuration file passed to the simulator. The main configuration file includes other files which contain specifc configs. You can change what is going to be simulated by the script by editing it. There is a list of main configuration files on the top of the script:

sim_files="
../../DRAMSys/library/resources/simulations/ddr3-example.xml
../../DRAMSys/library/resources/simulations/ddr3-single-device.xml
../../DRAMSys/library/resources/simulations/wideio-example.xml
"

Set the variable use_trace_list to yes in order to use all traces from the trace list for all simulation files. Each pair generates a new simulation. Otherwise it runs a simulation per simulation file. The trace specified inside the simulation file is used.

use_trace_list="yes"

When you choose yes then traces have to be explicitly added to the traces list. A new simulation is defined by a combination of two elements, one from the simulation files list and the other from the traces list. Thus the number of simulations executed is the number of simulation files multiplied by the number of traces.

traces="
ddr3_example.stl
mediabench-unepic_32.stl
"

The script runs one instance of DRAMSys for each of the files in the list. The multiple instances run in parallel.

For more information check the documentation in DRAMSylva folder.

Trace Generator Script

The trace_gen script for generating input traces for simple tests is provided.

Example on how to run the script:

$ cd DRAMSys/library/resources/scripts
$ ./trace_gen.py > trace.stl

Now change your configuration file to use the new generated trace file and run your simulation.

The script can be easily changed and provides a way to quickly generate accesses to all channels, all bank groups, all banks, all rows and all columns of a memory.

Be aware that a trace which covers all rows and all columns may be huge (several gigabytes) depending on your memory.

The defaul values in the script serve as an example. They consider the address mapping that follows.

DDR3-SDRAM DIMM Characteristics:
Byte Offset (Y): 8    [0:2]   (8-byte-wide memory module, i.e., 64-bit-wide data bus) -> 3 bit
Cols        (C): 1K   [3:12]  (A0 - A9)   -> 10 bit
Rows        (R): 128K [13:29] (A0 - A16)  -> 17 bit
Bank        (B): 8    [30:32] (BA0 - BA2) -> 3  bit

3 3 3 | 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 | 1 1 1
2 1 0 | 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 | 2 1 0 9 8 7 6 5 4 3 | 2 1 0
B B B | R R R R R R R R R R R R R R R R R | C C C C C C C C C C | Y Y Y

The parameters for the address mapping just described are presented below.

# Channel information.
num_ch = 1              # Number of channels
ch_shift = 34           # Shift to reach the frist bit reserved for channels in the address
ch_mask = 0x1           # Mask for all channel bits in the address

# Bank group information.
num_bank_groups = 1     # Number of bank groups
bgroup_shift = 33   	# Shift to reach the frist bit reserved for bank groups in the address
bgroup_mask = 0x1   	# Mask for all bits in the address related to bank groups

# Bank information
num_banks = 8           # Number of banks
bank_shift = 30         # Shift to reach the frist bit reserved for banks in the address
bank_mask = 0x7         # Mask for all bank bits in the address

# Row information
num_rows = 128 * 1024   # Number of rows
row_shift = 13          # Shift to reach the frist bit reserved for rows in the address
row_mask = 0x1ffff      # Mask for all row bits in the address

# Column information
num_col = 1 * 1024      # Number of columns
col_shift = 3           # Shift to reach the frist bit reserved for columns in the address
col_mask = 0x3ff        # Mask for all column bits in the address

# Burst length
burst_len = 8

Open the script with a text editor and change some parameters to fit your needs.

DRAMsys Diagrams

TLM Approximately Timed (AT)

The figure below shows a cheat sheet with the possibilities that the TLM AT protocol offers. The annotated references [X,Y] are placed into the source code for a better orientation.
Payload Extension information

GenerationExtension is added in TracePlayer and DramExtension is added in Arbiter.

DramExtension indicates the decoded address (channel, bank, colums, row) and the socket id (thread) of a payload. It is added in the Arbiter and is sent to the Controller.
Transaction object with Memory Manager

The TracePlayer allocates the memory for the transaction object by calling allocatePayload method.

The acquire method is called before passing the transaction object in TracePlayer, Arbiter and Controller.

The release method is called after each component is done with the transaction object. After the final call of release method, the free method of the memory manager is called to free the transaction object.
Architecture of the backend TLM model

The below figure shows our custom TLM protocol between the Controller and the Dram. A new transaction enters the Controller with the BEGIN_REQ phase is stored in frontendPEQ. The callback function of the frontendPEQ is called and send the payload to the Scheduler.

The Scheduler checks the address of payload and the current state to determine proper command (Active, Precharge, Read or Write). Then the ControllerCore sends the payload with the corresponding phase (BEGIN_ACT, BEGIN_PRE, BEGIN_RD or BEGIN_WR) to the Dram by calling nb_transport_fw method.

The Dram receives the transaction then send back to the Controller by calling nb_transport_bw with appropriate END phase (END_ACT, END_PRE, END_RD or END_WR).

DRAMSys Thermal Simulation

The thermal simulation is performed by a 3D-ICE [8] server accessed through the network. Therefore users interested in thermal simulation during their DRAMSys simulations need to make sure they have a 3D-ICE server up and running before starting. For more information about 3D-ICE visit the official website.

Installing the lastest 3D-ICE version

Download the lastest version. Make sure you got version 2.2.6 or greater:

$ wget http://esl.epfl.ch/files/content/sites/esl/files/3dice/releases/3d-ice-latest.zip
$ tar -xvzf 3d-ice-latest.zip

Install SuperLU dependencies:

$ sudo apt-get install build-essential git bison flex libblas-dev

Download and install SuperLU:

$ wget http://crd.lbl.gov/~xiaoye/SuperLU/superlu_4.3.tar.gz
$ tar xvfz superlu_4.3.tar.gz
$ cd SuperLU_4.3/
$ cp MAKE_INC/make.linux make.inc

Make sure the SuperLUroot variable in ./make.inc is properly set (in my case $(HOME)/repos/).

SuperLUroot = $(HOME)/repos/SuperLU_4.3

Compile the library:

$ make superlulib

Download and install bison-2.4.1:

$ wget http://ftp.gnu.org/gnu/bison/bison-2.4.1.tar.gz
$ tar xvzf bison-2.4.1.tar.gz
$ cd bison-2.4.1
$ ./configure --program-suffix=-2.4.1
$ make
$ sudo make install

Go to the 3d-ice directory:

$ cd 3d-ice-2.2.6

Open the file makefile.def and set some variables. Set the correct path to the SuperLU library you just compiled (in my case $(HOME)/repos/):

SLU_MAIN = $(HOME)/repos/SuperLU_$(SLU_VERSION)

Set the YACC variable to bison-2.4.1:

YACC = bison-2.4.1

Set the systemC architecture and main folder variables:

SYSTEMC_ARCH = [linux,linux64]
SYSTEMC_MAIN = $(HOME)/repos/systemc-$(SYSTEMC_VERSION)

Compile 3D-ICE with SystemC/TLM2.0 support:

$ make clean
$ make SYSTEMC_WRAPPER=y

Users interested in thermal simulation can also add some extra environment variables:

# Necessary for thermal simulation
export LIBTHREED_ICE_HOME=<path>
export LIBSUPERLU_HOME=<path>

Running DRAMSys with Thermal Simulation

Before starting make sure you have a clean repository without any previous automatic generated Makefiles. One way to ensure this is by running the command below inside your DRAMSys repository, but keep in mind that untracked files and directories will be removed from the repository.

$ git clean -fdx

This feature can be enabled via an environment variable.

$ export THERMALSIM=true
$ qtcreator &

$ mkdir build
$ cd build
$ export THERMALSIM=true
$ qmake ../DRAMSys/DRAMSys.pro
$ make

Before starting DRAMSys it is necessary to run the 3D-ICE server passing to it two arguments: a suitable configuration file and an Internet socket port number. And then wait until the server is ready to receive requests.

$ 3D-ICE-Server <stack file> <port>
Preparing stk data ... done !
Preparing thermal data ... done !
Creating socket ... done !
Waiting for client ... done !

The IP address and the port number related to the server shall be informed in DRAMSys' configuration to subsequent use by DRAMSys to access the thermal simulation server.

Usage Example with Thermal Simulation

The DRAMSys' main configuration file is presented below.

<simulation>
    <!-- Configuration for the DRAMSys Simulator -->
    <simconfig src="wideio_thermal.xml" />
    <!-- Temperature Simulator Configuration -->
    <thermalconfig src="confi.xml" />
    <!-- Memory Device Specification: Which Device is on the DDR3 DIMM -->
    <memspec src="WideIO.xml"></memspec>
    <!-- Addressmapping Configuration of the Memory Controller -->
    <addressmapping src="am_wideio.xml"></addressmapping>
    <!-- Memory Controller Configuration: -->
    <mcconfig src="fr_fcfs.xml"/>
    <!--
    The following trace setup is only used in standalone mode.
    In library mode e.g. in Platform Architect the trace setup is ignored.
    -->
    <tracesetup>
        <!--
        This device mimics an image processing application
        running on an FPGA with 200 Mhz.
        -->
        <device clkMhz="1000">test_error.stl</device>
    </tracesetup>
</simulation>

Enable the error model in fr_fcfs.xml.

<memconfig>
    <BankwiseLogic value="0"/>
    <OpenPagePolicy value="1" />
    <MaxNrOfTransactions value="8" />
    <Scheduler value="FR_FCFS" />
    <Capsize value="5" />
    <PowerDownMode value="NoPowerDown"/>
    <PowerDownTimeout value="100" />
    <!-- Error Model: -->
    <ErrorChipSeed value="42" />
    <ErrorCSVFile value="../../DRAMSys/library/src/error/error.csv" />
    <!-- Modes: NoStorage, Store (store data without errormodel), ErrorModel (store data with errormodel) -->
    <StoreMode value="ErrorModel" />
</memconfig>

Generate the input trace file for DRAMSys.

$ cd DRAMSys/tests/error/
$ ./generateErrorTest.pl > test_error.stl

Start the 3D-ICE server providing the stack file and the port number.

$ cd DRAMSys/library/resources/configs/thermalsim
$ 3D-ICE-Server stack.stk 11880

In another terminal or terminal tab start DRAMSys. Here the program's output is redirected to a file.

$ cd build/simulator/
$ ./DRAMSys > output

DRAMSys with gem5

Install gem5 by following the instructions on the gem5 wiki http://gem5.org/Documentation#Getting_Started. In order to understand the SystemC coupling with gem5 it is recommended to study util/tlm/README and [11]. The main steps for building gem5 with SystemC are:

scons build/ARM/gem5.opt
scons --with-cxx-config --without-python build/ARM/libgem5_opt.so
#for mac: scons --with-cxx-config --without-python build/ARM/libgem5_opt.dylib
cd util/tlm

In order to use gem5 with DRAMSys set the GEM5 environment variable to the path to gem5, for example in the QtCreator under Projects > Build & Run > Build Environment:

GEM5=/path/to/gem5/

DRAMSys will detect gem5 and configures automatically the according compile target gem5 for QtCreator.

In order to run gem5 with DRAMSys its mandatory to run gem5 first without DRAMSys in order to generate an ini file which will be read by the DRAMSys_gem5 binary. In the following several examples for DRAMSys gem5 bindings are shown.

DRAMSys with gem5 traffic generator

In the following we will run a simple example with a gem5 traffic generator:

Base System Architecture:
+-------------+ +------+    ^
| System Port | | TGEN |    |
+-------+-----+ +--+---+    |
        |           |       | gem5 World
        |      +----+       | (see this file)
        |      |            |
+-------v------v-------+    |
|        Membus        |    v
+---------------+------+    External Port (see sc_slave_port.*)
                |           ^
           +----v----+      | TLM World
           | DRAMSys |      | (see sc_target.*)
           +---------+      v

As mentioned before we first need to create a config.ini that represents the gem5 configuration. We do so by starting gem5 with the desired python configuration script.

cd gem5/utils/tlm/
../../build/ARM/gem5.opt conf/tlm_slave.py

The message

"fatal: Can't find port handler type 'tlm_slave'"

is totally okay. The configuration file config.ini will be stored in the m5out/ directory. Copy this configuration file to the building directory of DRAMSys:

dram.vp.system/build-DRAMSys-Desktop_Qt_5_7_0_clang_64bit-Debug/gem5

Also the traffic generatior configuration file (conf/tgen.cfg) must be stored in a conf directory of this building directory.

Then the simulation can be started with:

./DRAMSys_gem5 /path/to/dram.vp.system/DRAMSys/library/resources/simulations/ddr3-example.xml config.ini

Lets run the simulation for some seconds and then stop the simulation with ctrl+c. Observe the output of the simulation in the trace analyzer. The trace database file is stored in the gem5 directory in the building directory.

Boot Linux with gem5 and DRAMSys

The procedure is very similar to the traffic generator example above.

First we have to generate the config.ini file by starting gem5 with the following configuration:

build/ARM/gem5.opt configs/example/fs.py \
  --tlm-memory=transactor --cpu-type=TimingSimpleCPU --num-cpu=1 \
  --mem-type=SimpleMemory --mem-size=512MB --mem-channels=1 --caches \
  --l2cache --machine-type=VExpress_EMM \
  --dtb-filename=vexpress.aarch32.ll_20131205.0-gem5.1cpu.dtb \
  --kernel=vmlinux.aarch32.ll_20131205.0-gem5 \
  --disk-image=linux-aarch32-ael.img

The config.ini should be copied again to the DRAMSys_gem5 build folder.

The simconfig should be changed in order to support storage and address offsets:

<simconfig>
    <SimulationName value="ddr3" />
    <Debug value="0" />
    <DatabaseRecording value="1" />
    <PowerAnalysis value="1" />
    <EnableWindowing value = "1" />
    <WindowSize value="1000" />
    <ThermalSimulation value="0"/>
    <SimulationProgressBar value="1"/>
    <NumberOfMemChannels value="1"/>
    <NumberOfDevicesOnDIMM value = "8" />
    <CheckTLM2Protocol value = "0" />
    <AddressOffset value = "0" />
    <ECCControllerMode value = "Disabled" />
    <ErrorChipSeed value="42" />
    <ErrorCSVFile value="" />
    <!-- Modes:
	- NoStorage,
	- Store (store data without errormodel),
	- ErrorModel (store data with errormodel)
    -->

    <!-- Gem5 Related Configuration:
    In the memory controller file the storage mode should be set to Store
    E.g. the DRAM is located at 0x80000000 for gem5
    -->
    <StoreMode value="Store" />
    <AddressOffset value = "2147483648" />
</simconfig>

Then start DRAMSys_gem5 with the following command:

./DRAMSys_gem5 /Users/myzinsky/EMS/Programming/dram.vp.system/DRAMSys/library/resources/simulations/ddr3-example.xml config.ini

For further sophisticated address mappings or scenarios checkout the file DRAMSys/gem5/main.cpp

DRAMSys with gem5 Elastic Traces

For understanding elastic traces and their generation, study the gem5 wiki (http://gem5.org/TraceCPU) and the paper [13]. Some predefined configs are stored in dram.vp.system/DRAMSys/gem5/configs and the related python files are stored here: dram.vp.system/DRAMSys/gem5/examples

This is an example for running an elastic trace:

./DRAMSys_gem5 /path/to/dram.vp.system/DRAMSys/library/resources/simulations/ddr3-example.xml ../../DRAMSys/gem5/configs/singleElasticTraceReplay.ini

Note that the address offset is usually zero for elastic traces.

If two elastic traces should be used the main.cpp must be modified:

//#define CHOICE1
#define CHOICE2
//#define CHOICE3

Run the simulation with the following example:

./DRAMSys_gem5 /path/to/dram.vp.system/DRAMSys/library/resources/simulations/ddr3-example.xml ../../DRAMSys/gem5/configs/dualElasticTraceReplay.ini

For more spophisticated setups, even with l2 caches the proper ini file should be created. If you need help please contact Matthias Jung.

References

[1] TLM Modelling of 3D Stacked Wide I/O DRAM Subsystems, A Virtual Platform for Memory Controller Design Space Exploration M. Jung, C. Weis, N. Wehn, K. Chandrasekar. International Conference on High-Performance and Embedded Architectures and Compilers 2013 (HiPEAC), Workshop on: Rapid Simulation and Performance Evaluation: Methods and Tools (RAPIDO), January, 2013, Berlin.

[2] DRAMPower: Open-source DRAM Power & Energy Estimation Tool Karthik Chandrasekar, Christian Weis, Yonghui Li, Sven Goossens, Matthias Jung, Omar Naji, Benny Akesson, Norbert Wehn, and Kees Goossens URL: http://www.drampower.info

[3] Energy Optimization in 3D MPSoCs with Wide-I/O DRAM M. Sadri, M. Jung, C. Weis, N. Wehn, L. Benini. Conference Design, Automation and Test in Europe (DATE), March, 2014, Dresden, Germany.

[4] DRAMSys: A flexible DRAM Subsystem Design Space Exploration Framework M. Jung, C. Weis, N. Wehn. Accepted for publication, IPSJ Transactions on System LSI Design Methodology (T-SLDM), October, 2015.

[5] Optimized Active and Power-Down Mode Refresh Control in 3D-DRAMs M. Jung, M. Sadri, C. Weis, N. Wehn, L. Benini., VLSI-SoC, October, 2014, Playa del Carmen, Mexico.

[6] Retention Time Measurements and Modelling of Bit Error Rates of WIDE-I/O DRAM in MPSoCs C. Weis, M. Jung, P. Ehses, C. Santos, P. Vivet, S. Goossens, M. Koedam, N. Wehn. Accepted for publication, IEEE Conference Design, Automation and Test in Europe (DATE), March, 2015, Grenoble, France

[7] http://www.uni-kl.de/3d-dram/publications/

[8] A Sridhar, A Vincenzi, D Atienza, T Brunschwiler, 3D-ICE: a compact thermal model for early-stage design of liquid-cooled ICs, IEEE Transactions on Computers (TC 2013, accepted for publication).

[9] A Sridhar, A Vincenzi, M Ruggiero, T Brunschwiler, D Atienza, 3D-ICE: Fast compact transient thermal modeling for 3D-ICs with inter-tier liquid cooling, Proceedings of the 2010 International Conference on Computer-Aided Design (ICCAD 2010), San Jose, CA, USA, November 7-11 2010.

[10] A Sridhar, A Vincenzi, M Ruggiero, T Brunschwiler, D Atienza, Compact transient thermal model for 3D ICs with liquid cooling via enhanced heat transfer cavity geometries, Proceedings of the 16th International Workshop on Thermal Investigations of ICs and Systems (THERMINIC'10), Barcelona, Spain, 6-8 October, 2010.

[11] http://esl.epfl.ch/3D-ICE

[12] System Simulation with gem5 and SystemC: The Keystone for Full Interoperability C. Menard, M. Jung, J. Castrillon, N. Wehn. IEEE International Conference on Embedded Computer Systems Architectures Modeling and Simulation (SAMOS), July, 2017, Samos Island, Greece.

[13] Exploring System Performance using Elastic Traces: Fast, Accurate and Portable Radhika Jagtap, Stephan Diestelhorst, Andreas Hansson, Matthias Jung and Norbert Wehn, IEEE International Conference on Embedded Computer Systems Architectures Modeling and Simulation (SAMOS), 2016, Samos Island, Greece.

46 KiB Raw Blame History