## Processing-in-Memory
### Applicable Workloads
<hr/>

- Fully connected layers have a large weight matrix
  - Weight matrix does not fit onto on-chip cache
  - No data reuse in the matrix

<div class="flex justify-center">
<img src="/dnn.svg">
</div>

<!--
- Workload must be memory-bound

- memory-bound:
  - fully-connected layers
  - layers of recurrent neural networks (RNNs)
  
- not memory-bound:
  - convolutional layers
  	- data reuse
-->

---
preload: false
clicks: 1
---

## Processing-in-Memory
### Applicable Workloads
<hr/>

- Convolutional layers have a small filter matrix
  - Matrix does fit onto on-chip cache
  - Excessive data reuse in the matrix

<br>

<Transform :scale="1.4">
  <div class="absolute left-175px top-1px">
    <img src="/cnn_input.svg">
  </div>
  <div v-if="$slidev.nav.clicks === 0" class="absolute left-175px">
    <img src="/cnn_filter.svg">
  </div>
  <div v-if="$slidev.nav.clicks === 1"
  v-motion
  :initial="{ x: 175, y: 0}"
  :enter="{ x: 335, y: 0, transition: { duration: 5000 }}">
    <img src="/cnn_filter.svg">
  </div>
</Transform>

---

## Processing-in-Memory
### Applicable Workloads
<hr/>

<br>
<br>
<br>
<br>

<div class="grid grid-cols-2 gap-4">
<div>

### Suitable candidates for PIM:
 - Multilayer perceptrons (MLPs)
 - Layers in recurrent neural networks (RNNs)

</div>
<div>

### Unsuitable candidates for PIM:
   - Convolutional neural networks (CNNs)

</div>
</div>

<!--
To summarize...
-->

---

## Processing-in-Memory
### Architectures
<hr/>

<br>
<br>

<div class="grid grid-cols-2 gap-4">
<div>

<v-clicks>

- Inside the memory subarray
- Near the subarray in the PSA output region
- Near the bank in its peripheral region
- In the I/O region of the memory

</v-clicks>

</div>
<div>

<img v-click="[0,1]" class="absolute right-80px top-150px" src="/pim_positions_0.svg">
<img v-click="[1,2]" class="absolute right-80px top-150px" src="/pim_positions_1.svg">
<img v-click="[2,3]" class="absolute right-80px top-150px" src="/pim_positions_2.svg">
<img v-click="[3,4]" class="absolute right-80px top-150px" src="/pim_positions_3.svg">
<img v-click="4" class="absolute right-80px top-150px" src="/pim_positions_4.svg">

</div>
</div>

<br>
<br>
<br>
<br>

<div v-click class="text-xl"> The nearer the computation is to the memory cells, the higher the achievable bandwidth! </div>

<Footnotes separator>
  <Footnote>
  Sudarshan et al. „A Critical Assessment of DRAM-PIM Architectures - Trends, Challenges and Solutions“, 2022.
</Footnote>
</Footnotes>

<!--
- Inside the memory SA
  - Ambit
    - activate multiple rows at the same time
    - bulk logic operations
    
- Near SA in PSA output region
  - CMOS-based logic gates in the region
  
- Near a bank in its peripheral region
  - computation units with control at bank output
  
- I/O region of memory
  - more traditional accelerator approach
-->

---

## Processing-in-Memory
### Samsung's PIM-HBM
<hr/>


<br>

- Real-world PIM implementation based on HBM2
- PIM units embedded at the bank level

<br>


<div class="flex justify-center items-center">
<img src="/hbm-pim.svg">
</div>

<Footnotes separator>
  <Footnote>
  Lee et al. „Hardware Architecture and Software Stack for PIM Based on Commercial DRAM Technology : Industrial Product“, 2021.
</Footnote>
</Footnotes>

<!--
- Real-world PIM implementation based on HBM2
- SIMD FPUs are 16-wide, i.e., there are 16 FPU units
- Three execution modes
    - Single-Bank (SB)
    - All-Bank (AB)
    - All-Bank-PIM (AB-PIM)
-->

---

## Processing-in-Memory
### Samsung's PIM-HBM
<hr/>

<br>

- Two 16-wide 16-bit FPUs
- Register files and control unit

<div class="flex justify-center items-center">
<img src="/pu.svg">
</div>

<Footnotes separator>
  <Footnote>
  Lee et al. „Hardware Architecture and Software Stack for PIM Based on Commercial DRAM Technology : Industrial Product“, 2021.
</Footnote>
</Footnotes>

<!--
- Control unit executes RISC instructions
- Two SIMD FPUs
  - ADD
  - MUL

- CRF: 32 32-bit entries (32 instructions)
- GRF: 16 256-bit entries
- SRF: 16 16-bit entries

- One instruction is executed when RD or WR command is issued
-->

---
layout: figure
figureUrl: /gemv.svg
figureCaption: Procedure to perform a (128×8)×(128) GEMV operation
---

## Processing-in-Memory
### Samsung's PIM-HBM
<hr/>

<Footnotes separator>
  <Footnote>
  Lee et al. „Hardware Architecture and Software Stack for PIM Based on Commercial DRAM Technology : Industrial Product“, 2021.
</Footnote>
</Footnotes>

---
layout: figure
figureUrl: /layout.svg
figureCaption: Mapping of the weight matrix onto the memory banks
---

## Processing-in-Memory
### Samsung's PIM-HBM
<hr/>

<!--
- Data layout in program and address mapping must match
-->

---

## Processing-in-Memory
### Research
<hr/>

<br>
<br>
<br>
<br>

- To analyze the performance gains of PIM, simulation models are needed
- Research should not only focus on hardware but also explore the software side

<br>

- In the following, a virtual prototype of PIM-HBM is implemented