56 lines
5.5 KiB
TeX
56 lines
5.5 KiB
TeX
\section{Introduction}
|
|
\label{sec:introduction}
|
|
|
|
Emerging applications such as \acp{llm} revolutionize modern computing and fundamentally change how we interact with computing systems.
|
|
A key component of these models is the use of \acp{dnn}, which are a type of machine learning model inspired by the structure of the human brain - composed of multiple layers of interconnected nodes that mimic a network of neurons, \acp{dnn} are utilized to perform various tasks such as image recognition or natural language and speech processing.
|
|
Consequently, \acp{dnn} make it possible to tackle many new classes of problems that were previously beyond the reach of conventional algorithms.
|
|
|
|
However, the ever-increasing use of these technologies poses new challenges for hardware architectures, as the energy required to train and run these models reaches unprecedented levels.
|
|
Recently published numbers approximate that the development and training of Meta's LLaMA model over a period of about five months consumed around $\qty{2638}{\mega\watt\hour}$ of electrical energy and caused a total emission of $\qty{1015}{tCO_2eq}$ \cite{touvron2023}.
|
|
As these numbers are expected to increase in the future, it is clear that the energy footprint of current deployment of \ac{ai} applications is not sustainable \cite{blott2023}.
|
|
|
|
|
|
In a more general view, the energy demand of computing for new applications continues to grow exponentially, doubling about every two years, while the world's energy production only grows linearly, at about $\qty{2}{\percent}$ per year \cite{src2021}.
|
|
This dramatic increase in energy consumption is due to the fact that while the energy efficiency of compute processor units has continued to improve, the ever-increasing demand for computing however is outpacing this progress.
|
|
In addition, Moore's Law is slowing down as further device scaling approaches physical limits.
|
|
|
|
\begin{figure}[!ht]
|
|
\centering
|
|
\input{plots/energy_chart}
|
|
\caption[Total energy of computing.]{Total energy of computing \cite{src2021}.}
|
|
\label{plt:enery_chart}
|
|
\end{figure}
|
|
|
|
The exponential grow in compute energy will eventually be constrained by market dynamics, flattening the energy curve and making it impossible to meet future computing demands.
|
|
It is therefore required to achieve radical improvements in energy efficiency in order to avoid such a scenario.
|
|
|
|
In recent years, domain-specific accelerators, such as \acp{gpu} or \acp{tpu} have become very popular, as they provide orders of magnitude higher performance and energy efficiency for \ac{ai} applications than general-purpose processors \cite{kwon2021}.
|
|
However, research must also take into account off-chip memory - moving data between the computation unit and the \ac{dram} is very costly, as fetching operands consumes more power than performing the computation on them itself.
|
|
While performing a double precision floating point operation on a $\qty{28}{\nano\meter}$ technology might consume an energy of about $\qty{20}{\pico\joule}$, fetching the operands from \ac{dram} consumes almost 3 orders of magnitude more energy at about $\qty{16}{\nano\joule}$ \cite{dally2010}.
|
|
|
|
Furthermore, many types of \acp{dnn} used for language and speech processing, such as \acp{rnn}, \acp{mlp} and some layers of \acp{cnn}, are severely limited by the memory bandwidth that the \ac{dram} can provide, making them \textit{memory-bound} \cite{he2020}.
|
|
In contrast, compute-intensive workloads, such as visual processing, are referred to as \textit{compute-bound}.
|
|
|
|
\begin{figure}[!ht]
|
|
\centering
|
|
\input{plots/roofline}
|
|
\caption[Roofline model of GPT revisions.]{Roofline model of GPT revisions \cite{ivobolsens2023}.}
|
|
\label{plt:roofline}
|
|
\end{figure}
|
|
|
|
In the past, specialized types of \ac{dram} such as \ac{hbm} have been able to meet the high bandwidth requirements.
|
|
However, recent \ac{ai} technologies require even greater bandwidth than \ac{hbm} can provide \cite{kwon2021}.
|
|
|
|
All things considered, to meet the need for more energy-efficient computing systems, which are increasingly becoming memory-bound, new approaches to computing are required.
|
|
This has led researchers to reconsider past \ac{pim} architectures and advance them further \cite{lee2021}.
|
|
\Ac{pim} integrates computational logic into the \ac{dram} itself, to exploit minimal data movement cost and extensive internal data parallelism \cite{sudarshan2022}, making it a good fit for memory-bound problems.
|
|
|
|
This work analyzes various \ac{pim} architectures, identifies the challenges of integrating them into state-of-the-art \acp{dram}, examines the changes required in the way applications lay out their data in memory and explores a \ac{pim} implementation from one of the leading \ac{dram} vendors.
|
|
The remainder of this work is structured as follows:
|
|
\cref{sec:dram} gives a brief overview of the architecture of \acp{dram}, in detail that of \ac{hbm}.
|
|
In \cref{sec:pim} various types of \ac{pim} architectures are presented, with some concrete examples discussed in detail.
|
|
\cref{sec:vp} is an introduction to virtual prototyping and system-level hardware simulation.
|
|
After explaining the necessary prerequisites, \cref{sec:implementation} implements a concrete \ac{pim} architecture in software and provides a development library that applications can use to take advantage of in-memory processing.
|
|
The \cref{sec:results} demonstrates the possible performance enhancement of \ac{pim} by simulating a typical neural network inference.
|
|
Finally, \cref{sec:conclusion} concludes the findings and identifies future improvements in \ac{pim} architectures.
|