From b554efe3e88af70d1e22db61cd1e8219b0240942 Mon Sep 17 00:00:00 2001 From: Derek Christ Date: Sun, 11 Feb 2024 20:38:26 +0100 Subject: [PATCH] FIMDRAM Instruction Ordering --- src/acronyms.tex | 8 +++++ src/chapters/pim.tex | 80 ++++++++++++++++++++++++++++++++++++------- src/images/aam.pdf | Bin 0 -> 24413 bytes src/index.tex | 20 +++++------ 4 files changed, 85 insertions(+), 23 deletions(-) create mode 100644 src/images/aam.pdf diff --git a/src/acronyms.tex b/src/acronyms.tex index 7dce1fd..c71d8dc 100644 --- a/src/acronyms.tex +++ b/src/acronyms.tex @@ -227,6 +227,14 @@ short = AAM, long = address aligned mode, } +\DeclareAcronym{ld}{ + short = LD, + long = load, +} +\DeclareAcronym{st}{ + short = ST, + long = store, +} \DeclareAcronym{tlm}{ short = TLM, long = transaction-level modeling, diff --git a/src/chapters/pim.tex b/src/chapters/pim.tex index 44a4bf5..2799ce2 100644 --- a/src/chapters/pim.tex +++ b/src/chapters/pim.tex @@ -125,10 +125,12 @@ As a result, Newton promises a $\qtyrange{10}{54}{\times}$ speedup compared to a \subsection{FIMDRAM/HBM-PIM} \label{sec:pim_fim} -\subsubsection{Architecture} - One year after SK Hynix, the major \ac{dram} manufacturer Samsung announced its own \ac{pim} \ac{dram} implementation, called \ac{fimdram} or \ac{hbm}-\ac{pim}. -As the name suggests, it is based on the \aca{hbm} memory standard, and it integrates 16-wide \ac{simd} engines directly into the memory banks, exploiting bank-level parallelism, while retaining the highly optimized \acp{subarray} \cite{kwon2021}. +As this is the \ac{pim} architecture which was implemented during the work on this thesis, it will be explained in great detail. +The following subsections are mainly based on \cite{lee2021} and \cite{kwon2021}, with the subsection \ref{sec:memory_layout} being mainly based on \cite{kang2022}. + +\subsubsection{Architecture} +As the name of \ac{hbm}-\ac{pim} suggests, it is based on the \aca{hbm} memory standard, and it integrates 16-wide \ac{simd} engines directly into the memory banks, exploiting bank-level parallelism, while retaining the highly optimized \acp{subarray} \cite{kwon2021}. A major difference from Newton \ac{pim} is that \ac{hbm}-\ac{pim} does not require any changes to components of modern processors, such as the memory controller, i.e. it is agnostic to existing \aca{hbm} platforms. Consequently, mode switching is required for \ac{hbm}-\ac{pim}, making it less useful for interleaved \ac{pim} and non-\ac{pim} traffic. Fortunately, as discussed in Section \ref{sec:hbm}, the architecture of \ac{hbm} allows for many independent memory channels on a single stack, making it possible to cleanly separate the memory map into a \ac{pim}-enabled region and a normal \ac{hbm} region. @@ -167,16 +169,18 @@ Both in all-bank mode and in all-bank-\ac{pim} mode, the total \aca{hbm} bandwid \subsubsection{Processing Unit} -Due to the focus on \ac{dnn} applications in \ac{hbm}-\ac{pim}, the native data type for the \acp{fpu} is \ac{fp16}, which is motivated by the significantly lower area and power requirements for \acp{fpu} compared to \ac{fp32}, as well as the good support of \ac{fp16} for modern processor architectures. +Due to the focus on \ac{dnn} applications in \ac{hbm}-\ac{pim}, the native data type for the \acp{fpu} is \ac{fp16}, which is motivated by the significantly lower area and power requirements for \acp{fpu} compared to \ac{fp32}. +In addition, \ac{fp16} is well supported on modern processor architectures such as ARMv8. The \ac{simd} \ac{fpu} is implemented once as a \ac{fp16} multiplier unit, and once as a \ac{fp16} adder unit, providing support for these basic algorithmic operations. In addition to the \acp{fpu}, a processing unit consists also of \acp{crf}, \acp{srf} and \acp{grf}. -The \ac{crf} acts as an instruction buffer, holding the 32 32-bit instructions to be executed by the processor when accessing memory. -As explained earlier, the operands come either directly from the bank or from the \acp{srf} or \acp{grf}. -Each \ac{grf} consists of 16 256-bit registers, each with the \aca{hbm} prefetch size of 256 bits, where each entry can hold the data of a full memory burst. +The \ac{crf} acts as an instruction buffer, holding the 32 32-bit instructions to be executed by the processor when performing a memory access. +One program that is stored in the \ac{crf} is called a \textit{microkernel}. +As explained earlier, the operands of an instruction come either directly from the bank or from the \acp{srf} or \acp{grf}. +Each \ac{grf} consists of 16 registers, each with the \aca{hbm} prefetch size of 256 bits, where each entry can hold the data of a full memory burst. The \ac{grf} of a processing unit is divided into two halves (\ac{grf}-A and \ac{grf}-B), with 8 register entries allocated to each of the two banks. Finally, in the \acp{srf}, a 16-bit scalar value is replicated 16 times as it is fed into the 16-wide \ac{simd} \ac{fpu} as a constant summand or factor for an addition or multiplication. It is also divided into two halves (\ac{srf}-A and \ac{srf}-M) for addition and multiplication with 8 entries each. -This processing unit architecture is illustrated in Figure \ref{img:pcu}, along with the local bus interfaces to its even and odd banks, and the control unit that, among other things, decodes the instructions and keeps track of the program counter. +This processing unit architecture is illustrated in Figure \ref{img:pcu}, along with the local bus interfaces to its even and odd bank, and the control unit that decodes the instructions and keeps track of the program counter. \begin{figure} \centering @@ -198,10 +202,12 @@ The data layout of these three instruction groups is shown in Table \ref{tab:isa \end{table} For the control flow instructions, there is NOP, which does not perform any operation, JUMP, which performs a fixed iteration jump to an offset instruction, and EXIT, which restores the internal state of the processing unit. +It is important to note that the JUMP instruction is a zero-cycle instruction, i.e. it is executed together with the instruction that precedes it. The arithmetic instructions perform operations such as simple ADD and MUL, but also support \ac{mac} and \ac{mad} operations, which are key for accelerating \ac{dnn} applications. Finally, the MOV and FILL instructions are used to move data between the memory banks and the \ac{grf} and \ac{srf} register files. -The DST and SRC fields specify the operand type, i.e., the register file or bank affected by the operation. +The DST and SRC fields specify the operand type. +That is, the register file or bank affected by the operation. Depending on the source or destination operand types, the instruction encodes indices for the concrete element in the register files, which are denoted in the Table \ref{tab:isa} by \textit{\#} symbols. The special field \textit{R} for the data movement instruction type enables a \ac{relu} operation, i.e., clamping negative values to zero, while the data is moved to another location. Another special field \textit{A} enabled the \ac{aam}, which will be explained in more detail in Section \ref{sec:instruction_ordering}. @@ -222,11 +228,11 @@ Another special field \textit{A} enabled the \ac{aam}, which will be explained i Data & MOV & {move data\\from bank/register\\to register} & GRF, SRF & GRF, BANK & & \\ Data & FILL & {move data\\from bank/register\\to bank/register} & GRF, BANK & GRF, BANK & & \\ Arithmetic & ADD & addition & GRF & GRF, BANK, SRF & GRF, BANK, SRF & \\ - Arithmetic & MUL & multiplication & GRF & GRF, BANK & GRF, BANK, SRF & GRF, BANK, SRF \\ + Arithmetic & MUL & multiplication & GRF & GRF, BANK & GRF, BANK, SRF & \\ Arithmetic & MAC & multiply-accumulate & GRF-B & GRF, BANK & GRF, BANK, SRF & GRF, BANK, SRF \\ - Arithmetic & MAD & multiply-and-add & GRF & GRF, BANK & GRF, BANK, SRF & + Arithmetic & MAD & multiply-and-add & GRF & GRF, BANK & GRF, BANK, SRF & GRF, BANK, SRF \end{tblr}} - \caption[A list of the supported instructions with possible sources and destinations]{A list of the supported instructions with possible sources and destinations \cite{shin-haengkang2023}.} + \caption[A list of the supported instructions their possible sources and destinations]{A list of the supported instructions their possible sources and destinations \cite{shin-haengkang2023}.} \label{tab:instruction_set} \end{table} @@ -234,12 +240,60 @@ The Table \ref{tab:instruction_set} gives an overview of all available instructi It is to note, that some operations do require either a \ac{rd} or a \ac{wr} access to execute properly. For example, to write the resulting output vector from a \ac{grf} to the memory banks, the memory controller must issue a \ac{wr} command to write to the bank. Likewise, reading from the banks, requires a \ac{rd} command. -For the control types and arithmetic instructions without the bank as a source operand, an either a \ac{rd} or a \ac{wr} can be issued to execute the instruction. +For the control types and arithmetic instructions without the bank as a source operand, either a \ac{rd} or a \ac{wr} can be issued to execute the instruction. The rest of this thesis, it is assumed, that a \ac{rd} is issued for these instructions. \subsubsection{Instruction Ordering} \label{sec:instruction_ordering} +Since the execution of an instruction in the microkernel is initiated by a memory access, the host processor must execute \ac{ld} or \ac{st} store instructions in a sequence that perfectly matches the loaded \ac{pim} microkernel. +When an instruction has a bank as its specified source or destination, the addresses of these memory accesses specify the exact row and column where the data should be loaded from or stored to. +This means that the order of the respective memory accesses for such instructions must not be reordered, as it must match the corresponding instruction in the microkernel. +For example, as shown in Listing \ref{lst:reorder}, two consecutive \ac{mac} instructions with the memory bank as of the one operand source already specify the respective register index, but must wait for the actual memory access to get the row and column address of the bank access. + +\begin{listing} +\begin{verbatim} +MAC GRF_B #0, BANK, GRF_A #0 +MAC GRF_B #1, BANK, GRF_A #1 +\end{verbatim} + \caption[Exemplary sequence of \ac{mac} instructions in a microkernel]{Exemplary sequence of \ac{mac} instructions in a microkernel.} + \label{lst:reorder} +\end{listing} + +Unfortunately, the memory controller between the host processor and the \ac{pim} memory is allowed to reorder memory fetches as long as they do not introduce hazards. +This causes the register sources and destinations to be out of sync with the bank addresses. +One solution to this problem would be to introduce memory barriers between each \ac{ld} and \ac{st} instruction of the processor, to prevent any reordering, as only one memory transaction is handled by the controller at a time. +However, this comes at a significant performance cost and results in memory bandwidth being underutilized as the host processor has to wait for every memory access to complete. +Disabling memory controller reordering completely, on the other hand, interferes with non-\ac{pim} traffic and significantly reduces its performance. + +To solve this overhead, Samsung has implemented the \ac{aam} mode for arithmetic instructions. +In the \ac{aam} mode, the register indices of an instruction are ignored and decoded from the column and row address of the memory access itself, as demonstrated in Figure \ref{img:aam}. +With this method, the register indices and the bank address cannot get out of sync, as they are tightly coupled, even if the memory controller reorders the order of the accesses. + +\begin{figure} + \centering + \includegraphics[width=0.5\linewidth]{images/aam} + \caption[Exemplary calculation of the GRF-A and GRF-B index using the row and column address]{Exemplary calculation of the GRF-A and GRF-B index using the row and column address \cite{lee2021}.} + \label{img:aam} +\end{figure} + +As a side effect, this method also allows looping of an instruction in the microkernel, as otherwise the indices are always fixed and would therefore apply to the same register entry each time. +At the core of a \ac{gemv} microkernel is an iterative \ac{mac} instruction, followed by a JUMP instruction that executes the \ac{mac} operation a total of eight times, as shown in Listing \ref{lst:gemv}. + +\begin{listing} +\begin{verbatim} +MAC(AAM) GRF_B, BANK, GRF_A +JUMP -1, 7 +\end{verbatim} + \caption[The core of a \ac{gemv} microkernel]{The core of a \ac{gemv} microkernel.} + \label{lst:gemv} +\end{listing} + +Since the column address of the memory access is incremented after each iteration, all entries of the GRF-A register file, where the input vector is stored, are used to multiply it with the matrix weights loaded on the fly from the memory banks. +To achieve this particular operation, where the addresses can be used to calculate the register indices, the memory layout of the weight matrix has to follow a special pattern. +This memory layout is explained in detail in the following section. + \subsubsection{Memory Layout} +\label{sec:memory_layout} \subsubsection{Performance and Power Efficiency Effects} diff --git a/src/images/aam.pdf b/src/images/aam.pdf new file mode 100644 index 0000000000000000000000000000000000000000..a2045d35e475eea0d7a973315917873d75037438 GIT binary patch literal 24413 zcmd6Pby!tF_bw?Z-4chA?(Ptf?v(Cs5Rj7Yl#mqZ?oR3M22r{j0qF}U>i6mW-TTkg z%QFvWpE>(Evu574*1Yr1oJ}k(AVft+%>+$cu~T>f%?O|cSn8NSb8-S`#Pux+Mqk?;+A+~*sN}I1FM`1EW$x4? zgerz9-^53xz-(6Dl)VAHr%?W9FN{6M!!(4;bgbj%Oe zEC9NPHtARax8?giYyi54cIjyWx8++sJsse-e7~j#&_A?G&j7eB-|HCx^bhUQGXZYP z_j+ak{X@I-EP&hcy`B|7|IjWy8{oElt7o7E+?MaxbO45jb{Xgax8-|11AyV7T?R(L zZTVi$1Yme*mw_2@TfWz`02sctdprKO_ci?dmW}|`+poU`?`|L&0P6Skvii1`_BOiuwg86v{{BYcy zzve}FcEKhnNt+OHTt@VZ8Ud-|?YE5V+uKPt=-glPZ0{2hyLct!3*86#aMt zi>(Zz1z)wuzP_1Ee%$oA(Aa`VE}@s#DtL=$QgBLV^y*X8&2eF$%g+RCg{X>C(WnIZ z`s1r>$jFbd*1^AUU_|`^pu0Z*0EqFyd+#m$w*dWgksk@4@je#+20-@-;6DJO`Xzvh z4KDL{p}?B?soMEbz=ruL>l+Cvj1Qe^h^8%c>U_SAO$x8RRe#~+ixegiLi!SuWC$^j zba;_otpHL%de#4Cg^^wE`K!dCmKB3*7cE|rr21?K^gqlH_x=3=4($(c?yUQ_;M~s| z-%R+4&V5Gw8|d7p;r~E~DTogGH7x?zg>!r3ybXG)hz0->&4dgs72uB+*|PTps&ssN z`;*=Y(G)UNBkbco*|+A|#IHD2D$-ev;_)e`gu!ukIVaWH5EQU`+#9-VQ5T{L;Hy(m zIda^&=~A5cKuz<&(TU5fc{fbOTnzX8zweE44g-OfMX^BZ6~ zEkMJvlmhsG7ydqzEClql`>VE*66l2LZI0tC#YdeTKJnvA0}5iEAstX=7!*)x zt#NCSppVJb7B6TZoaD&JyEW!t+X_`&J9WMmj=(+E#z_4`dV5IoxAOws&-sV(=j?nx z9RFbcx#w5^1rXEUA_$CsA_%`kP(3p88+1rd!SY*z0O^m`{18D;a~G-nS)dw&adInY zCkFy85A??0ET%Yl6l;;7!U{VT*6a`DF_PZwq{f8!DZHGR_r$VjyLQpxPvNCG2FFPJ z9iaOz{~>~YX8(T%(9bOUBgHZOEr!7KCx-AVKyTWJXb`}HjP6q$8IjLt7J|DJry;+i zQf!D+kNHBrw^Qvx$g@>l{&}ERCT5Fgi2TqJxh<98K41I%MNt6uERa&!t01rnv-GpG zHviiQ`tk-N<_`e<2SNCe;_eOn$0_cP4g7Zm{Vj&T^gBZ!4&sA;4L}h%y+N@Bxf0Dx z20*@0?lnRL=U?Vmv%5lx8&7b0Bhu00?}I_L1?Kb;Z6;)!s*$}veSgU9HTcrgcIEKA zLX(9gCC-$aA^}BZ^91EyZ%Ij*rNECqA0=$6$OzEiZ(END)%e7E`wjyYfHvC5W@53X!*cWq==Y*41|z zY+fiB74{68c$|+x0;S7eSUX1W74Y%>RhVBJ`I1{n3_O-onptsm2|zX-emW9E_7C{{ z2Sxaa&(Fd6X9@1XvR{^{On-|aF#S#usDgi?2yu@O-XMRUfo_+mA`8bMq;d^s>N-^d zW+MJduAm(5W@_s6^3}FufjY6)AdgD|k#GvtCe6IrNx2%YY#N^nHMBxe1sk3%c>v3# zayq)B+x`KZU+4iF+mG1!|9~C*G~q|=F#j!f!1OyipjaM}qG6U6rdg$z25JU4fL{qf zY7Ta_^nDmndyh@J(ypH!-j0fsr zfcbZFK>W-42T=1y;0Jp{5D3y*IBh_voSB{$KXJia6AO;a@tZSYkuYL=`PB|f*lM`c z^YIcuJGd)1$P**A_}SH77%p)&0lAB`s!?zlE{g$?=oipw3=GLX;PW5c;74w|x9=a% zJwILKXZ-vvZovFIH@H_19^{=emEfnh^3DT!yOVc##E)0|vGpe8ij#>PN5*MKJM$Z( zq!)T}@qAL))|kAF{{f)? zAO}Cl+r5GR7NC3i_nQen0s33yfcbZFp#O$1RF@7?;FQ}(Y91pP7%wf1HgIf1u=MT5 zg_ydQ1CsH}dU$5RxSJ|9i_Z<88pSMwVi6`kwMCl~q_i^@T-S~FzM~|t7|<|G=-o=g zU1k>EYKUaqUBbzl;(J{HcJPAN^byp!X~wianXnR&RGTm%Hi3W01oa!lxsRmH8Dqn> zQ}{qI?|KBRrPiT`qC1r_=ig{1W!NtwVJQI}qn7YP0KFc4Dozp7_){!F)_SI{8iBy?VK4I zYe}-@G~z5XJ^USjN!}^}f1tNEq39(eQ`Rvek{@z(%qHCx+@jxfI+4E8L#{5dw)CEE z#vtRH>|1%u@L>^Ohfs%+b`|o27_6O06&2FLnwx*U03*!-u6=8#$CSU3lPj@^D_)TP zflEm^!$@Mj93^NL2Y#>4isED`YA1i)yC*&;AgiAHQQmR7G~D})=1UD|Q_FO1M@5 z9usy5sxH`P{s-TGpf|UZ2P4xD@$4S^f9(5z&}Nwb7V%>Kop_nQk+2}5gY>!ceco9N zGnZg&d~)MgXx#+eR~eiiGA4w}@}w@gc(RKQ9k0vJFUdmm%y!F|<9qe&T&&z&c(#_f zBIQLw@`k1qdGaI|8$F;et+w+&6yy=u4{Aie^=rif!BE{&NRwi)dMX4hk)+^bwgVcQ z=HIB(|7@ma699_x7&x|!pG?TXcAqVh0qN0Bf81LP*D@37C7Io1aSGKA)~5%dL%=z( z&o7-Fy7zj=;n%w%?cf-7u+WyJwBxreec9eF&XM@ZjQ}sm*Rt+r&HHTnKW$eBPpk18 z=WFvaSLriQ0y#|!O)!N8jgCLJ(&fmEd3I2WIVv^8VZ-j%UrD>nIuz}CD1J(Z9BNEM zGn3j$0m!8r>uu9c#rUvFq9Yv-gYy;{3nM;d)Mm#P$3Lt5y@Pm8!e`Oe+|aq2HQ_HW z7lSCORwT+)m&DuPsMQV zyg_bwl_*Id^hA-tB>rM!LySs2bg`M_aq%T-5OmnrQ&b)|6ND^LhC)X4&cUvn$r8jYHd9JA zeX?OWgGVVs2S~yVprwuDG9W{XCZsWuj*KDl!Xq_~SQ(!JW7ErfqCU(OW{#x4MMM+; zfCQdAv1Y3Zd4a_b!B!4NYnsIt(^etx{CGy3yh5N=Isa-F3X6U>l0lw$E#Q?LlWd3* zFU_08l@NPU?ks0drOO5qat{eR?ewwQXOj9D;lPlj?DLgf-UlPA_Qqa?3_IlJ{f}s; zNQ-iDXxV@>_~0J7yf&S+I!0nF(1sMi=)yt;+N!Apcqp$M5WMN#rUbW|v^KRoot}biK0hI+$SNRq&Cy(ok6?rN`x(*cJ6uE#*e|q z@@XaL9|8^%(>@g)M!lWzp#9;jNQJV-a(5{K4G74;gXuFdSjQd!sxrnCfD8CaE zAk(%bp{cnFk^oa~Nf!wO{eq*LB`Hio`jn>2g@6Crf4LAuOS8R+`FSH_=PGKliNS); zIF%>3spc_EmR@1;My)mi+SqQ@$1r^`lSMT6QG}0K&X1T3fhMi&rf}U6sQ4Hpu~W8P-fu2lGtW}^GVJIAJZtbE5XUA;5CF5)>SUr}_7=(<_6~?CeN`clPr124 zw}y?Wb?Rz26owy}l-r_X|dt~W8S)6e9BVnOEl+&Vv_10!rq z<)W(P?M5Z4nlQzz5)dU~2hx{#dK^<=lTK9&y8*i;*UWe*2N@w#?n}WsTrX=OgM)$7 z5)Os5gvZafR4$fC3%}mPT|)D;@aQP``23z3?qlsP1>9x^76#_K8}8oSN5((S41ds4 zS^m~?iRJg@l6s6-=n6Be;F^Yy(tHeQWMoVPF(YnxPl{>ymuruG8j`eJT@@tvQ|=qr ztFPyCyG5UsoUP2(U8?-nT83Bf_T8_~YTYl(V;^zBOg6l$*tl+l;3;hux$b=AU@5hg zB4A4f+;Sz8Y)bJaU&FKZxz%X(o7!zY_i2peMrU&$zmm!3{q4rxExFUstcy|sA|Rv) zh^>7YA`HXFQySaF(KBsgQtGx_bzk$Dwg>2<2|ODeWCVla;#WMeF7)FnGe}Mif@~_< zfqD-u8il<|-S$S@cAH8k4yi>mKcBZG27uJ>sEy@!FUrp#Y7seEB*-r@dojQLsf0c{Z%- zP<&P$MPwY8A_xYuWJKu58I%m*>BZvm_%U;K# ztnjtQ$qJOyJ_{5;oUl|Bl|$0!yF-Un$AG)a1H#*kblKtnY63x5En94Xy1$!g=xZ zidY|EVsaW%Vgl7ATN2fEs!Z|VT4 z26XNd=?Q`P6@+9qz?do8%m%_x1$9?OhZvDl=2i=Y*_c18Q>=`fP3pT40#AwGhx~jH zyWA?9E3!i6N$J22{dM2;v18BR6KqRF+@xMrFhs!d+o%SymzLcHyHrkAPGV|X6G*@{ z?gGUdRGuNP3OCHczMwI8*VhZ;eyz`~fD`~?hZaGl)HjF;e1#TOVP!{KFKBi4I=f1l z_d|K=5INC4(=cVk zPZZ>J@UHlZea5v(32iH%EI53qLxH49Jf~J}UN%Y>TR+xM3~FQBZKdfN1W#+csgR@C zsdXB&4g9lcAx)jPZ+=y)cc)6jQGfanl6 z({~9nj2{8vHLA|&-)~Ga$^&N8L+G_cb0f7exD~dgi6&@3^M?%kz;NybDdr#K>psQ& z;}rX_)c=)y{Vm0V<#)wHKbp@52On18)HaZ<83PJo9fOU(N5L9x;C4}wmuk`Ms8h5J z5o}aJbiKE*TB!?|*%()N=E&yzPL?NPt&hV^ z9X}OOUyhh%HqkOW`l85pkX2#usMp9pajJ|FHy9v<$!R!vT^fo+=Y@j#CC|H}tKti> zXtk;UUn{-+Rwtp?2ZgSUgowJ~N+bhsCb6`WF9<#vts%auTfDYS-DfLXUaFVV@7PW3 zJ413OYigk;g_QiyW1_f9!D$w!Wjb^n6f4Ue4c4YjZBEO>d;35Xe90+!3$Upy8!3yN z$F$A{pFNgn^)SNVR^=LFH#9F6;b9feYh*Jzn|ay?J&IMrg2t+bd=efy!hHCTU;p}3bXlh~^-4u2<2Wf&hCWMe-1brc@kgQ>P zVW(I8c3C$QhqUd!LSif{s>mcU{&TKm4d0eD;_3Z(DF{-Cs4z-RJS?vBD->%C=7pRF z*A|eZqbS=!wL>mFg_GGUo;}F;CZf(Y@UN(@eek(cN}VjEn=!LIaI^cs`NxUl=T6iQC6489DRC^nD{(kI4T!G+u#bkgW+!d!gro^DudkncbW7HkiWNqVRJF{?%sO{V~>Wj_?zOzoiMW{H_V<$4EpHlfZhP zCZ`3VWh_^ZiJg(H^ZFR$uzKN;vAp*v25RLluN=h%m5M+nt&&(E;U z90F}f%P#dWvreYBt3A04?g!b#8G0^~A_LC4>R-cfF=M?(Ktz=ydlA&;@U>pqber)B zX$y`Mh*=@RD{wfO4nj2+L4ubpDDUEEXSh6_3`0U(0_0?pkwN-ZJK73g8Y<1Gv@%X&Lkg8tV>I@X|^V!g6i-gW62Y4H7q zjE|@xP;?}8FWD$o$#GOW*bLV}P(MxvansDLs@po*A=i(Ecbm8_t?T3ShhuDYq_yz# z!ZR#B{v1{P3L7ptUhRdCuF8aNj+?<$k-bHR1zqO?x68U(r%sT#GzmNcO`dNzek);9 zlZPltkUN|qPSg_*4?C{ikDg>G+zomjqWyY=_D3*q#~kv6N#ZMeC1ao24BU6ddmEf9 z5>mTq%G?)XCo8^Yd@*dWr9SCXhQ+*G>#qip&J=b--WYLdC9|Caqy5x9CXx!_cf%Si z!rfy^^x?~m`#eGo1mQG8Cp<2lES&gZHfoGV z)F?v>F7;Z>6+Wv-HgZ6-lL$x_htCUkR@8AyLEnIy0-;AXRUYgWThVW*I-?bWYJ}zF z6^r%}^MFDt%)qGcvMyNkE0 zqvM{=td#pIcVWwjdmxTEiJn#w+FU?sz5tUlcfr`PL`Cf*#0Rq{{v z9VB*v#$O4_+~A=}p_W9s9r5bwHu@Wz8oIfN(^7XHEhN{@*+`prrGrz;8IGBnVB(5bqpJQ zs_cD&b1?OBOA6Oe-hoiRzo;LPeK9YomRxRxyi(E#AWajI1z!MWJtap{l{>tZ(xSa| zz?{CDE?Jj)l+)Q0LiM?l>sck6W-G(F@>8m3VXs}|9D~Vpl#pgw=E1o<2pR_-jXv|& z0Q!pBHND=zht`ne=lRvFx0t{aU_~veuIuJ&;^Qsph`3g{t~p_paG>VfB!Jy-#|1pF zV?7A2k|jd~Mkh?uX$D_mg#Iyvr^wGq)#ye^FsPwtnG)c zdKUz)>#%I<31Qx3LCb-c6N-T=)qAtTjG3h)a1a2`dr|{RdWCs&1?ir_GvZEtH6ZW@ z8t{vFOUJ-O_k#!g-0%NqGtf^He#}6ue@ne(`Ga~JEn$Oww=2Oe_%5XYkN4*S;@~Q; z#aVcX%$Y?K27ey-sH(o+b!=-VZOl2lCM55ZVit_s*P+Jc^V63Nm+29kT*oF`6Wm2S zlR{0g1zT4mPAnIJ4WC>G+OVoTGel~}EcW2WhIvhJ^mNsV=4DVh{a$Y>6KT5Ghf08V zyz1T0T6!8X_C>bFHNp#uLTP#y}u7=Nah4EJaJG~Nc$O>iq{xeEdWY@$Q zsts}t-S&`51LIoqIsQlfX;G!Vu+oKZe9f8K=rNmmIBFb{9da2S<>0#oyecPV7<6PVrT&bq*K>F2Y|k}=XZ zYUo$D`Ay;wyU22oTuY0V0G)oMTr$SYUSHj)YKZlS<3 zO#1Y8;RAV{fQ(&`Q7kr-7eRAmi20LgG`Ug7GC_Rn@hQmA%m9*jTa~X%SDr4`#0Xn} zY0emx(+m*g46 zLin~cpfca;@M7Wl&@iJLE_ot;b?E4=vMKDyd!~u#3M;3&uYB`X)7NV3ggcPKlN% zmFgu!oc(}(GZ%`s!FI}|jq3yyo^;JRr2jESRe#t)DQ-fh>)KK?R; zfe8Ezwv&Y3v;FklNeuUBycM2w7O#chbI5&A{@}1bN$k%f5C1HMJY=_DGziweCG)cW zF7x8_)S-R=K!Oh?lYIj7_XO+cf+R!QTEAXf9-UkK^ld3I!=;A5UFzcsxkL5XHU-=L zu)~$1yeHD5$N1QsJiuu&H3@f+22tz}F#Mtg{ai}iTk($s=7bo%_8s74p@FYoIBw|pcGC)i^$(gn2NqRU z6sj?aTNvfxECDHsiAY0hCgAZR$Ler%8PgWr*?HpT{Ge&Ib0EdXCg_PE4LoK>?(9C>)-r!Ss))z&&2aBX*vrmFE;P+=3 zFHPRhN59L!C#kfp1wbW~1O(d_#ELujj;s=wqE`)raKQV9P=UsHxl%aQQcVtJZ7sE0 zM_B2H*&p(AV2m~=1A(GGb=ts~BydpjV+9sF`S6Hy) ztQAzMGL&O+ zcEMK7Q0nIuz>P25B1>a_u$t>YE-J=HA)EeeoawU6u$1W!%a^fAR2 zGLC}yBA=)vlF8EmdXCDfgKa7n`DER$VwdJyCxyaX7td|!8POl*ZN?X8na8cq^7+cI zkJn6vM~bb%@}H6!KGsu;d=`3b+~8XR4_6d8Ud<3$5>A(<>5XW;kPBMq63?dM=^(J0 zA@T_nCoo!VK%ne(f<;t#Qs@?nnBHTme$>ps8r>*kX9xdAR4wQ++H_}iPOSwnA*JAe z03H}w^(ReJiAP^O_pv}V@)INw3e`T_mfk!rhhFxw!@u>P)i+4zeyS_$zLh;wX+7hH zp|1(xwZYhJR6wv$#Jbkcudst(#9d{V3^0QGh1tI zEmu5HnD~uepJP1l=h)6(O?gkxTx%SpIOod;?VI|xT24`}McgxPfR&Nqcm9wJf04J? zSbrS0xz7gwHXGjm-uJ(R71qBcZ?XO^Z~bzTRvxH1)$aBrE$!zP`N63%`DTH^kq_d{ zQv!oyANZSxlm~}e1m2S>;D=Cz!vJM=KNkGMxc_p2l@v#5lj1%kM(z959 z*RvEt_~2S;A;I?C+8bZLHurZDdN(9C_w1GMp?eN-Nv?p}+i}W#<4umVy#T~Ad+|Z9 z1X(mc#vX7y*nGwWZ4bT1ovg}voQ6#{kdrbcd3=T%U%=v6{zp*L)*D@7LosyC0MqrQ zPvA?l?{D^_aW`O$@hCL^0M`RSy4ydZW4*iK4yJp1|1n+Xf2@C4{YCMG zsC05;z58H)8;n1f3(kUh^so+MiBMGhUm@AgKs$zT28zEBeIku6{L9 z;cB{dy1R;uS7_R1cH>rQ`5ANBZLdQ0i=zPk(9L|xssf!0_f?RFVge=NuJsBt3N?sOqi)GMA$HZ

%K z7BF@1kUe6v2s&xqLtjKf*X})f4RzpQ>v`FnYJZ7NdRKiJBgJ4onVEqJ*Y_`RF)=E( zGrEt4_nikP`!}xUMJ4q_V-4V*hBc;O!j3Hz%%Dd=Bdot`hxHP&eTNLO{mhwY$~!qP zSBQ?RPQiDmA<6|?q)~_MkrFO?lpstNUNs{9k;IW_Xcu8du~_(Telj4XoO*hDO+|;oEc3vCla}{$GXZ|gI;a6Zm}{}9_S0eF-;*w z=a*BcQl$z&wss`J69h)O;=+y9Eq@Rz;Lm--J^M;Ze8E&;!b#!g!wk+F6@(&|EP7RP ztkF}5OqnH|3l3NNozwd3#_QT^(4Kh<{@-)Oee~TWg}bc^rXO3)_p$PibHyJ{^RWG` zWggrAX_=S$T2~J&sExh$hV9I*s+uLE2Hf^dF*A%DWP;-u{QH70flLt{?_Moob+zn< z_Ngza$HPnn1i`%mYMe*J`vVLQiSa*njP9-Y$0$5du3t0u-&*Fe{l3gIR}zcFVT1fQ zN%jd!@5)2B4-uP2XC~-Gr{gpXqbn+ROtodO>Jrvwk)yi-Bq^s1NY8BK+q-ko>Gy74~z0=?`n?D#F!Lxpmg5eko zVOmiPMZW3eNGTo~&Gz$%8T+C*HI>B7QT%kvOSnBEdkw~_V`t%WLN!NX0kFT1l`sj2 zk*e(g?QF&9&<844wA5N4`?;#;6VTp@9wv|7J1~k^9!I0$8N7!Xi0AD{bMk&~74*_p zInW=-=Cr!Z*_o^OxFG%_A|pP2%JQp-vjnsE*Ym^o(~`CU^wBRAUt`q|bI5^FWYPO$ z&XZe(<{rz_?}mK!hzpB^K(JMKYCj9BvUBvJr`e(u#fYF^A3tgYhHpILSklzcRqsQZ zPkv6`qgF6uM^a`vydB|*5QcbKZbqBZR7=C$$1!rxFh*NFs-e=by9Kn0$x>d*I%1_n zl`L=svdQi($!E(Od4*4Su?8!s_Q9wIgri;2mZ?MJ2OYFYrpoT6C0Ec_X$vBJsWq?y z@omYnnf&NlXoKQ9KN%ybhE5W_(cAo5Q4!~CAr7nWh9R>>4Y`jjC%dboVHlYw6f%H- zo2(K$;H1I`=@k{O*O~#bYyqXfbDJ_%cCW<*i19F5rG#V7p*A-gaEkh^ z@$3$}nVrFD(4|ObP2_9lroFrdxaHcPZ5)TQSI{8g%CrusMPaN5YSn+5CR}P-jk#3P z8`M3rBABHUM5upv4OWN@CI=3^2!0bDh+~*Lp{xN%St*r*4>6tQj92)4r+`!TAT@HM zK;uJ$R`8A!ejas79K%ylp5?QhGMx>4H^G;e;gB}tj!@5ij;uE%#}f)X`%B{TA+Hu{ ze7z>|6@z6$Y4AD%AEg#Lr!9RVsYH7>F0u?kbE8|E=pRaCAZ7G{&kq&#O(k}s7g@@4 zCr7WD7mhug7?mMHxag@CMMvSG2^ZJSk(YJ8e8f$9Lq>XCT?=ohSlp$f`V5G_jFpRc zlhKEqwh4q%T7(I;$M?S)Za&OckZSB62JSJKZuz)^tDYP+JW=1I$X&vzGs=))m9rHx zPXoGNzNQNdRz)WtxckUx$1&X-`^{lEtS`$;c6MC+bi3P ze;w4JW*kZkyHJC}%kQ6KmBxf>v8}Cfk?u#c!<+}X?)n*vA$cSel_J`o2+mHXCIy@{ zIj9;7EEszMSWmBh$#+e**oQ4RXwFldb2PB3tAzx`1ZH&vbh1yCpkAgVbe>2^L7lqu zE#O5CFN|f*0;z_!&rR}tQs0R72UGs1C`+nrmT0f&>ko#yZK)s^3Y>o7rneOz(QR9j%4%yhrx!0Hd8erx9pjK{uoHsYJsl8ud*_}{{geEVxI9Ny4fU=kOFL!9a#P0Z!-LvFoJ)Qqk!+#vmYuy6 z!F>ZccCc)dZ#z2=+I2S%+}^%xoRCV+TBl?d8Izw)L|<^`73ru#3k}kB%R%@eWs8)@7BfpjQ}9j{-#d zLz{ada8J5!l2CQ@XAG$(?uROiB(@kLfOce3kfryvD9~@@4 zH}riNl!TR9MU7&U^dt0We-t^r3;Q*a^H3H2k)4n?iV{DhDlIDYV;PkJQT}~-cB?!{ zq>nC~C6S+%GU81ylBTT-bUu_Z1THHH7)5Y=BzS95@rAp>qhoocAlE}MdrTkJ(U zrqPSdq2Mh8=J_Q9@fXn7Sx!9NrUbri`|8-^`fxCKG6t`7_|;h;R`NttntAc(MX5m3 zXx^ATO`%DZO4Z;~3`vatFjlu~(t>CcJVL=U+$QZ(;uE<+{#=a~E40d1DO>;D9tKfc zsdqRnS!w(VPIu*~(ww`~EAY*_536dX*C3e7n%qgf(|3Is>W)_s#3tz7YO{|2Kif$NrsFuOiSK|F*gN^;X8Fh|s2zH#T zm9C&dvsBcxbxwh43X6Fsb0d~ax-YkO3PDy73r6FueUqkb@>ODJ?O17zPEqSXMw&_i zvR`PPO}Aha`FeG>E0hWz0DE$#?zzPgmktAS)#Y;UvnfN?oeKzmmMsvro8@eM1>e1< z#O0Unj^w*l817Y?J3P{<-e!Nu8owwvEX+TCnt2~I|2SzpSoSNi{9Dou+wam1PEQHq zYufu=oNF*Jet%kDLS-Y9H&=UoKM#cUmzww$3&|gaXoRf>vPl>6WR$f7@2IuDPn2+A z5b1jYpY#Q|KMYa$4>!68HDs3Vhaoc{q^cq3_|)4VzwpLrQ(YZWYTXgRi1~ z-Ns}K4(+6cOF(11M<87iizVEXdd_NVjgHEksMuanrVJPl6>AmSnd`8skqs0Bdmn@p z%B#C>{{cLjrLw9EaHz$Wk0S~mo2EEoNfA|EwUO#tnBOW*ZI?(jG$dlcI`^rp#ojylMV%yqRyzc8r!bwm9+-vZVG)2}gwPZh6Bje4UI@srC&FCn zi}MRQlhsZ`nCcx?(^$f#ZsU05nG3({J?}eu`X2i_Bp9kYI}+fy%ggjKt4kdNsR4tw zZV{`S1~qUi)kynYx%@}`bP1x({{8CwS12Uq+!ymFm`LXRU-f%lr*=x)AdzOdSqD2}McY{VF5#y-cWqkXI;V_1UqVj3M% zpO!Qtn)q7QEj_8VyRd@xx-dog35i#v_DgV-32{Ft?NM2O%jl;BymSb3G_#VY__Jvu z!hVZe`}2e@gJ5bJZ>dc+Xf`vYigj5l8J@6DgkIFkIjqJoWP8R+q=R};NUuUuaxk1s9>zEv;O0K1hO7g~-Vdt%Mik!$I{JmFh!K-_(@ z(HKBX-b)3*YL>k2J}J&4g!CDP1L<+lt65WO#+9u<;J%9tu z=U)WTfDVp1V%Yyh;)HHT#B%cS(o4ge^tV~;I`381&U$cQLUbEyYTIKD1*Ie&b*FzKv}tnOLk;*?xYb|IY({%D{reXv@_b;8JA8vEu`9M zV_9G**$!+=|0({L#H$u50fK2sv~H%=Gd=6+eRKnQ@mZLu3jKWDkpr|g9Sf$-PF2Ch zxSWsHXtf7Vtv+==$DO8k$nr2JuP~I&H-DmK|m>t*^Ew{wgB_eCFQ|wSz@%5gxwV zbr{0IQa>zgA~e$MBi|F8x4WS;fQzX5fIKnHl#oUcpf(B3d)+_9vq{{cS_3`a)o~o?v{8 zno3*5wztS0}TRdiP&6vN>^xr1WyGi|S zp~gb@W4r8ruKUN6{R0cR<6Pe!0xYFtB4_vQiND{UcTGq8P5ZnxpGFcucYn%9$k@i# z4#4>R`hR{3HiI1PeBf%}3JB06R$%o%4ba1;-Q58F`28py?SlmRKO6URT`37*`o%Z~ z`u{8I6lBy*TWC>S_lj6)bbC@HMmFW!AW#KPJhClq;Z+`2^f3{ObY7m;8Jn*`5+%NI zw%KD_&;mB3dvq;KyXx^;*ca@Puy2|cyI^@!6!fr=LC+(g)T~RRgGa!=RG&;u-siP= zK$NKzI2JT!w|u5`tbbOXKDU4<4~(P^!y<&2Cy)6>Htxw|+(VhJAr3aB84;9mGXp)R z9=RtbZ;me{c+%2GiB!?wF-J4v!7oYA50M>h`Jm%iDAZ8LjF5eaL2edk0bBF59_WA% z{>(68`TW8*gu`a__=#~;)ker~WIWSt3NRL>PRGBPHuGG(b=eB~T>NY33c+3jYR7f{u=*O%2tbl`Fz5?5!a$r!v| zv%pW{+}bt?y*}+o=o~zJmK31Z@0@-G9>W?Y{U>lAX#Fj?3=B+ncij0r9qq3?^ZWCs zW%Uj2G5hX#4D>AjTL|x*QO@|4{_W#6S?_OB0;t|T(HlVj{h`#BmUg!veUk$2?iH~x zu)MAKCRE=(j`_zk+KGpyNL#hBI}Z%qFL9W$bSp}|3Zxk!VwmMBGoCVw#1Us864%J| zNbE`mqM*0?*rW(Y1SM7rFMklkW;5pAQ`5LQ=aQ0oH+X6qOoo#G)Dm`zaZ($8jS@Oyvf+X&N+aPy6KXT_Fg@Xm zCV@`RbxyQ}V*jUtgEi|(=m!!JW+QvEfrdZ5-l_@e3{ej=Qa$wvC;m%`Kqx3~Y3C(8R=oQbN%G E3u&|qg8%>k literal 0 HcmV?d00001 diff --git a/src/index.tex b/src/index.tex index 05a57e6..13ea58a 100644 --- a/src/index.tex +++ b/src/index.tex @@ -106,16 +106,16 @@ % %List of Listings % %\renewcommand{\lstlistlistingname}{Verzeichnis der Quellcodes} -% \begingroup -% \phantomsection -% \addcontentsline{toc}{section}{List of Listings} -% \setlength{\itemsep}{20pt} -% \setlength{\parskip}{10pt} -% \renewcommand{\listlistingname}{List of Listings} -% \listoflistings -% \endgroup -% \newpage -% \clearpage +\begingroup + \phantomsection + \addcontentsline{toc}{section}{List of Listings} + \setlength{\itemsep}{20pt} + \setlength{\parskip}{10pt} + \renewcommand{\listlistingname}{List of Listings} + \listoflistings +\endgroup +\newpage +\clearpage % List of Abbreviations % TODO näher beieinander und serifen für abkürzungen