%! TEX root = ../thesis.tex
\chapter{Experiments}
Now that the theoretical model is complete, experiments can be done to start checking that model and get results to use for in system components.

\section{Experimental Setup}
But before diving into the measurements ashort tour of both experimental setups.
THe first setup was used during the calibration phase, while the secon setup was used for creating the regulation model.

\subsection{Part 1}

    To calibrate a Powerit a setup is required, that can sweep the input voltage, as well as draw different current from the PowerIt (see \autoref{fig:expsetup1}).
    For that a setup with a bench power supply an electronic load and an external voltmeter are used.
    Additionally a STM32-Discovery board and a RaspberryPi microcomputer were connected to flash new firmware onto the PowerIt.
    \begin{figure}[H]
        \centering
        \includegraphics[width=.35\columnwidth]{./pics/setup_ps_el_mm.jpg}
        \includegraphics[width=.45\columnwidth]{./pics/poweritv1_test_setup_corr.jpg}
        \caption{%
            Photographs of the first experimental setup.
            On the left side visible are a Keithley K2100 voltmeter (top), a PS9080 bench power supply (middle) and a EL9000 electronic load (bottom).
            On the right side visible are a PowerIt with connected STM32-Discovery board (left), and Raspberry PI (top).
            Also in the picture is  the power supply connection (cables at top of PowerIt).%
        }%
        \label{fig:expsetup1}
    \end{figure}

    To now calibrate the board the bench supply could be controlled, to sweep through a voltage range, or in a similar fashio the electronic load could sweep through different current draw scenarios.
    This setup includes cable connections to voltmeter, power supply and load as to be able to control them with a piece of software.

\subsection{Part 2}

    To obtain the required measurements for creating a regulation model the second setup was used (\autoref{fig:expsetup2}).

    \begin{figure}[H]
        \centering
        \includegraphics[height=.55\columnwidth]{tikz/setup2.pdf}
        \hspace{1.5cm}
        \includegraphics[height=.53\columnwidth]{pics/setup_pw_back_2.png}
        \caption{%
            Photographs of the second experimental setup.
            In this setup the Wafer system assembly was used.
            This module has a height and length of 50cm and a width of 15cm.
            The left side shows the back side of the assembly.
            Here are the PowerIt (1), CURE (3) and AnaB (2) boards mounted, as well as a RaspberryPi (4) and a STM32-Discovery (5).
            The right side shows the empty front side of the MainPCB and the wafer heatsink.%
        }%
        \label{fig:expsetup2}
    \end{figure}
    
    This setup is similar to a BrainScaleS wafer module as it exists inside the system.
    But in contrast to these systems there are no FPGAs, AuxPwr or FCP boards (reference~\cite{waferembedding},fig 2.2)
    The MainPCB has the PowerWafer embedded and is also connected to 8 CURE boards, 2 AnaBs and a PowerIt.

\section{Characterization}

The first experiments to run are the characterization of hardware behavior. These will then result in a PowerIt Calibration, which later then can be used as basis for creating a regulation method.

\subsection{Sampling Time}
    First up was selecting an optimal number of cycles for which the adc will probe a to it connected pin, like described in \autoref{sec:adc}.

    In this case the uncalibrated measurement of input voltage was taken as example, and repeated with each of the possible 8 settings.

    To be able to compare a reference voltage measurement was taken with an external Voltmeter\footnote{Keithley K2100}.
    The resulting errors, from a set Voltage, can be seen in figures \ref{sampleticks1}

    \begin{figure}
        \centering
        \hspace*{-.175\columnwidth}
        \includegraphics[width=1.3\columnwidth]{./data/m04_cycledepends/cycledepends_20180529.pdf}
        \caption{up: input difference from set voltage vs set voltage for different possible scaler values; down: gain error of the linear fitted curves vs set scaler value (May 29th 2018, $\approx$32\si\degree C)}%
        \label{sampleticks1}
    \end{figure}

    \autoref{sampleticks1} contains the absolute error of the measured voltage compared to the theoretical, set input voltages. therefore the reference measurements (yellow), taken with an external Voltmeter, are not at 0.
    Also shown are the calculated gain errors, in case of all 8 settings.

    Important to note is the relative error in only the 0th case, here the \verb|cycleTime|-Setting was set to 0 and therefore the smallest available sampling time of 3 Ticks. This excludes 0 a possible value to use. All other measurements are within error margin of each other, and because a smaller time frame is preferred, the best value to use is 1, resulting in a measure time of 15 Ticks. 

    % TODO: move to appendix
    %\begin{figure}[H]
    %    \centering
    %    \hspace*{-.175\columnwidth}
    %    \includegraphics[width=1.3\columnwidth]{./data/m04_cycledepends/cycledepends_20180530.pdf}
    %    \caption{plotted difference from set input voltage, and fitted linearly, May 30th 2018, $\approx$25\si\degree C}
    %    \label{sampleticks2}
    %\end{figure}

\subsection{Voltages}

    Now that a sample time is chosen, it is possible to proceed with the voltage calibration measurements.
    Note, that measurements are expected to be less accurate, the more components are contained in their respective measurement circuit. Because small errors will accumulate and in e.g. the case of 48V's be amplified by a factor of 8.

    \subsubsection{48V Input}

        \begin{figure}[H]
            \centering
            \vspace{-1cm}
            \hspace*{-.16\columnwidth}
            \includegraphics[width=1.3\columnwidth]{../pitstop/20180821/calib_v48.pdf}
            \caption{Calibration of input voltage, plotted are a external measurement and internal values, vs the recalculated pin voltage based on the internal value and used default function (default coefficients see \autoref{pitdb-example})}%
            \label{v48_precalib}
        \end{figure}

        When looking at calibrating the input voltage (\autoref{v48_precalib}), we can clearly see a relatively constant offset of $\approx$1V.
        In \autoref{v48_precalib} a polynomial fit of 2nd degree\footnote{A Fit of second degree will be used in the complete calibration process} is done and its coefficients extracted (\autoref{pitdb}, line 9).
        These coefficients not only show an offset, but also some deviation in the incline and curve from the default values.

        \subsubsection{9.6V Output}
        
        \begin{figure}[H]
            \centering
            \vspace{-1cm}
            \hspace*{-.16\columnwidth}
            \includegraphics[width=1.3\columnwidth]{../pitstop/20180821/calib_v10.pdf}
            \caption{Calibration of 9.6V output Voltage, plotted are an external measurement and internal values vs the recalculated pin voltage based on the default coefficients (\autoref{pitdb-example})}%
            \label{v10_precalib}
        \end{figure}

        The 9.6V Calibration, in contrast, shows only a slight deviation of the internal values and the reference measurement, which results in a list of coefficients (\autoref{}, line 7), very similar to those set in the theoretical defaults.

        This small difference is explained by the simple voltage division used as our circuitry, and no amplification, as for the input voltage circuit.

    \subsubsection{1.8V Output}

        The last Voltage to calibrate is divided into two domains, one for supplying the analog circuitry inside the wafer system, and one for the digital side. Each deliver between 1.5%TODO: value!!
        and 2.022V and each is settable by its own circuit (both as in \autoref{fig:gen18v}).

        \begin{figure}[H]
            \centering
            \vspace{-1cm}
            \hspace*{-.15\columnwidth}
            \includegraphics[width=1.3\columnwidth]{./data/m03_poticalib/adccalib_02.pdf}
            \caption{Calibration: analog 1.8V Output voltage, plotted are external measurement and internal values vs set resistance at the BCU Voltage Module.}%
            \label{fig:v18_precalib}
        \end{figure}

        Visualized in \autoref{fig:v18_precalib} is the analog domains calibration, showing nearly no difference in board and reference measurements. Mostly due to direct connection between created voltage and the STM-Chips pin.

\subsection{Currents}
    
With now calibrated Voltages, the next step is to measure the behavior of the measuring circuits.
Note that the 9.6V Output does in fact not have a include circuit for measuring its current draw, and that this number will be obtainable from all other (calibrated) measurements.

    \subsubsection{48V Input}

        
        \begin{figure}[H]
            \centering
            \hspace*{-.16\columnwidth}
            %TODO: transparent
            %\vspace{-1cm}
            \includegraphics[width=1.3\columnwidth]{../pitstop/20180821/calib_i48.pdf}
            \caption{Calibration of input current ADCs 21.06.2018}
            \label{}
        \end{figure}

    \subsubsection{1.8V Output}
        \begin{figure}[H]
            \centering
            \hspace*{-.15\columnwidth}
            %TODO: transparent
            %\vspace{-1cm}
            \includegraphics[width=1.3\columnwidth]{../pitstop/20180821/calib_i18.pdf}
            \caption{Pre Calibration Measurement of Output Current at the 1.8V Analog and Digital Terminal (2.7.2018)}
            \label{precalib18iana}
        \end{figure}


%\section{after Calibration}


%    \subsubsection{9.6V Output}

%    \subsubsection{1.8V Output}

%\subsection{Currents}

%    \subsubsection{48V Input}

%    \subsubsection{9.6V Output}

%    \subsubsection{1.8V Output}
%        \begin{figure}[H]
    %            \centering
    %            \hspace*{-.16\columnwidth}
    %            \includegraphics[width=1.3\columnwidth]{./pitstop/20180702/i18ana_postcalib.pdf}
    %            \caption{Post Calibration Measurement of Output Current at the 1.8V Analog Terminal (29.06.2018}
    %            \label{postcalib18iana}
%        \end{figure}

\section{1.8V Regulation}

As Described beforehand the Output Voltages for both analog and digital can be adjusted to some degree and therefore we can compensate for the dropoff occurring between PowerIt Output Terminals and Reticles.

\begin{figure}[H]
    \centering
    \includegraphics[width=\columnwidth]{./data/theory/wpattern.pdf}
    \caption{Used regular patterns for current tests on PowerWafer}%
    \label{fig:wpattern}
\end{figure}

\subsection{Characterization of Dropoff}

Wanting to observe and characterize the voltage drop, first the connections between PowerIt and Reticles can be measured with the in \autoref{fig:retmodel} described connections, which in actuality are the PowerIT Terminal and corresponding analog readout pin on a Analog readout board.

To use the PowerWafer for testing one of the patterns in \autoref{fig:wpattern} will be used, each pattern has a approximate current draw of 120A and will distribute heat and draw per terminal evenly.

In \autoref{1v8dip} a single reticles (40) Voltage Dip for different Current Draws is visualized.
A relatively linear trend and residuals of a trigonometric behavior can be observed (most likely the result of the inaccurately measurable current draw, which here is done inside the PowerIt).

\begin{figure}[H]
    \centering
    %\vspace*{-1cm}
    \hspace*{-.16\columnwidth}
    \includegraphics[width=1.3\columnwidth]{./pitstop/20180807/ret_vdip.pdf}
    \caption{Voltage dip observed between PowerIt and HICANN, each point represents a state after enabling additional Reticles on the PowerWafer (right upper wafer in \autoref{fig:wpattern})}
    \label{1v8dip}
\end{figure}

Here a Voltage Drop vs. Current draw of the wafer shows a linear behavior and therefore can be regulated on basis of the current measurement done by on board Measurement circuit.

\subsection{Numerical-Correction (Regulation)}

The initial idea, to approach the correction of this dropoff is a Numerical: the SWRM (\autoref{sec:swrm}) and its corresponding Equations can be applied here.
\autoref{eq:fullreg}, which maps the measured output current to a corresponding potentiometer setting, requires the Dropoff to be linear, which was observed.

To apply this approach, two assumptions need to be made:
\begin{itemize}
    \item all reticles have the same current draw (already not accurate, see \autoref{1v8dip})
    \item all reticles experience the same Voltage Dip (as observed for reticle 40)
\end{itemize}

and the following four values are required, before a regulation can be attempted:
\begin{itemize}
    \item \(I_{ret}\), the current draw of a single reticle,
    \item \(R_0\), the resistance between PowerIt and FET,
    \item \(R_1\), the resistance of a single Reticle

    \item \(V_{off}\), the wanted Voltage at a Reticle
\end{itemize}

To get a representative value of \(I_{ret}\) for use in the SWRM, the mean of all reticles current draw was taken (\autoref{fig:ihist}):

\begin{align}
    \pyval{iretmeancorr}
\end{align}

\begin{figure}[H]
    \centering
    \vspace{-1cm}
    \hspace*{-.15\columnwidth}
    \includegraphics[width=1.3\columnwidth]{../pitstop/20180819/reticle_ihist.pdf}
    \caption{Distribution of analog current draw for all reticles on the PowerWafer (which were possible to measure \(\rightarrow\) \autoref{sec:pitfalls})}%
    \label{fig:ihist}
\end{figure}~\\
The \autoref{fig:ihist} was obtained by measuring the increase in current draw for each reticle, for each of the 4 patterns (\autoref{fig:wpattern}).\\\\
To obtain \(R_0\), the pattern in \autoref{fig:wafer-ret5} was used to take measurements for both the Neighborhood as well as the Farthest Reticles.

\begin{figure}[H]
    \centering
    \hspace*{.1\columnwidth}
    \includegraphics[width=.6\columnwidth]{../pitstop/processing/neighborhoood_5.pdf}
    \vspace{-1cm}
    \caption{Reticles used to determine correlation between distance and Voltage Drop}%
    \label{fig:wafer-ret5}
\end{figure}

\begin{figure}[H]
    \centering
    \vspace*{-1cm}
    \hspace*{-.15\columnwidth}
    \includegraphics[width=1.3\columnwidth]{../pitstop/20180820/reticle_corr}
    \caption{Voltage Dip vs current for both Reticles in direct neighborhood and farthest possible Reticles}%
    \label{fig:ret5corr}
\end{figure}

From \autoref{fig:ret5corr} it is possible to see that the distance between reticles that are used gives different behavior of the Voltage Dip. Both Inclines happen to be the extreme cases, while either being completely uncorrelated, the case for farthest Reticles, or being directly correlated by their distance, here observable for the neighboring Reticles.

Therefore we obtain two values for \(R_0\):

    \begin{align}
        \pyval{r0_from_neighbor}\\
        \pyval{r0_from_farthest}
    \end{align}

from the same measurement it is also possible to extract \(R_1\) by extrapolating to 0, which results in:

    \begin{align}
        \pyval{r1_from_neighbor}\\
        \pyval{r1_from_farthest}
    \end{align}

here the values obtained are within error margin of each other.

So applying these Values, the following behavior for regulation can be visualized:

\begin{figure}[H]
    \centering
    \hspace*{-.16\columnwidth}
    \includegraphics[width=1.3\columnwidth]{./data/theory/reg.pdf}
    \caption{possible \(P_{val}\) curves after SWRM, dotted lines represent not achievable values}%
    \label{fig:regswrm}
\end{figure}

The in \autoref{fig:regswrm} visualized values show the theoretical \(P_{val}\) for the corresponding Current, while all dotted parts depict the values which would be needed to achieve full correction at the Reticle level. 
Note that the 1.8V regulation, should fail at about 80A of current draw.

Now that the SWRM is applicable, what about the DWRM, which removes the assumption of a equal Voltage Dip per Reticle, applying an offset to the initially observed Voltage of each Reticle.

To account for that, the Voltage Dip per Reticle, in a single Reticle power state, was observed:

\begin{figure}[H]
    \centering
    \vspace*{-1cm}
    \hspace*{-.15\columnwidth}
    \includegraphics[width=1.3\columnwidth]{../pitstop/20180820/reticle_vdiphist.pdf}
    \caption{initially observed Voltage Dip, Red values are ignored for corrected mean}%
    \label{fig:vdiphist}
\end{figure}

and a mean of:
\begin{align}
    \pyval{vdipmeancorr}
\end{align}

can be observed.

\begin{figure}[t]
    \centering
    % \hspace*{-.15\columnwidth}
    \includegraphics[width=.7\columnwidth]{../pitstop/20180815/reticel_rdist.pdf}
    \caption{\(V_{dip}\) Distribution over full Power Wafer; White have no measurement; Red an Orange are marked red in \autoref{fig:vdiphist}}%
    \label{fig:wrdist}
\end{figure}

\autoref{fig:wrdist} shows how those Voltages are Distributed over the complete PowerWafer.
All white Reticles are not measurable, and those colored in Red and Yellow are the outliers in \autoref{fig:vdiphist}. Note that in a deployed, working, Wafer System inside BrainScaleS the middle two Reticles (19 \& 28) are not used and also give grounds to ignoring the outliers.

This results in a distribution, which when combined with the spread of \(R_0\) from \autoref{fig:ret5corr}, gives an approximate range for all Reticles Voltage Dip at a given Current Draw (\autoref{fig:vrange}).

\begin{figure}[H]
    \centering
    \hspace*{-.15\columnwidth}
    \includegraphics[width=1.3\columnwidth]{../pitstop/20180820/reticle_variance.pdf}
    \caption{Experimentally obtained Voltage Ranges in which most Reticles Voltage Dip will lie, this does not include outliers}%
    \label{fig:vrange}
\end{figure}


\section{Pitfalls} \label{sec:pitfalls}

