\documentclass[../thesis.tex]{subfiles}
\begin{document}
\chapter{Perun} \label{perun}

\emph{\textbf{Per}formance \textbf{Un}der Control} (\emph{Perun}) is an open source \emph{Performance Version System} used for continuous tracking of a project's performance~\cite{fiedorPerunDocumentation2022}. The project integrates with \Glspl{vcs} like \emph{Git} or \emph{SVN}, thus keeping performance testing closely tied to it. \emph{Perun} also provides a tool suite for specification of test runs, performance metrics collectors, performance data postprocessors and analyzers. The results of this thesis will be in near future integrated in Perun. To provide a better understanding of the content of the work, we will provide an overview of Perun (Section~\ref{perun-overview}), describe its architecture (Section~\ref{perun-architecture}) and workflow (Section~\ref{perun-workflow}) in the following sections.

\section{Overview} \label{perun-overview}

\Gls{vcs} track how the code base of a project evolves, how the functionality changes, provides versions snapshots (e.g., tags) and often provide additional generic functionality in order to satisfy as many needs as possible. These systems are often flexible enough for keeping track of additional data but they are not optimized for such use cases. Perun fills the gap for tracking a project's performance by using \Gls{vcs}s to gain insight into a project development history and store its results into a \verb|.perun| directory in the project tree (similarly to \emph{Git} storing its state in a \verb|.git| directory).

Perun offers the following advantages over the sole use of \Gls{vcs} or databases~\cite{fiedorPerunDocumentation2022}:

\begin{itemize}
    \item \textbf{Context.} Results of test runs (so called performance profiles) are tied to an exact version of the corresponding \Gls{vcs}, thus providing the necessary context (e.g., changed lines of code) to focus the analysis efforts on the recent differences in the source code only.
    
    \item \textbf{Automation.} Manual analysis is a common activity, but not in the context of automated test system. Perun provides a concept of \emph{jobs} for defining test runs and a concept of \emph{hooks} reacting to \Gls{vcs} events (e.g., commit, push, tag).
    
    \item \textbf{Genericity.} Every project is different and their expectations of Perun may vary. Perun thus provides a framework allowing for a straightforward extension of its capabilities (collectors, postprocessors and visualisations) via modules. Its data format (based on a \emph{JSON} notation) is flexible enough for a quick adaption.
    
    \item \textbf{Ease of use.} Perun is available as a CLI\footnote{Command Line Interface} application with commands similar to common \Gls{vcs} applications, which lowers the learning curve of using the tool.
\end{itemize}

\section{Architecture} \label{perun-architecture}

Perun's architecture, illustrated in Figure \ref{fig:perun-architecture}, is made of four main components: \emph{logic}, \emph{data}, \emph{check}, and \emph{view}. The main components are complemented by a collection of utility methods as well as move advanced or specialized features: \emph{\Gls{vcs}}, \emph{workloads}, \emph{fuzzing}, etc.

\begin{figure}[h]
    \centering
    \includegraphics[width=0.75\textwidth]{perun-architecture}
    \caption{An overview of Perun's architecture showing its main components, \emph{\Gls{vcs}} integrations and the flow of data in the program.~\cite{fiedorPerunLightweightPerformance2022}}
    \label{fig:perun-architecture}
\end{figure}

\paragraph{Logic} is responsible for the generation of performance profiles, automation (hooks) and higher-logic manipulations~\cite{FITMT19092, fiedorPerunLightweightPerformance2022}. Its two major components are:

\begin{itemize}
    \item \textbf{Collectors} collecting performance data. They can be implemented either as wrappers of existing profilers and utilities or they can be fully-featured profileres on their own. Notable collectors are: \emph{trace collector}, \emph{complexity collector}, or the simple \emph{time collector}.
    
    \item \textbf{Postprocess} applies various statistical analysers to the collected performance profiles. Notable postproceessors are: \emph{moving average postprocessor}, \emph{regression analysis postprocessor} or \emph{clusterizer postprocessor}
\end{itemize}

\paragraph{Data} is the core of Perun \textendash performance profile manipulation is done in this component together with its storage in a \Gls{vcs}. It is the middleware between \emph{logic} and \emph{view}.~\cite{FITMT19092, fiedorPerunLightweightPerformance2022}

\paragraph{Check} implements various performance regression detections across different revisions (profiles) of a project. Several detection methods are available due to the high sensitivity of the used statistical techniques to the type of analysed data. Notable detection methods are: \emph{linear regression}, \emph{integral comparison}, or \emph{average amount threshold}.~\cite{FITMT19092}

\paragraph{View} provides means for the collected and processed performance profiles to be visualized. Some of the visualizers are: \emph{bars plot}, \emph{flame graph} or \emph{scatter plot}~\cite{FITMT19092, fiedorPerunLightweightPerformance2022}

\section{Workflow} \label{perun-workflow}

Perun's workflow can be illustrated in a series of steps the user should perform. The description of these steps is based on~\cite[p.24]{FITMT19092}. See Figure~\ref{fig:perun-flow} for a visualization of Perun's workflow.

\begin{figure}[h]
    \centering
    \includegraphics[width=0.75\textwidth]{perun-flow}
    \caption{An illustration of the Perun workflow on a project repository using the Git \Gls{vcs}.~\cite{fiedorPerunLightweightPerformance2022}}
    \label{fig:perun-flow}
\end{figure}

The steps of an usual Perun's workflow can be described as follows: 

\begin{enumerate}
    \item We initialize a new Perun repository in a project managed by a \Gls{vcs} using \verb|perun init| (the initialized Perun repository is not managed by the \Gls{vcs}).

    \item We configure the initialized Perun repository and select the desired collectors, post-processors, workloads and other options.

    \item We make changes to the tracked project and commit them into \Gls{vcs} (this may trigger the next step automatically depending on the configuration from the previous step).

    \item We collect raw data using either the pre-configured collectors or using a manually selected collector using \verb|perun collect|. The raw data are stored as a profile in the Perun repository.

    \item We optionally process the raw data using either the pre-configured post-processors (e.g., regression analysis, clusterizer) or using a manually selected post-processor using \verb|perun postprocessby|. The processed data are stored in the current profile in the Perun repository.

    \item We compare the current profile with profiles paired with previous revisions of the project using either the pre-configured degradation checker (e.g., integral comparison) or a manually selected degradation checker using \verb|perun check|. The check is done only for matching pairs of tested $binaries + workloads + configurations$. The results of the check are stored as a new object in the Perun repository.

    \item We assess the severity, location and confidence parameters of reported potential performance changes.
\end{enumerate}

\end{document}