\documentclass[../thesis.tex]{subfiles}
\begin{document}

\chapter{Design and Implementation} \label{sysrapl}

This chapter introduces an implementation of the novel energy profiler called \verb|sysrapl| using methods from Chapter \ref{existing-methods-technology} and using the \emph{eBPF} technology from Chapter \ref{ebpf}. At the beginning of the chapter we describe its implementation (Section \ref{sysrapl-profiler}). Next we describe the implementation of a reporting tool (Section \ref{sysrapl-reporter}) called \verb|sysrapl-report| capable of creating text reports and visualizations. At the end of the chapter we discuss the performance (Section \ref{sysrapl-performance}) and limitations (Section \ref{sysrapl-limitations}) of the created tooling.

\section{Energy profiler} \label{sysrapl-profiler}

% architecture, languages

In this section we'll present the different parts of \verb|sysrapl|: \begin{enumerate*}[(\arabic*)] \item \emph{Engine}, and \item \emph{Post-processor}\end{enumerate*}. \emph{Engine} is responsible for the instrumentation, and \emph{Post-processor} converts raw data generated by the \emph{Engine} into a final profile in JSON format. A high-level diagram of the profiler can be seen in Figure \ref{fig:sysrapl-diagram}.

\begin{figure}[h]
    \centering
    \includegraphics[width=0.40\textwidth]{sysrapl-diagram}
    \caption{Schema of the \emph{sysrapl} profiler. The runtime of the profiler has two phases: profiling and post-processing. The profiling phase instruments \emph{eBPF} programs into the kernel generating raw data. The post-processor then processes the data into the final profile.}
    \label{fig:sysrapl-diagram}
\end{figure}

For the implementation of the energy profiler were used two programming languages: Python and C. We chose Python because Perun (see Chapter \ref{perun}) is implemented in Python and thus using Python will help us ultimately achieve the \textbf{FR\_PI} requirement. The C~language was chosen because it is basically the standard language for Linux system programming and the \verb|libbpf| library (see Section \ref{ebpf-libbpf}) we use for making use of the \emph{eBPF} instrumentation framework is implemented in the C language. The build system used for the project is \emph{Meson}.

The profiler is implemented across these files:

\begin{itemize}
    \item \verb|sysrapl.py| containing the \Gls{cli}, interface for the C shared library and the \emph{post-processor} implementation,
    \item \verb|sysrapl.c| implementing the \Gls{userspace} side of the profiler which handles the instrumentation of \emph{eBPF} programs,
    \item \verb|sysrapl.h| defining data types and structures used in the project, and,
    \item \verb|sysrapl.bpf.c| implementing the different instrumented \emph{eBPF} programs for tracing/sampling.
\end{itemize}

In this thesis we focus primarily on the \textbf{FR\_IU} requirement rather than the \textbf{FR\_PI} requirement. We still structured the top-level of the implementation of \verb|sysrapl| to mirror the structure of Perun's collector (see Section \ref{perun-architecture}) to make future integration in Perun more straightforward.

\subsection{Engine} \label{sysrapl-engine}

% How does the Python module integrate the C library? Give a short example of how to pull in a C shared library.

The \emph{Engine} is in charge of collecting raw performance data by instrumenting \emph{eBPF} programs. In the model of a Perun \emph{collector} it implements the \verb|collect()| function.

Because it is implemented in C, it needs to be made somehow available in the Python script (\verb|sysrapl.py|). Python does support \Gls{ffi} for using shared libraries. There are multiple modules providing complex and robust capabilities, but in line with the \textbf{NFR\_MD} requirement, we chose to use the built-in \verb|ctypes| module~\cite{CtypesForeignFunction}. The module is ideal for simple use-cases where minimum number of functions is called and a small number of structures from the shared library is used as introspection of the loaded library is very limited. However, it has the major downside of requiring to know the absolute path to the shared library. We overcame this limitation in a robust manner by making it mandatory to use the build system to both build \verb|sysrapl| and to install it.

% High-level view of the C library. What libraries are used? Define the entry-point and the lifetime of the profiler. Describe the different phases

The \emph{Engine}, again in line with the \textbf{NFR\_MD} requirement, only requires the standard C~library and the \texttt{libbpf} at runtime. The \emph{Engine} provides a single function \texttt{sysrapl\_profile()} which serves as the entry-point for profiling. It takes as a parameter a structure used for configuring the behaviour of the \emph{Engine} (see Listing \ref{lst:sysrapl-engine-opts}).

\begin{lstlisting}[language=c,caption={Structure for configuring the behaviour of \emph{Engine}},label={lst:sysrapl-engine-opts}]
struct sysrapl_profile_opts {                                                                                    
    char *output_file; // path to a file for outputing raw profiling data
    int delay; // delay before starting attaching the instrumented eBPF programs
    int filter_pid; // PID of the profiled process
    int frequency; // Frequency at which energy readings are sampled
}; 
\end{lstlisting}

% eBPF programs, how they are created, their lifetime and what exactly do they do. How does bpftool fit in?

The \emph{Engine} implements in total three \emph{eBPF} profiling programs:

\begin{enumerate}[\textbf{\alph*})]
    \item \verb|sys_enter()| is attached to the \textbf{tracepoint/raw\_syscalls/sys\_enter} tracepoint. It marks the start of a system call by using the current \Gls{tid} as a key in a hashtable (map \verb|syscall_start|) where the value is the time when the system call started (in nanoseconds). The program only tracks a system call when the current \Gls{pid} equals to the profiled \Gls{pid}.
    
    \item \verb|sys_exit()| is attached to the \textbf{tracepoint/raw\_syscalls/sys\_exit} tracepoint. It marks the end of a system call by checking whether in the map \verb|syscall_start| is the current \Gls{tid}. If yes, the program reserves a chunk of memory in the \Gls{ringbuffer} big enough to hold the \verb|sysrapl_data_t| type. In the chunk of memory is written the id of the system call, its entry and exit times (in nanoseconds), the current CPU and the current \Gls{tid}. Then the chunk of memory is submitted to the \Gls{ringbuffer}.
     
    \item \verb|collect_rapl_info()| is attached to a \textbf{perf event}. This is a special case which requires the program to be manually attached to a perf event during the setup of the \emph{eBPF} program. This program is intended to be attached to the \verb|cpu-clock| software event, which is always available. The software event can be configured with a frequency, which in \verb|sysrapl| is by default 99Hz, and can be configured using the CLI. This fulfils the \textbf{FR\_RDS} requirement. Every time this event is triggered, the \emph{eBPF} is run and all enabled \emph{RAPL} perf events are sampled. And because all of these events are CPU-independent, each needs to be sampled just once. All the sampled values are then put into a memory buffer reserved in and then submitted to the \Gls{ringbuffer}.
\end{enumerate}

The implementation of the two \emph{eBPF} programs: \verb|sys_enter()| and \verb|sys_exit()| fulfils the \textbf{FR\_SCT} requirement. The implementation of the \verb|collect_rapl_info()| fulfils the \textbf{FR\_EDS} requirement.

% bpf helper for reading perf

The \emph{Engine}'s \verb|sysrapl_profile()| function operates in phases which are tied to the lifetime of \emph{eBPF} programs as bootstrapped by \emph{libbpf} (see Section \ref{ebpf-libbpf}):

\begin{enumerate}
    \item \textbf{Open phase.} The first step of the phase is to open a temporary file under \verb|/tmp| directory to contain the raw profiling data. 

    \notebox{Most Linux distributions mount \emph{/tmp} as a \Gls{tmpfs}, making the \emph{Engine} write the raw profile data into operating memory. This is desired due to I/O on storage devices being generally more expensive.}

    The next step is to "open" the group of \emph{eBPF} programs and start initializing the read-only section of the programs. The read-only section of the created programs contains:
    
    \begin{itemize}
        \item \Gls{pid} of the profiled process
        \item Booleans for controlling which RAPL domains to sample 
        \item Maps (see Section \ref{ebpf-overview}) for:
        \begin{itemize}
            \item file descriptors of \emph{perf\_event} for every sampled RAPL domain (one map per file descriptor),
            \item tracking of running system calls,
            \item \Gls{ringbuffer} for exchanging data between \Gls{kernelspace} and \Gls{userspace}
        \end{itemize}
    \end{itemize}
    
    Because not every machine supports all of the \emph{RAPL} domains, a check needs to be done, to only open the perf events that exist. This mechanism helps to fulfil the \textbf{FR\_EDC} requirement. And because the \emph{RAPL} perf events are \Glspl{pmu}, their \emph{type} value and \emph{config} value (see Section \ref{existing-methods-technology-rapl}), used when "opening" the perf events, can vary across machines, so during the check these values are collected. The values for \emph{RAPL} perf events can be found in \Gls{sysfs} under \verb|/sys/devices/power|. The present events are then opened and marked as usable using the booleans in the read-only section. The events are not enabled and are not put yet into the maps, as the programs are not loaded in-memory, yet.
    
    \item \textbf{Load phase.} Following the load of \emph{eBPF} programs the maps of the sampled \emph{RAPL} perf events are populated with file descriptors of the present perf events. A \Gls{ringbuffer} is also created and assigned to the prepared map in the \emph{eBPF} programs. The ringbuffer is used for sending data from the \emph{eBPF} programs running in \Gls{kernelspace} to the \emph{Engine} running in \Gls{userspace} During the creation of the \Gls{ringbuffer} a function is assigned to it to handle data arriving from \Gls{kernelspace} during polling. The function is \verb|handle_ring_buffer()| and it outputs the received data in the temporary file opened during the \emph{Open phase}.
    
    To make the \verb|collect_rapl_info()| program work, it needs to be attached to the \verb|cpu-clock| perf event. A helper function \verb|attach_perf_event_sampling()| opens the \verb|cpu-clock| perf event and attaches the program to it.
    
    If the profiler is configured to wait for a period of time before the start of profiling, the profiler waits in this phase. This capability exists to help mitigate the effect of noise caused by starting the tool or other software.
    
    Right before the next phase, the \emph{RAPL} perf events are reset, to ensure proper initial values in the counters, and are enabled.
    
    \item \textbf{Attachment phase.} Once an \emph{eBPF} program is attached, it starts functioning right away and can no longer be configured. In this phase the \emph{Engine} enters a poll loop where it waits for data to arrive via the \Gls{ringbuffer}. When data arrive, the poll function triggers a handler function, assigned to the \Gls{ringbuffer} in the \emph{Load phase}.
    
    The polling continues until the user sends to the profiler the \emph{SIGINT} signal.

    \item \textbf{Teardown phase.} In this phase the different resources, including the \emph{eBPF} programs are cleaned up. The only resource left intact is the temporary file holding raw profiled data.
\end{enumerate}

\subsection{Post-processor} \label{sysrapl-postprocessor}

The \emph{Post-processor} is in charge of transforming the raw profile data into a readable format, which can then be used for further analysis.

The \emph{Post-processor} operates in two stages:

\begin{enumerate}
    \item \textbf{Processing of energy reading samples.} In the raw data the energy reading samples contain values present at the time in the counters of the perf events. To make sense of these values the \emph{Post-processor} needs to scale these values using the scale assigned to every single perf event (see Section \ref{existing-methods-technology-rapl}) to convert them to proper units (usually Joules). After the conversion the values represent the energy consumed since the last overflow of the counter. In further analysis we're more interested in how much energy was consumed during each sampled time window. So, two neighbouring samples are used to calculate the difference between them and the result is assigned to a time window.

    \item \textbf{Processing of system calls.} After the previous stage the energy consumption of different time windows hence computed. This stage uses this data to annotate energy consumption scores to every single system call.
    
    To fulfil the \textbf{FR\_SCS} requirement this stage implements a scoring algorithm but as explained in Section \ref{requirements-analysis} the implementation is intentionally trivial. The steps of the algorithm are:
    
    \begin{enumerate}[\arabic*)]
        \item Select a system call from the raw data.
        \item Set the system call's energy score to 0.
        \item Find all time windows overlapping the system call run time.
        \item Calculate the ratio of the runtime of the system call in the first time window.
        \item Multiply the time window's energy readings by the ratio.
        \item Add the energy reading of the time window to the system call energy score.
        \item Remove the first time window from the list of windows. 
        \item If there is another time window, go to step 4.
    \end{enumerate}
\end{enumerate}

After the end of the two stages, the post-processor outputs the processed profile data in the JSON format. By default \verb|sysrapl| outputs the profile to \verb|stdout| but it can be configured to save it into a file. The advantage of this approach is that it can be used in pipes with other UNIX utilities.

\subsubsection{Profile data format} \label{sysrapl-postprocessor-data}

In previous sections we've mentioned that \verb|sysrapl| outputs the final profile data in the JSON format. In this section we'll describe what information are put into the profile data and what is its structure.

\paragraph{Machine information.} Energy profiling varies largely between different hardware configurations, specific hardware models and what software runs on the hardware. In the profile data \verb|sysrapl| includes, alongside the information about the profiled process, the time when it was profiled, the duration of the profiling session (in nanoseconds), and information about the machine on which \verb|sysrapl| ran. The system information are only high-level and created by combining the output of \verb|platform.uname()| and the content of \verb|/proc/cpuinfo|. Including this data satisfies the \textbf{NFR\_PCP} requirement. Listing \ref{lst:sysrapl-profile-info} shows a snippet from the data JSON with a section containing this information.

\begin{lstlisting}[caption={\emph{info} section of the final profile data format containing information about the profiled software and the host system}, label={lst:sysrapl-profile-info}]
{
    "info": {
        "process-name": "gnome-shell",
        "pid": 1598,
        "time": "2023-04-26T09:03:46",
        "duration": 30215400875,
        "frequency": 99,
        "host-info": {
            "system": "Linux",
            "node": "localhost",
            "release": "6.2.9-300.fc38.x86_64",
            "version": "#1 SMP PREEMPT_DYNAMIC Thu Mar 30 22:32:58 UTC 2023",
            "machine": "x86_64",
            "processor": "Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz"
        }
    },
    ...
}
\end{lstlisting}

\paragraph{Energy consumption samples.}

This section of the data JSON contains all data related to system-wide energy consumption: \begin{enumerate*}[\alph*)]\item the unit and the scale of the energy values, \item sampled energy domains, \item total consumption across all domains (corresponding to those described in Section \ref{existing-methods-technology-rapl}), and, \item a list of all sampled time windows with energy consumed (using the unit and the scale in the profile)\end{enumerate*}. Listing \ref{lst:sysrapl-profile-rapl} shows a snippet from the data JSON with a~section containing this information.

\begin{lstlisting}[caption={\emph{rapl} section of the final profile data format containing system-wide energy consumption data}, label={lst:sysrapl-profile-rapl}]
{
    ...
    "rapl": {
        "unit": "Joules",
        "scale": 0.23283064365386963,
        "domains": ["cores", "pkg", "ram", "gpu", "psys"],
        "total": {
            "time": 30151676016,
            "cores": 5413635254,
            "pkg": 28118835449,
            "ram": 24002197265,
            "gpu": 869750976,
            "psys": 115713439942
        },
        "events": [{
            "time": 921404653026,
            "cores": 9521485,
            "pkg": 22277832,
            "ram": 11596679,
            "gpu": 1647949,
            "psys": 72692871
        },...]
    },
    ...
}
\end{lstlisting}

\paragraph{System calls.}

This section contains a list of all system calls ordered by their exit time. Each item in the list contains: \begin{enumerate*}[\alph*)]\item the system call ID, \item the system call name, \item the entry time (in nanoseconds), \item the exit time (in nanoseconds), and, \item the energy consumption data (using the unit and the scale in the profile)\end{enumerate*}. Listing \ref{lst:sysrapl-profile-syscalls} shows a snippet from the data JSON with a section containing this information.

\begin{lstlisting}[caption={\emph{syscalls} section of the final profile data format containing a list of all the traced system calls}, label={lst:sysrapl-profile-syscalls}]
{
    ...
    "syscalls": [{
        "id": 39,
        "name": "getpid",
        "cpu": 2,
        "tid": 2779,
        "entry_time": 1202281290743,
        "exit_time": 1202281292402,
        "cores": 2796.828618969,
        "pkg": 5082.4088385840005,
        "ram": 3007.342549827,
        "gpu": 120.293721702,
        "psys": 18204.446818404
    }...]
}
\end{lstlisting}

\subsection{Performance} \label{sysrapl-performance}

In this section we want to showcase the performance of \verb|sysrapl| on a synthetic benchmark to see how big of an overhead does the energy profiler introduce. As the benchmark we used an open-source TCP server capable of handling simple echo messages implemented using \verb|io_uring|~\cite{hielkedevriesIoUringechoserver} coupled with an open-source TCP client application for testing such simple servers~\cite{hoyerRustEchoBench}. In the benchmarks the provider of the results is the client application which at the end of its runtime prints statistics collected during its runtime, where the statistics show the number of requests sent and responses received.

For this benchmark we use the same testing environment as in Chapter \ref{experiments}. In the benchmarks the server is run as \verb|io_uring_echo_server 8080| where \verb|8080| is the port the server listens on, and the client is run as \verb|echo_bench -a 127.0.0.1:8080 -t 10 -c 4|, where \verb|127.0.0.1:8080| is the address and the port the client application connects to, \verb|10| is the number of seconds for which the application is run and \verb|4| is the number of clients the applications spawns in separate threads. In total we will conduct four different benchmarks:

\begin{enumerate}[\textbf{\alph*)}]
    \item \textbf{clean}, where the server is not profiled. This benchmark will be used as the baseline.
    \item \textbf{sysrapl-default}, where the server is profiled using \verb|sysrapl| with default parameters (\emph{RAPL} value sampling at 99Hz).
    \item \textbf{sysrapl-highfreq}, where the server is profiled using \verb|sysrapl| sampling \emph{RAPL} values at 9999Hz.
    \item \textbf{strace}, where the server is profiled using the \verb|strace| tool. \verb|strace| is known for introducing relatively high overhead to the profiled software and will serve as additional context for this benchmark. In this benchmark the output of the \verb|strace| tool is redirected to \verb|/dev/null| to reduce the effect of printing text into the terminal window on the benchmark.
\end{enumerate}

Table \ref{table:sysrapl-performance} shows the results of the different benchmarks where the results are the average of three sets of results for every benchmark. The results show that in applications with highly frequent system calls in both benchmarks \emph{sysrapl-default} and \emph{sysrapl-highfreq}  \verb|sysrapl| degrades the performance by about 20\%. The small difference between \emph{Performance ratio} of the \emph{sysrapl-default} and \emph{sysrapl-highfreq} benchmarks shows that the majority of the performance overhead comes from tracing system calls and not from sampling \emph{RAPL} energy readings, making our method of sampling of the events quite efficient. The \emph{strace} benchmarks confirms the general believe that the \verb|strace| tool does introduce high overhead to the profiled software. The method of tracing system calls in \verb|sysrapl| compared to the method of tracing in \verb|strace| shows significantly better results and reaffirms our believe in the efficiency of the \emph{eBPF} instrumentation framework.

\begin{table}[h]
\centering
\begin{tabular}{|l|r r|r|} 
    \hline
    Benchmark & Requests [req/sec] & Responses [res/sec] & Performance ratio [\%] \\
    \hline
    \textbf{clean} & $78,780$ & $78,780$ & $100.0\%$ \\
    \hline
    \textbf{sysrapl-default} & $65,249$ & $65,249$ & $82.8\%$ \\
    \hline
    \textbf{sysrapl-highfreq} & $62,790$ & $62,790$ & $79.7\%$ \\
    \hline
    \textbf{strace} & $29,316$ & $29,316$ & $37.2\%$ \\
    \hline
\end{tabular}
\caption{Results of the benchmarks showing the runtime overhead of \texttt{sysrapl} with different configurations and \text{strace}.}
\label{table:sysrapl-performance}
\end{table}

\section{Reporter} \label{sysrapl-reporter}

% Written in Python using these libraries -> xxx, xxx, xxx,...

In this section, we'll describe the implementation and capabilities of a reporting tool called \verb|sysrapl-report|. Statistical analysis and generation of graphs is commonly done using languages like R or Python. As stated in Section \ref{sysrapl-profiler}, Perun is written in Python, so using it for the implementation of \verb|sysrapl-report| is a natural choice. In \verb|sysrapl-report| we make use of a number of modules which are well-maintained, widely used, and already used by Perun. For data processing we used \emph{Pandas}~\cite{mckinneyDataStructuresStatistical2010,teamPandasdevPandasPandas2023}, and for plotting \emph{Seaborn}~\cite{waskomSeabornStatisticalData2021} (which uses \emph{matplotlib}~\cite{hunterMatplotlib2DGraphics2007}).

% File info, synopsis

\verb|sysrapl-report| is implemented as a single \verb|sysrapl-report.py| script. By default it outputs its results into a directory named based on the data from the interpreted profile file. \verb|sysrapl-report| takes as its input a single profile data JSON file and it only uses this file to produce its results. This is to fulfil the \textbf{NFR\_RP} requirement.

% Most details of the implementation are not crucial and thus won't be elaborated on in this work. The one part of \verb|sysrapl-report| deserving being elaborated on is the processing of profile data.

% outliers (IQR)

The number of system calls that a process can invoke in a matter of seconds can easily reach thousand digits. \verb|sysrapl-report| mainly cares about the \emph{energy scores} of the different system calls whose values can differ largely. Presence of outliers is commonplace which can significantly affect the final outputs of \verb|sysrapl-report|. To detect outliers we use the \emph{interquartile range} (\Gls{iqr}) method which we then filter out from the datasets. The method separates the data into four quartiles ($Q_1$, $Q_2$, $Q_3$, $Q_4$) where the first quartile ($Q_1$) and the third quartile ($Q_3$) are used to calculate the \Gls{iqr}. Then the \Gls{iqr} value is multiplied by $1.5$ and subtracted from $Q_1$ and added to $Q_3$. Then all values below $Q_1 - 1.5 * IQR$ and above $Q_3 + 1.5 * IQR$ are considered outliers. 

\subsection{Text report} \label{sysrapl-text-report}

Visualizations and graphs are a great way to convey complex data. For simple data a text report is often more suitable for its simplicity. \verb|sysrapl-report| outputs a brief text report for an interpreted profile in the output directory in a file called \verb|report.txt|.

\notebox{A concrete example of the text report can be seen in Appendix \ref{appendix-full-profile}.}

% Explain the different sections in the output

The text report contains the following sections:

\begin{enumerate}
    \item \textbf{General information.} Contains the name and PID of the profiled process, time of profiling, its duration and the sample rate of the \emph{RAPL} energy domains.
    \item \textbf{System information.} Because energy profiling is hardware specific, a section about the profiled system is included based on JSON section described in Section \ref{sysrapl-postprocessor-data}.
    \item \textbf{Ratio of energy consumption.} By mapping energy consumption to system calls it can be possible to estimate what percentage of the system energy consumption can be attributed to the profiled process.
    \item \textbf{Top consuming system calls.} Shows the top five contributors among system calls to the process's energy consumption. The number of shown top contributors can be configured.
\end{enumerate}

\subsection{Visualizations} \label{sysrapl-visualizations}

Every kind of data can be visualised using a different method. \verb|sysrapl| does generate heterogeneous data and thus different visualizations are needed for accurate assessment of its findings. In \verb|sysrapl-report| we implemented a number of graphs which we categorise into two groups (due to two categories of data being provided by \verb|sysrapl|):

\begin{enumerate}[\textbf{\alph*})]
    \item \textbf{Raw energy consumption} (Section \ref{sysrapl-report-raw-energy-consumption}), and,
    \item \textbf{System calls as energy consumers} (Section \ref{sysrapl-report-system-calls-as-energy-consumers}).
\end{enumerate} 

We designed these visualizations to be easy to understand to fulfil the \textbf{NFR\_CP} requirement.

\notebox{The following images have been cropped to better show the form of the different graphs. The exact readings are not relevant in this section. An example report with full forms of the graphs can be seen in Appendix \ref{appendix-full-profile}.}

\subsubsection{Raw energy consumption} \label{sysrapl-report-raw-energy-consumption}

The first data category provided by \verb|sysrapl| includes the raw energy consumption readings. The readings serve as a basis for further analysis but even their raw form provides interesting insights. The energy readings are separated into domains explained in Section \ref{existing-methods-technology-rapl}.

\paragraph{Consumption at a certain point of time.} \label{sysrapl-report-raw-energy-consumption-rapl-time}

The most basic graph plots the sampled energy readings in time using line plot. It shows the energy behaviour of the whole system and serves as a good entry point for getting a basic understanding of the system's energy behaviour. But due to no analysis being involved, it is highly susceptible to noise.

The X axis represents time and the Y axis represents the consumed energy.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.65\textwidth]{sysrapl-report-rapl-time}
    \caption{Line plot of per-energy domain energy consumption in time}
\end{figure}

\paragraph{Cumulative consumption}

An alternative representation of the above graph with the difference of showing the energy readings in a cumulative manner.

The X axis represents time and the Y axis represents the consumed energy.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.65\textwidth]{sysrapl-report-rapl-cumulative}
    \caption{Line plot of cumulative per-energy domain energy consumption in time}
\end{figure}

\subsubsection{System calls as energy consumers} \label{sysrapl-report-system-calls-as-energy-consumers}

The second data category contains all system calls called by the profiled process. Using the raw energy consumption data every single system call has been given an \emph{energy score}. This process helps to reduce noise significantly. The scoring algorithm is described in Section \ref{sysrapl-postprocessor}.

\paragraph{Energy consumption in time.}

Similarly to the graph in Section \ref{sysrapl-report-raw-energy-consumption-rapl-time}, the most basic plot is showing the energy consumption of system calls in time. But this data category is more challenging due to higher number of data categories and uneven number of data points. Every type of system call can have up to five \emph{energy scores} and by default \verb|sysrapl-report| reports more than one system call. For displaying high number of samples across many domains we chose two types of graphs: \begin{enumerate*}[\emph{\alph*)}] \item \emph{heatmap}, and, \item \emph{waterfall graph} \end{enumerate*}.

\begin{enumerate}[label=\emph{\alph*)}]
    \item \textbf{Heatmap} \label{sysrapl-report-system-calls-energy-time-heatmap}

    Plotting values in time using heatmaps provides a great way of spotting exact points of time where a significant chunk of energy was consumed. To allow seeing the maximums reliably the dataset extremes are used as the border values. The graph has its limits as the number of plotted samples needs to be capped at a low enough number to prevent the cells from becoming too small to perceive their colour. In this work we achieve this by downsampling the data points and summing them up. The sample width is adjustable.

    The X axis represents time, on the Y axis are different (\emph{system call}, \emph{energy domain}) combinations and the colour of the cells represents the consumed energy. The brighter the colour, the more energy was consumed by given pair at given time.

    \begin{figure}[H]
        \centering
        \includegraphics[width=0.65\textwidth]{sysrapl-report-syscall-time-heatmap}
        \caption{Heatmap of per-energy domain system call energy consumption in time}
    \end{figure}

    \item \textbf{Waterfall graph} \label{sysrapl-report-system-calls-energy-time-waterfall}

    The graph is based on Brendan Gregg's \emph{frequency trails}~\cite{brendanFrequencyTrails2014} which he designed to show the distribution of sampled data across many domains. In this work we adopted \emph{frequency trails} into a so called \emph{waterfall graph} displaying on a normalized scale the energy consumption of system calls in time. Similarly to the heatmap, every plot represents a~different (\emph{system call}, \emph{energy domain}) combinations. These combinations are sorted from the most expensive ones to the least expensive ones. This allows at a single glance to see both the highest and smallest consumers. Unlike the heatmap, the waterfall graph does not require the data points to be downsampled to fit into the graph. Still the data points are downsampled and then upsampled again to achieve a smoothing effect in the graph.

    The X axis represents time and the Y axis represents the consumed energy on a~normalised scale.

    \begin{figure}[H]
        \centering
        \includegraphics[width=0.65\textwidth]{sysrapl-report-syscall-time-waterfall}
        \caption{Waterfall graph of per-energy domain system call energy consumption in time}
    \end{figure}
\end{enumerate}

\paragraph{Cumulative consumption in time.}

The \emph{tsunami graph} is a variant of the \emph{waterfall graph} (\ref{sysrapl-report-system-calls-energy-time-waterfall}) where instead of plotting the values at a certain point of time we plot the cumulative values, i.e. the sum of all previous values. This allows us to show (\emph{system call}, \emph{energy domain}) combinations with the highest energy consumptions.

The X axis represents time and the Y axis represents the consumed energy on a normalised scale.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.65\textwidth]{sysrapl-report-syscall-cumulative-tsunami}
    \caption{Tsunami graph of cumulative per-energy domain system call energy consumption}
\end{figure}

\paragraph{Distribution of energy consumption.}

Different system calls serve for different purposes and thus their energy behaviour can largely differ. Using a box plot we can show the distribution of the consumed energy.

The X axis shows different system calls across energy domains and the Y axis shows the energy consumption. The box represents three quartiles (first, median and third), the whiskers represent maximum and minimum. Values outside of the whiskers are outliers.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.65\textwidth]{sysrapl-report-syscall-distribution}
    \caption{Box plot showing distribution of energy consumed by system calls across different energy domains}
\end{figure}

\section{Limitations} \label{sysrapl-limitations}

\paragraph{Use of phases.} During development we've focused mainly on the \textbf{FR\_RO} requirement which lead us to design \verb|sysrapl| in phases. This decision in some cases causes problems when the post-processing time (\textbf{FR\_PPT}) becomes so high, that the use of the tool is no longer practical. Designing the post-processor capable of processing the raw data continuously and in batches could be viable compromise. 

\paragraph{Number of profiled processes.} At the moment \verb|sysrapl| can profile only a single process. Removing this limitation could help with the \textbf{FR\_NS} requirement because the energy consumption could be split between the different processes. With such a change the scoring algorithm (\textbf{FR\_SCS}) would need to be adjusted as well.

%% possible optimization for the bpf helper by adding support for group reads
\paragraph{Support for \texttt{perf\_event} groups in \emph{eBPF} programs.} At the moment \emph{eBPF} programs can read counter values of perf events only one at a time while \Gls{userspace} applications may make use of the capability to group different perf events and read their values in bulk. Having this capability available for \emph{eBPF} programs could provide some performance boost.

% Visualization
% Speed (maybe include a high-level callgraph showing the bottlenecks; or just an output of 'time sysrapl-report ...')

\paragraph{Speed.} Despite the \textbf{FR\_IT} requirement for \verb|sysrapl-report|, the average runtime of the tool is \~35 seconds for a profile with >11000 sampled system calls on a Lenovo ThinkPad T460 laptop with Intel Core i5-6300U processor (used during experiments in Chapter \ref{experiments}). We believe this is due to \emph{matplotlib} being a single-threaded CPU-bound module which does not scale well with multiple plots with a high number of samples. Use of plotting libraries capable of utilizing multiple threads or GPU capabilities would significantly reduce the average runtime of the tool.

% Separation of text report and graphs -> need for a dashboard

\paragraph{Separation of output.} During the design of \verb|sysrapl-report| we've focused greatly on the \textbf{NFR\_RCP} requirement. We believe we met this requirement and can only further raise the bar. At the moment \verb|sysrapl-report| outputs the text report and graphs as separate files. Bundling these into a single report would with interpreting the results and also archiving the results as only a single file would be needed.

\end{document}