\setcounter{chapter}{4}
\chapter{Evaluation of results}

All algorithms were tested by using real radiological sources. In the first part of this chapter, the most successful results are presented. They are compared to simulations previously introduced in this thesis. In the second part of this chapter, results of different algorithms are compared to each other. The experiments were realized with reconnaissance robot Orpheus-X3 introduced in Sec.~\ref{sec.orpheus}.

%\begin{figure}[!h]
%  \begin{center}
%    \includegraphics[scale=0.9]{images/radiace_tomL.jpg}
%  \end{center}
%  \caption{Experimental result of mapping}
%  \label{fig.map0}
%\end{figure}

Stated activities of all sources are valid to date when the corresponding experiment was performed. The activity is given by a calibration value to a specified date and a radioactive decay. An example of a calibration protocol for one of the used sources can be found in the Appendix~\ref{ap.calib}. Activity $A$ in time $t$ is defined by formula \cite{decay}:

\begin{equation}
A = A_0e^{-\frac{t}{T_{1/2}\ln(2)}} \myunit{Bq; Bq, y, y}
\end{equation}
where $A_0$ is activity in time $t=0$, and $T_{1/2}$ is a half-life of given radionuclide. The half-life and time may be expressed by arbitrary time unit but both by the same. Half-life of the used radionuclide is: 30.021 years for $^{137}$Cs and 5.275 years for $^{60}$Co \cite{half-life}. For example, used $^{60}$Co source had activity 100 MBq to 5th December 2013. The first experiment has been performed 2nd March 2017. Activity to this date is therefore:

\begin{equation}
A = 100e^{-\frac{3.241}{5.275\ln(2)}} \hspace{2mm}\unit{MBq} = 41.2 \hspace{2mm}\unit{MBq}
\end{equation}

Activities of other sources to other dates may be calculated analogically.


\section{Experimental results}

Setup of mapping experiment can be seen in Tab.~\ref{tab.mapping}. It is very similar to the setup of simulation in order to provide better comparison. An unprocessed result of mapping is shown in Fig.~\ref{fig.map0}. It might be rather difficult to realize that three sources are present in the polygon, because the weak sources are effaced by the strong one. However, after the interpolation is performed, individual sources can be easily identified. Autonomous localization is performed as well using parameters gained in simulation. The interpolated map with marked computed positions of sources is shown in Fig.~\ref{fig.map1}. The parameters were well-chosen, an error of computed position is of the order of centimeters.

\begin{table}[ht]
\catcode`\-=12

\begin{center}
\caption{Setup of the mapping experiment}
\label{tab.mapping}

\begin{tabular}{|p{5cm}|l|l|l|}

\hline
& $S_1$ & $S_2$ & $S_3$\\
\hline
Radionuclide & $^{60}$Co & $^{137}$Cs & $^{60}$Co\\
\hline
Activity & 8.0 MBq & 220 kBq & 350 kBq\\
\hline
Position (relative to BS) [m] & \vect{40.6,-19.3} & \vect{39.1,-10.7} & \vect{48.5,-22.8}\\
\hline

\end{tabular}
\end{center}
\end{table}

\begin{figure}[!h]
  \begin{center}
    \includegraphics[height=9.9cm]{images/mapping0.pdf}
  \end{center}
  \caption{Experimental result of mapping}
  \label{fig.map0}
\end{figure}

\begin{figure}[!h]
  \begin{center}
    \includegraphics[height=9.9cm]{images/mapping1.pdf}
  \end{center}
  \caption{Interpolated radiation map}
  \label{fig.map1}
\end{figure}

Two different setups were used for testing the strong source search algorithm. Their overview is in Tab.~\ref{tab.locating}. The result of the experiment with the second setup is shown in Fig.~\ref{fig.loc}. The actual position of the source is marked by a black cross. Data from the first two lines were sufficient to estimate direction of source with enough precision. An error of computed position is also of the order of centimeters. It is important to use the final loop. In case of the first setup, initial estimation of direction was less precise but thanks to looping, position could be determined well.

\begin{table}[ht]
\catcode`\-=12

\begin{center}
\caption{Setup of the strong source seach experiment}
\label{tab.locating}

\begin{tabular}{|p{5cm}|l|l|}

\hline
& Setup 1 & Setup 2\\
\hline
Radionuclide & $^{60}$Co & $^{137}$Cs\\
\hline
Activity & 8.0 MBq & 65.6 MBq\\
\hline
Position (relative to BS) [m] & \vect{38.0,-18.7} & \vect{40.6,-19.3}\\
\hline
Distance to base line [m] & 5.7 & 7.5\\
\hline

\end{tabular}
\end{center}
\end{table}

\begin{figure}[!h]
  \begin{center}
    \includegraphics[height=9.9cm]{images/locating.pdf}
  \end{center}
  \caption{Experimental result of the strong source search}
  \label{fig.loc}
\end{figure}

Setups of experiments with first and second circular algorithms are in Tab.~\ref{tab.circ12}. For both experiments a common circle was used – its parameters are the same as parameters of circle $c_2$ in Tab.~\ref{tab.circ3circ}. The experimental result of the first circular algorithm is shown in Fig.~\ref{fig.circ1}. The robot approached the circle along tangent as planned and then went round the whole circle. It has been discovered that the choice of value of parameter $p$ (peak shift) according to simulation is not correct. In real situation, the peak shift is better characterized by $p=-0.2$. After a correction of the parameter, the experiment could be  repeated. Thanks to proximity of the source to the circle and its intensity, the initial estimation of direction was precise enough and there was almost no need to update direction during approach to the source.

The experimental result of the second circular algorithm can be seen in Fig.~\ref{fig.circ2}. Localization of the source $S_1$ was not a problem, the result is very similar to the previous experiment. Regarding the other source, due to its higher distance to the circle, initial estimation of a direction was very inaccurate – approximately 15° error. During the first attempt the control algorithm was not able to navigate the robot to the source correctly due to a problem in updating the direction. It was necessary to alter values of respective parameters $K$ and $\Delta\phi$ in order to manage smoother change of direction. New values of parameters are: $K=1.1, \Delta\phi = 9^\circ$. After that change, the source $S_2$ was successfully located as well.


\begin{table}[ht]
\catcode`\-=12

\begin{center}
\caption{Setup of the first and second circular algorithm experiment}
\label{tab.circ12}

\begin{tabular}{|p{5cm}|l|l|l|}

\hline
& 1st circ. alg. & \multicolumn{2}{|l|}{2nd circ. alg.}\\
\cline{1-1}\cline{3-4}
& & $S_1$ & $S_2$\\
\hline
Radionuclide & $^{60}$Co & $^{60}$Co & $^{60}$Co\\
\hline
Activity & 40.1 MBq & 7.8 MBq & 40.0 MBq\\
\hline
Position (relative to BS) [m] & \vect{47.4,-24.6} & \vect{47.4,-24.6} & \vect{38.0,-18.7}\\
\hline
Position (relative to circle) & Outside & Outside & Outside\\
\hline
Distance to circle [m] & 6.0 & 6.0 & 10.6\\
\hline

\end{tabular}
\end{center}
\end{table}

\begin{figure}[!h]
  \begin{center}
    \includegraphics[height=9.9cm]{images/circle1.pdf}
  \end{center}
  \caption{Experimental result of the first circular algorithm}
  \label{fig.circ1}
\end{figure}

\begin{figure}[!h]
  \begin{center}
    \includegraphics[height=9.9cm]{images/circle2.pdf}
  \end{center}
  \caption{Experimental result of the second circular algorithm}
  \label{fig.circ2}
\end{figure}

A setup for the third circular algorithm is divided into Tab.~\ref{tab.circ3circ} and Tab.~\ref{tab.circ3src}. The experimental result is shown in Fig.~\ref{fig.circ3}. The robot explores sequentially circles $c_3$, $c_2$ and $c_1$, and for each source the direction is estimated. Then the source $S_1$ is approached along directional line and is closer explored by using the loop. Source $S_2$ is explored in the same way. Source $S_3$ is passed, and its position is evaluated within half a meter distance from the circle, therefore the loop may be skipped and the experiment is completed.

\begin{table}[ht]
\catcode`\-=12

\begin{center}
\caption{Setup of circles for the third circular algorithm experiment}
\label{tab.circ3circ}

\begin{tabular}{|p{5cm}|l|l|l|}

\hline
& $c_1$ & $c_2$ & $c_3$\\
\hline
Position of center (relative to BS) [m] & \vect{31.7,-9.7} & \vect{40.3,-19.1} & \vect{46.2,-28.7}\\
\hline
Radius [m] & 3.5 & 4.0 & 4.0\\
\hline

\end{tabular}
\end{center}
\end{table}

A setup for the fourth circular algorithm is divided into Tab.\ref{tab.circ4circ} where six circles are defined, and Tab.~\ref{tab.circ3src} including description of the sources. All the circles are explored first, and then intersections of directional rays are evaluated. First experimental result is shown in Fig.~\ref{fig.circ4i}. Estimations $E_2$ and $E_3$ are quire precise, but the error of estimation $E_1$ is approximately 2 meters. It is caused by wrong estimation of direction from data on circle $c_2$, probably due to their shift caused by a nearer source $S_2$. The sources were not localized precisely because the experiment was interrupted by software malfunction.

The second experimental result is shown in Fig.~\ref{fig.circ4ii}. In order to keep the figure well-arranged, only the last circle is displayed followed by approach to detected sources. One false positive detection of a non-existing fourth source marked as $E_4$ can be seen in the figure. Looping around sources has been disabled by configuration in this experiment. The purpose is to test if sources can be localized precisely enough purely by updating direction. The result shows that in case of the source $S_3$ it would be better to do a loop. Since the false estimation $E_4$ lies between two true sources, the algorithm is actually not much affected by it. The source $S_1$ was not found due to poor initial estimation and low value of parameter $\Delta\phi$. There was not enough space to alter direction sufficiently.

\clearpage

\begin{table}[p]
\catcode`\-=12

\begin{center}
\caption{Setup of soruces for the third circular algorithm experiment}
\label{tab.circ3src}

\begin{tabular}{|p{5cm}|l|l|l|}

\hline
& $S_1$ & $S_2$ & $S_3$\\
\hline
Radionuclide & $^{60}$Co & $^{137}$Cs & $^{60}$Co\\
\hline
Activity & 40.0 MBq & 65.3 MBq & 7.8 MBq\\
\hline
Position (relative to BS) [m] & \vect{30.8,-9.4} & \vect{38.0,-18.7} & \vect{47.4,-24.6}\\
\hline
Nearest circle & $c_1$ & $c_2$ & $c_3$\\
\hline
Position (relative to the nearest circle) & Inside & Inside & Outside\\
\hline
Distance to the nearest circle [m] & 2.5 & 1.7 & 0.3\\
\hline

\end{tabular}
\end{center}
\end{table}

\begin{table}[p]
\catcode`\-=12

\begin{center}
\caption{Setup of circles for the fourth circular algorithm experiment}
\label{tab.circ4circ}

\begin{tabular}{|p{5cm}|l|l|l|}

\hline
& $c_1$ & $c_2$ & $c_3$\\
\hline
Position of center (relative to BS) [m] & \vect{31.4,-8.6} & \vect{34.6,-16.6} & \vect{40.9,-15.7}\\
\hline
Radius [m] & 3 & 2 & 3\\
\hline
\hline
& $c_4$ & $c_5$ & $c_6$\\
\hline
Position of center (relative to BS) [m] & \vect{39.3,-24.8} & \vect{47.8,-25.1} & \vect{45.0,-33.2}\\
\hline
Radius [m] & 2 & 3 & 2\\
\hline

\end{tabular}
\end{center}
\end{table}

\begin{table}[p]
\catcode`\-=12

\begin{center}
\caption{Setup of soruces for the fourth circular algorithm experiment}
\label{tab.circ4src}

\begin{tabular}{|p{5cm}|l|l|l|}

\hline
& $S_1$ & $S_2$ & $S_3$\\
\hline
Radionuclide & $^{60}$Co & $^{137}$Cs & $^{60}$Co\\
\hline
Activity & 39.3 MBq & 65.1 MBq & 7.7 MBq\\
\hline
Position (relative to BS) [m] & \vect{30.8,-9.4} & \vect{38.0,-18.7} & \vect{47.4,-24.6}\\
\hline
Nearest circle & $c_1$ & $c_3$ & $c_5$\\
\hline
Position (relative to the nearest circle) & Inside & Outside & Inside\\
\hline
Distance to the nearest circle [m] & 2.0 & 1.2 & 2.4\\
\hline

\end{tabular}
\end{center}
\end{table}

\clearpage

\begin{figure}[!h]
  \begin{center}
    \includegraphics[height=9.9cm]{images/circle3.pdf}
  \end{center}
  \caption{Experimental result of the third circular algorithm}
  \label{fig.circ3}
\end{figure}

\begin{figure}[!h]
  \begin{center}
    \includegraphics[height=9.9cm]{images/circle4_1.pdf}
  \end{center}
  \caption{Experimental result of the fourth circular algorithm I}
  \label{fig.circ4i}
\end{figure}

\begin{figure}[!h]
  \begin{center}
    \includegraphics[height=9.9cm]{images/circle4_2.pdf}
  \end{center}
  \caption{Experimental result of the fourth circular algorithm II}
  \label{fig.circ4ii}
\end{figure}

\afterpage{\clearpage}

\section{Comparison of the algorithms}

The algorithms are mutually compared regarding time efficiency and precision of localization in Tab.~\ref{tab.comp}. Beside the total time, average time needed for localizing one source is computed, and also how much time is needed to get initial estimation of the direction or location (it is equal to the time before the first update of trajectory). The localization error is a simple distance between a reference position and interpolated position. In case of algorithms where more sources are localized, a mean value is used.

In case of mapping, only total time is mentioned because the trajectory is not updated during the experiment, and therefore the needed time does not depend on a number of sources. In both experiments with strong source search algorithm, two lines were passed before the trajectory could be updated, and it took almost two minutes. Total time does not differ much – time needed for the final loop is almost constant, therefore the time depends mostly on length of the perpendicular line (assuming the same number of lines). Localization error was as good as in the mapping experiment in one case and worse by an order in the other case. It implies strong dependency of the error on accuracy of the initial estimation of the direction.

The first and the second circular algorithms provide a very similar result. They need virtually the same time per source, and the localization error is in same order. The third circular algorithm is approximately by 40 \% slower with similar localization error, but it covers the whole area of interest. Such increase of the needed time is a reasonable price for the coverage. The fourth circular algorithm requires more time to perform a basic exploration. Although the sum of circles’ circumference is lower than in case of the third algorithm, time costs bonded with movement between the individual circles are higher. The localization error is not evaluated because no fully successful experiment has been realized yet.

Usage of the direction-sensitive detection systems seems promising in terms of localization of the lost radiation sources. All proposed algorithms that use dynamic update of trajectory provide better time efficiency than simple mapping. In comparison to the strong source search algorithm (which does not work with directional sensitivity), the circular algorithms based on estimation of direction have stable results for different type and number of sources. The circular algorithm based on estimation of location should provide the best time efficiency for higher numbers of sources, because it has the highest initial time costs which would pay back in situations where the previous algorithms probably fail due to their design. For example, if there was  a source detectable from two circles in case of the third circular algorithm. However, it is currently a rather theoretical assumption than a proved fact. The algorithm has to be improved and more experiments need to be realized.

\begin{table}[ht]
\catcode`\-=12

\begin{center}
\caption{Comparison of the algorithms}
\label{tab.comp}

\begin{tabular}{|p{3cm}|l|p{1.8cm}|p{3cm}|p{2.1cm}|}

\hline
& Total time & Time per source & Time to first trajectory update & Localization error\\
& [mm:ss] & [mm:ss] & [mm:ss] & [m]\\
\hline
Mapping & 15:03 & – & – & 0.06\\
\hline
Source search I & 2:47 & 2:47 & 1:54 & 0.94\\
\hline
Source search II & 2:53 & 2:53 & 1:54 & 0.04\\
\hline
1st circ. alg. & 1:28 & 1:28 & 0:48 & 0.52\\
\hline
2nd circ. alg. & 2:54 & 1:27 & 0:48 & 0.40\\
\hline
3rd circ. alg. & 6:06 & 2:02 & 3:17 & 0.36\\
\hline
4th circ. alg. & 5:21 & 1:47 & 4:04 & –\\
\hline

\end{tabular}
\end{center}
\end{table}