GREGOROVÁ, J. Útoky na algoritmy AI a obrany proti nim [online]. Brno: Vysoké učení technické v Brně. Fakulta informačních technologií. 2024.
Řešitelka nastudovala opravdu velké množství vysoce odborných pramenů a sepsala velice podrobnou technickou zprávu. Realizační výstup by mohl být obsáhlejší a dojít k ambicióznějším zjištěním a výsledkům.
| Kritérium | Známka | Body | Slovní hodnocení |
|---|---|---|---|
| Informace k zadání | Zadání práce bylo iniciováno řešitelkou, protože se o problematiku zajímá z etických důvodů. Při řešení byla motivovaná problematiku opravdu široce nastudovat a do hloubky pochopit, což je patrné i z koncepce a rozsahu technické zprávy. | ||
| Práce s literaturou | Řešitelka prostudovala a do svého textu přetavila opravdu enormní množství vysoce odborných zdrojů. Je patrné, že studium a třídění odborných informací je řešitelce vlastní a zajímá ji. | ||
| Aktivita během řešení, konzultace, komunikace | Řešitelka věnovala opravdu velké množství času studiu pramenů a sepsání zjištěných a utříděných poznatků do textové zprávy. Během řešení diplomové práce docházela na domluvené konzultace. | ||
| Aktivita při dokončování | Práce byla dokončena včas a opakovaně konzultována. Rozsah a odbornost technické zprávy (sepsané v angličtině) je nadstandardní a technická zpráva si vyžádala velké množství úsilí. | ||
| Publikační činnost, ocenění | N/A |
I evaluate the thesis positively; if I am not considering the missing experiments in adversarial attack prevention or detection. The positives are mainly the state-of-the-art review and experiments with adversarial attacks. The given claims may be used to guide future research. Overall, the thesis is above average, and I suggest a grade of B.
| Kritérium | Známka | Body | Slovní hodnocení |
|---|---|---|---|
| Náročnost zadání | The assignment requires extensive knowledge of machine learning, computer vision, adversarial attacks, and an overview of attack defense approaches. Thus, I would consider the topic research-focused and more complex than average. | ||
| Rozsah splnění požadavků zadání | The solution primarily involves conducting experiments with multiple models for adversarial attacks. The following point of the assignment appears to be incomplete: Experiment with methods of defense against various adversarial attacks; showcase the possibilities. The incomplete nature of the assignment seems to be partially justified, as the student is experimenting and searching for patterns to use in future research. However, the student needs to explain this to the commission more. | ||
| Rozsah technické zprávy | |||
| Prezentační úroveň technické zprávy | 75 | The thesis has a logical structure and is readable with logically ordered chapters. The text itself contains certain technical inaccuracies. Examples from section 2.1 include: In the training phase, the system experiences (2.1.1) Once the training error is sufficiently low, it is necessary to reduce generalization error. (2.1.1) Another minor problem is the lack of references to certain figures (Figure 3.1, Figure 3.2). The caption is descriptive, but it is unclear which part of the surrounding text these figures are relevant to. | |
| Formální úprava technické zprávy | 95 | From a typographical point of view, I consider the thesis high quality. The thesis is written in high-level technical English and with high typographic standards. It contains barely any grammatical mistakes. | |
| Práce s literaturou | 95 | Overall, the thesis contains 126 references, which is quite a lot. The references clearly show that the student researched the topic thoroughly, and all are good quality. The student cites mainly peer-reviewed papers published at conferences or journals. Some references are also to published books. The usage of these sources is good, and all significant claims in the theoretical part of the thesis are backed by citations. The references are used thoroughly throughout the thesis. | |
| Realizační výstup | 75 | The student supplied a single executable script containing documentation and a guide to set and run the script in a specified environment. The script is configurable and mainly used to evaluate how the different attacks are perceived by a human running the experiments. Presented claims, however, seem more subjective. See more information on the usability of the results. | |
| Využitelnost výsledků | The thesis itself is a compilation of multiple sources. It contains a good review of the state-of-the-art in adversarial attacks and adversarial attack prevention, which is a good basis for future research. Similarly, the supplied script for adversarial attacks can be used in the future mainly due to its simple and documented interface. The overall evaluation includes model classification performance degradation on adversarial samples. The thesis claims that neural networks perceive certain patterns that lead to misclassification. The major missing link is the quantification of claimed patterns eighter by other experiments or their consistency. From this point, the claim seems to be rather subjective. The idea, however, might be used in future work. |
eVSKP id 156862