SEDLÁČEK, Š. Aligning pre-trained models for spoken language translation [online]. Brno: Vysoké učení technické v Brně. Fakulta informačních technologií. 2024.
- The overall work is excellent in terms of experiments, analysis, implementation. - The findings from the thesis are found to be very useful, and we are planning to use the ideas and source code for other research works. - The student is planning to continue the work after the thesis.
| Kritérium | Známka | Body | Slovní hodnocení |
|---|---|---|---|
| Informace k zadání | - The diploma thesis is a challenging research topic, that is also very relevant to the current advancements in deep learning / artificial intelligence for speech and language applications. - The thesis work infact exceeded the specified requirements, especially the point number (5). | ||
| Práce s literaturou | - The student has shown sufficient understanding of the relevant literature, including the recent ones, that is required to carry out the assignment. | ||
| Aktivita během řešení, konzultace, komunikace | - The student was consulting regularly during the whole period of the work. Some consultations were online, while some were direct. | ||
| Aktivita při dokončování | - The work was completed in advance, and has allowed sufficient time to review the final text of the thesis. | ||
| Publikační činnost, ocenění | - The student is planning to submit an article by June 15, 2024 to ACL Rolling Reviewing, aiming at EMNLP'2024 conference. |
This is an excellent piece of very relevant, well-executed experimental science. The student has demonstrated that he can grasp cutting-edge research, design and conduct valuable experiments based on it and also propose effective extensions to the State-of-the-Art -- again evaluating them in a carefully thought-through set of experiments. I recommend this diploma project to represent BUT FIT in competitions such as IT SPY.
| Kritérium | Známka | Body | Slovní hodnocení |
|---|---|---|---|
| Náročnost zadání | This assignment requires the student to acquire deep knowledge of cutting-edge research in spoken language translation. The student did not attempt to find workarounds and faced the assignment in its full depth. | ||
| Rozsah splnění požadavků zadání | On top of fulfilling the requirements, the student developed an elaborate pretraining scheme utilizing single-language data. | ||
| Rozsah technické zprávy | The extent of experimental work done by the student is inspiring and correspondingly, the thesis is packed with exciting findings and their critical discussion. Thanks to a concise style of writing, the text is still of manageable length. | ||
| Prezentační úroveň technické zprávy | 90 | First of all, the Introduction alone is outstanding enough to overshadow any writing-related weaknesess mentioned later in this review. Sections and even paragraphs of the background chapters 2 and 3 are of varied quality. On the high level: the architecture of Transformers is described in a very clear journal-level style, but the description of training procedure and inference-time deployment suffer badly from not being described formally using the corresponding objective functions. Moving through the State-of-the-Art chapter 4, the clarity of presentation stabilizes and finally, the experimental chapters 5 and 6 are the jewels of the thesis -- as is appropriate for this kind of diploma project. The design of the experiments is well motivated and unambigiously described and their results are presented clearly. Then, not only solid conclusions are drawn, but raised questions are answered by the subsequent experiments, making for a surprisingly complete picture of the behaviour of the tested methods. | |
| Formální úprava technické zprávy | 90 | The thesis is written in good English with occassional mistakes that do not disturb reading. Very uncommon for a diploma project, author plural is used consistently throughout the text, but it does not occlude the authorship itself and is in fact fitting for the scientific genre where this project belongs. Typographically, the thesis is a solid output from TeX, with correct dashes, aesthetically pleasing equations etc. I personally appreciate the care that was evidently invested into proper treatement of tables, which not only present the data in a useful and concise manner, but also come properly formatted: with captions above them, without useless vertical rules and with horizontal rules correctly vertically spaced. | |
| Práce s literaturou | 95 | The student has familiriazed himself with an impressive body of knowledge; the distinction between the previously known and original contributions is clear. The only weakness in working with literature is the excessive use of citations as part of sentence (e.g. "approaches presented in [50, 17]"). | |
| Realizační výstup | 90 | The code supporting the experiments is well structured and easy to follow. | |
| Využitelnost výsledků | The research done in this diploma project is worth at least a Core A publication. |
eVSKP id 157031