BRAVENEC, T. Počítačové vidění a detekce gest rukou a prstů [online]. Brno: Vysoké učení technické v Brně. Fakulta elektrotechniky a komunikačních technologií. 2019.
During the summer semester, student worked on computer vision issues, namely on hands modeling and recognizing of individual gestures in a video sequence. The presented work corresponds to the requirements, is comprehensible and at a high formal level. The student has designed and developed a multi-platform software tool to describe and subsequently detect basic hand gestures. As required, the system can be easily extended with “any” static gesture. (Note, the system is available in the online repository at https://gitlab.com/tbravenec). I appreciate the student's very active approach during the semester, his ability to use available tools and projects, his frequent consultations, where he presented partial results, studied materials and suggestions on how to proceed. The software system is functional, although it would be advisable to perform more tests and focused on further optimizing of computational complexity. In the summer semester, the student took part in the EEICT 2019 competition, organized by the Faculty of Electrical Engineering and Communication, Brno University of Technology, where he took 3rd place in the Master section of “Signal Processing, Image and Data Processing”. Currently (May 2019), the core of the thesis is being reviewed at the international conference "13th International Conference on Parallel Processing and Applied Mathematics", indexed in Web of Science and Scopus databases and the student is enrolled in full-time doctoral studies at the Department of Radioelectronics, FEEC, BUT.
The goal of the thesis was to study and analyze possible approaches to hand detection, gesture recognition and finger tracking, and to create a multiplatform application capable of processing images and video sequences. Based on the analysis of pros and cons of different approaches, the Author selected the deep learning method as the most suitable. This choice should be assesed very positively. It corresponds to the modern trend that uses machine learning/AI methods to extend the functionality and improve the reliability of computer vision. The implementation of the imaging system is based on using two neural networks: (i) YOLOv2 – a fully convolutional naural network dedicated to hand detection, and (ii) neural network from the project OpnePose dedicated to finger detection. Because neural networks need a lot of training data to produce practicable results, the Author contributed a lot of effort to create adequate training datasets by combining existing datasets and recording own data, as well as getting videos from Internet. This is a valuable part of the thesis. At the same time, to ensure an ease of use, a graphical user interface was developed using the open-source cross-platform Python library - Kivy, with its KV proprietary language. To provide the platform independency when implementing the application functionality on different hardware/software configurations containing both CPUs and highly parallel GPUs, the Author applied two matured libraries – PyTorch and OpenCV. In my opinion, these choices and their practical usage are justified and demonstrate good professional skills and knowledge of the Author. Finally, the developed system is extensively evaluated and tested, including dynamic video sequences with difficult to detect movements, such as rapper gestures. What is important, modern NVIDIA graphic cards are applied for the neural network inference to speed up the deep learning process. Concerning correctness of detection results, they can be considered as generally satisfying. The thesis is concluded with presenting the possibilities for expanding the computational backend of the application to further improve detection results and extend functionality. Summarizing, my general evaluation of this thesis is highly positive. It confirms not only good professional skills and knowledge of the Author, but also his high capabilities for research activity.
eVSKP id 118437