A Novel Gesture-Based Control System for Fluorescence Volumetric Data in Virtual Reality

Abstract

With the development of light microscopy, it is becoming increasingly easy to obtain detailed multicolor fluorescence volumetric data. The need for their appropriate visualization has become an integral part of fluorescence imaging. Virtual reality (VR) technology provides a new way of visualizing multidimensional image data or models so that the entire 3D structure can be intui-tively observed, together with different object features or details on or within the object. With the need for imaging advanced volumetric data, the demands on the control of virtual object prop-erties are increasing, it happens especially for multicolor objects obtained by fluorescent mi-croscopy. Existing solutions with universal VR controllers or software-based controllers with the need to define sufficient space for the user to manipulate data in VR are not usable in many practical applications. Therefore, we developed a custom gesture-based VR control system with a custom controller connected with the FluoRender visualization environment. The multi-touch sensor disc was used for this purpose. Our control system may be a good choice for easier and more com-fortable manipulation of virtual objects and their properties, especially using confocal microscopy, which is the most widely used technique for acquiring volumetric fluorescence data so far.With the development of light microscopy, it is becoming increasingly easy to obtain detailed multicolor fluorescence volumetric data. The need for their appropriate visualization has become an integral part of fluorescence imaging. Virtual reality (VR) technology provides a new way of visualizing multidimensional image data or models so that the entire 3D structure can be intui-tively observed, together with different object features or details on or within the object. With the need for imaging advanced volumetric data, the demands on the control of virtual object prop-erties are increasing, it happens especially for multicolor objects obtained by fluorescent mi-croscopy. Existing solutions with universal VR controllers or software-based controllers with the need to define sufficient space for the user to manipulate data in VR are not usable in many practical applications. Therefore, we developed a custom gesture-based VR control system with a custom controller connected with the FluoRender visualization environment. The multi-touch sensor disc was used for this purpose. Our control system may be a good choice for easier and more com-fortable manipulation of virtual objects and their properties, especially using confocal microscopy, which is the most widely used technique for acquiring volumetric fluorescence data so far.