Synergy of Eye and Hand: Multimodal Biomarkers for Cognitive Assessment

Abstract

Traditional handwriting analysis for neurological assessment captures motor output but largely misses the guiding cognitive processes like visuospatial planning and attention. This study introduces a multimodal approach, combining online handwriting kinematics with concurrent eye-tracking data from 48 older adults performing the Pentagon Copy Test (PCT). We extracted novel feature sets, including Hand-Eye Coupling (HEC) and Fractional Derivative (FD) biomarkers, and used an XGBoost classifier with Recursive Feature Elimination (RFE) to predict a binarized PCT performance score. Our final model, integrating all features, achieved a Balanced Accuracy (BACC) of 90%, significantly outperforming a model trained on baseline features alone (79% BACC). The findings demonstrate that integrating eye-tracking data with advanced handwriting analysis provides a powerful and holistic tool for objectively assessing cognitive-motor performance, highlighting its potential as a sensitive digital biomarker.Traditional handwriting analysis for neurological assessment captures motor output but largely misses the guiding cognitive processes like visuospatial planning and attention. This study introduces a multimodal approach, combining online handwriting kinematics with concurrent eye-tracking data from 48 older adults performing the Pentagon Copy Test (PCT). We extracted novel feature sets, including Hand-Eye Coupling (HEC) and Fractional Derivative (FD) biomarkers, and used an XGBoost classifier with Recursive Feature Elimination (RFE) to predict a binarized PCT performance score. Our final model, integrating all features, achieved a Balanced Accuracy (BACC) of 90%, significantly outperforming a model trained on baseline features alone (79% BACC). The findings demonstrate that integrating eye-tracking data with advanced handwriting analysis provides a powerful and holistic tool for objectively assessing cognitive-motor performance, highlighting its potential as a sensitive digital biomarker.