CNN for Hand Washing Movement Classification: What Matters More - the Approach or the Dataset?

Good hand hygiene is one of the key factors in preventing infectious diseases, including COVID-19. Advances in machine learning have enabled automated hand hygiene evaluation, with research papers reporting highly accurate hand washing movement classification from video data. However, existing studies typically use datasets collected in lab conditions. In this paper, we apply state-of-the-art techniques such as MobileNetV2 based CNN, including two-stream and recurrent CNN, to three different datasets: a good-quality and uniform lab-based dataset, a more diverse lab-based dataset, and a large-scale real-life dataset collected in a hospital. The results show that while many of the approaches show good accuracy on the first dataset, the accuracy drops significantly on the more complex datasets. Moreover, all approaches fail to generalize on the third dataset, and only show slightly-better-than random accuracy on videos held out from the training set. This suggests that despite the high accuracy routinely reported in the research literature, the transition to real-world applications for hand washing quality monitoring is not going to be straightforward.