Investigation into the reliability of facial recognition systems under the simultaneous influences of mood variation and makeup

Facial recognition systems are increasingly popular and prevalent in our everyday lives, especially on mobile cell phones. This paper is an attempt to investigate the effects that makeup and facial expressions have on the reliability of such systems. While these factors have been shown not to be significant by themselves, it has not been clearly demonstrated whether a combination of both these factors can affect the matching accuracy of the same system in a statistically meaningful way. We carried out numerical experiments through two databases: Radboud Faces database and Psychological Image Collection at Stirling (PICS), using a state of the art algorithm, namely dlib. Then, in order to be able to reliably validate the results, we used two more algorithms (Verilook and VGGFace) to give similarity scores. The results showed that, while the effects of makeup and varied mood expressions are not significant by themselves, the joint effect is. An equal error rate (EER) of 4.68% was achieved when identifying faces under the joint influences of full makeup and mood variation, while the EER under the effect of each of these factors separately is less than 1%.