Face recognition seemed to be a trend in science fiction movies not so long ago, and now we can experience it almost every day. The number of advantages it offers to its users is surprising, putting this technology in the position of one of the most important security measures. Yet, as it is still only technology, it may be prone to mistakes, which in some cases may lead to serious security breaches. How do you evaluate the accuracy of face recognition algorithms?

Testing dataset

Face recognition methods are mostly based on machine learning or deep learning, based on large datasets of labeled images. The performance of the algorithms is greatly relevant to the quality and nature of the dataset. That leads to a conclusion that the source data provided is the highest quality, the software will work most efficiently finding the best matches. 

In order to test algorithms for face recognition, their accuracy and reliability, it’s necessary to test it on a hold-out dataset. It’s recommended to use a testing dataset, similar to the data which needs to be processed, to achieve most reliable results. To create similar environments, the camera type, filming conditions, age, and gender of the individuals appearing in the test dataset need to be replicated.

Face Recognition Vendor Test (FRVT)

The National Institute of Standards and Technology (NIST) conducts ongoing evaluations to assess the performance of face recognition algorithms developed by various vendors. The test called Face Recognition Vendor Test aims at providing an independent and objective evaluation of the capabilities of different face recognition systems. It is capable of assessing the accuracy and effectiveness of the systems in various scenarios. 

FRVT has a number of advantages, providing thorough analysis. Objective evaluation focuses on performance of face recognition algorithms, their accuracy, speed, and reliability. The evaluations are conducted regularly, allowing vendors to submit their algorithms for testing against specified datasets and performance metrics.

The testing covers a range of testing scenarios to fit real-world situations. It focuses on variations in lighting, pose, age, and other environmental factors. FRVT evaluation participants use the results to present the strengths of their algorithms. 

Overfitting

Machine learning and computer vision related to face recognition have one disadvantage, which is the tendency to overfitting. The system deals well with already used databases but may fail unexpectedly with new data. The already trained algorithm grants access to present employees, but stops those who have just been employed and introduced into the database. 

To avoid the situation it’s recommended to have a reserve testing dataset, or the public dataset which the vendor didn’t use during testing and calibration. It’s better to choose the database not present on the ranking list, and choose a similar one to the dataset that is going to be introduced into the software. 

Read more about algorithm accuracy of facial recognition technology to ensure accuracy and reliability of the system.