Face recognition is the automated process of comparing two images of faces to determine whether they represent the same individual.
Before face recognition can identify someone, an algorithm must first find that person’s face within the photo. This is called face detection. Once detected, a face is “normalized”—scaled, rotated, and aligned so that every face that the algorithm processes is in the same position. This makes it easier to compare the faces. Next, the algorithm extracts features from the face—characteristics that can be numerically quantified, like eye position or skin texture. Finally, the algorithm examines pairs of faces and issues a numerical score reflecting the similarity of their features.
Face recognition is inherently probabilistic: It does not produce binary “yes” or “no” answers, but rather identifies more likely or less likely matches.8 Most police face recognition systems will output either the top few most similar photos or all photos above a certain similarity threshold. Law enforcement agencies call these photos “candidates” for further investigation.
Some facial features may be better indicators of similarity than others. Many face recognition algorithms figure out which features matter most through training. During training, an algorithm is given pairs of face images of the same person. Over time, the algorithm learns to pay more attention to the features that were the most reliable signals that the two images contained the same person.
- 8. See generally Joseph N. Pato and Lynette I. Millett, eds., Biometric Recognition: Challenges and Opportunities, 36–45 (National Academies Press 2010) (hereinafter “Pato Report”).