
Algorithmic Similarity and the Limits of Proof: Facial Recognition
The spread of automated facial recognition systems in law-enforcement practice is usually discussed as a question of accuracy: how often the algorithm errs, whether it errs equally across demographic groups, and whether…
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
The spread of automated facial recognition systems in law-enforcement practice is usually discussed as a question of accuracy: how often the algorithm errs, whether it errs equally across demographic groups, and whether these errors can be reduced through better training data or stricter matching thresholds. Such a framing is necessary, but it overlooks a deeper change that the technology introduces into the relationship between knowledge, suspicion, and power. The cases of wrongful detention that have occurred in recent years in the United Kingdom and the United States show that harm arises not only from an incorrect computation. It is connected above all with how institutions receive, transmit, and act upon a probabilistic result, and with whom they make bear the consequences of the uncertainty embedded in it.
Two cases may serve as examples. In the first, a young engineer was detained at his home in Southampton on suspicion of a theft committed in a city he had never visited. He was held in custody for almost ten hours, although, by his account, the police officers, having compared the images, admitted that he did not resemble the suspect. In the second, a fifty-year-old woman from Tennessee was arrested on a warrant from North Dakota, a state she had never been to. She spent more than five months in jail, and in that time lost her home, her car, and her dog. The charges were dropped only after the defense presented bank records confirming that during the period of the crimes she had been hundreds of miles from the places where they were committed. The cases differ in legal system and circumstances, but they are built the same way: a claim that two faces resemble each other with a certain degree of confidence was taken for the establishment of identity.
The essay examines in turn the mechanisms through which such a substitution takes place. Among them are the transformation of a supposition into an administrative fact, the coincidence of perceptual and machine bias, the statistical nature of false matches, the role of the composition of the search database, the de facto shift of the burden of proof, the temporal asymmetry between accusation and rehabilitation, and the limits of so-called human oversight. The conclusion outlines the contours of a normative framework that could correspond to the character of these risks.
To see the fundamental change, it is convenient to compare two situations with identical content. In the first, a police officer looks at a blurry frame from a surveillance camera and says that the person captured resembles a certain individual. Colleagues and superiors take this as a subjective opinion that can be contested, rejected, or checked by other means. In the second situation, a program returns a name and a numerical similarity score. In substance the claim remains the same, since it is still a supposition that two images look alike. Its institutional status, however, changes. The result of a technical procedure is entered into a report, passed to another unit, and used as the basis for the next decision, while its probabilistic character gradually ceases to be noticed.
This process can be called the laundering of uncertainty. Probabilistic similarity, that is, a mathematical category, enters the bureaucratic chain as a hypothesis and leaves it as grounds for detention, that is, as a legal category. Between these states no new evidence necessarily appears, although it is precisely such an intermediate link that should determine the lawfulness of the transition from suspicion to coercion. At the same time, the algorithm need not be especially accurate to acquire such power. It is enough that institutions regard its output as more objective than human judgment, and therefore each subsequent participant in the chain works no longer with a hypothesis but with a fact supposedly established before them. Formally, no one makes the decision to arrest. It is assembled from a sequence of procedural steps, and responsibility is distributed among them in such a way that it is hard to attribute to any particular person. In this sense one can speak of a kind of synthetic probable cause. The algorithm does not in fact provide such grounds, but bureaucratic practice begins to treat the number as sufficient.
The uneven distribution of errors across demographic groups cannot be considered a rumor or a journalistic exaggeration: it has been measured repeatedly. A large-scale study by the U.S. National Institute of Standards and Technology (NIST), covering almost two hundred algorithms from almost a hundred developers, revealed marked differences in the rate of false matches between groups. An independent evaluation carried out for Essex Police jointly with researchers at the University of Cambridge showed that the live recognition system flagged Black people considerably more often than members of other groups. After that, use of the technology was suspended. Nevertheless, the demographic skew of machine systems is useful to consider together with long-known features of human perception, since it is their combination that determines what happens in actual practice.
Social psychology has long described the other-race effect (own-race bias): people distinguish the faces of their own group better than the faces of other groups. Closely related to it is the outgroup homogeneity effect, that is, the tendency to perceive members of another community as more similar to one another than they are. An outside observer often confuses people from ethnically or culturally close groups who, to their own members, are distinguished by a multitude of small and instantly recognizable features. It is telling that passive familiarity with other people’s faces through film, television, and video platforms helps hardly at all. In face recognition one can distinguish at least two modes. Categorization assigns a face to a broad group, while individuation distinguishes one member of the group from another. Mediated familiarity trains the first mode and consolidates a generalized template, whereas the skill of individuation is usually formed in direct and practically significant contact, when social interaction depends on telling specific people apart.
Machine systems can reproduce a similar asymmetry, although the mechanisms require careful description. Modern algorithms, as a rule, do not compare images pixel by pixel. They align the face by a set of landmarks and then, by means of a neural network, convert it into a feature vector whose position in a multidimensional space determines the similarity score. If the training data disproportionately represented some groups and insufficiently represented others, the model may distinguish people less well within the underrepresented groups, and with a common matching threshold for all groups this yields an unequal rate of false matches. At the same time, it would be wrong to claim that all modern systems necessarily reduce faces to a single Caucasian template or universally compress non-European faces into a narrow region of the feature space. The NIST results record demographic differences in errors, but they also show considerable differences between the algorithms of different developers, and therefore they do not confirm the existence of a single universal mechanism. The skew may arise from different sources: the composition of the training data, the distribution of similarity scores, image quality, the chosen threshold, and the composition of the database in which the search is conducted. What remains common to all these variants is that similarity is determined by the design of the measuring system and not by any absolute characteristic of the face.
If perceptual and machine bias are superimposed on one another, their effect is more likely multiplied than added. An operator who already finds it hard to tell two faces from another group apart receives from the system a confident numerical score. His own uncertainty, which under other conditions could have become grounds for doubt, no longer restrains the decision. Instead, automation bias arises, that is, the tendency to regard the machine’s conclusion as more objective than one’s own judgment. When a human template formed by mediated familiarity coincides with a machine template created by unbalanced data, this coincidence is mistakenly perceived as independent confirmation, although in fact these are two versions of one and the same systematic error.
This problem has a reverse side: unjustified confidence in the uniqueness of biometric features. Here it is important to distinguish uniqueness in nature from uniqueness in measurement. Even if two identical fingerprint patterns do not exist in nature, the system never works with the pattern as a whole. It compares a limited number of features, often extracted from an incomplete or damaged trace, with reference samples in a database. A textbook example is the case of Brandon Mayfield, an American lawyer whom the FBI in 2004 erroneously linked to the Madrid bombings by a fingerprint, although the Spanish police had questioned this match from the very start. The error was not in the feature itself. The system returned a close candidate, and the experts equated closeness with identity. With the iris of the eye such errors happen less often, but there too what is compared are not the biological structures themselves but their encoded templates, the quality of which depends on lighting, the state of the pupil, and the characteristics of the camera. With the face the situation is more complicated than with fingerprints and the iris. The face changes with age, weight, facial hair, lighting, and angle, and therefore has no stable reference and exists for the system only as a set of current projections.
To this is added a statistical problem that manifests itself regardless of the quality of a particular algorithm. In a one-to-many search through a database of millions of images, even a very low rate of false matches per comparison yields a significant absolute number of false candidates. At such a scale a false match is not a malfunction but an expected consequence of normal operation. If, however, every hit is interpreted as an event with a high posterior probability of guilt, this reproduces the classic error of ignoring the base rate, well known from the screening of rare diseases. From this follows a conclusion that at first glance seems paradoxical: the larger the search database, the less evidentiary weight an individual match should carry. The expansion of the database, usually described as a strengthening of the tool, in fact reduces the value of each of its results.
It is telling that judicial practice in a number of jurisdictions is gradually moving away from the language of identification toward the language of compatibility. Instead of asserting that a trace belongs to a particular person, it is increasingly said that the trace is compatible with that person and incompatible with a certain share of the population. Such a formulation is more precise, but it deprives biometric evidence of the appearance of absolute certainty on which its authority largely rested.
One of the least noticeable, but perhaps most significant, problems is connected not with the comparison algorithm but with the population among which the search is conducted. If the database consists of photographs taken during previous arrests, it includes not all citizens but only those who have already come into the view of the police, including those who were never subsequently charged. Consider two people with identical faces and the same probability of resembling the image of a suspect. The first has never interacted with the police; the second was once detained and then released. The first in principle cannot become a candidate in a search; the second can. There are known cases in which a person ended up in the database precisely because of a previous erron
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。