WASHINGTON -- The federal government is making progress on developing a surveillance system that would pair computers with video cameras to scan crowds and automatically identify people by their faces, according to newly disclosed documents and interviews with researchers working on the project.
The Department of Homeland Security recently tested a crowd-scanning project called the Biometric Optical Surveillance System -- or BOSS -- after two years of government-financed development. Although the system is not ready for use, researchers say they are making significant advances on it. That alarms privacy advocates, who say now is the time for the government to establish oversight rules and limits on how it will someday be used.
There have been stabs for more than a decade at building a system that would help match faces in a crowd with names on a watch list -- whether in searching for terrorism suspects at high-profile events such as a presidential inaugural parade, looking for criminal fugitives in places like Times Square or identifying card cheats in crowded casinos.
The automated matching of close-up photographs has improved greatly in recent years, and companies including Facebook have experimented with using it on still pictures.
But even with advances in computer processing power, the technical hurdles involving crowd scans from a distance have proved to be far more challenging. Despite occasional much-hyped tests, including as far back as the 2001 Super Bowl, technical specialists say crowd scanning is still too slow and unreliable.
The release of the documents about the government's efforts to overcome those challenges comes amid a surge of interest in surveillance matters inspired by the leaks of former National Security Agency contractor Edward J. Snowden. Interest in video surveillance has also been fueled by the Boston Marathon bomb attack, in which the bombers were identified by officials looking through camera footage.
In a sign of how use of such technologies can be developed for one use, but then expanded to another, the BOSS research began as an effort to help the military detect potential suicide bombers and other terrorists overseas at "outdoor polling places in Afghanistan and Iraq," among other sites, the documents show. But in 2010, the effort was transferred to the Department of Homeland Security, to be developed for use instead by police in the United States.
After a recent test of the system, the department recommended against deploying it until more improvements could be made.
"I would say we're at least five years off, but it all depends on what kind of goals they have in mind" for such a system, said Anil Jain, a Michigan State University computer vision and biometrics engineering specialist who was not involved in the BOSS project.
The effort to build the BOSS system involved a two-year, $5.2 million federal contract given to Electronic Warfare Associates, a Washington-area defense contractor with a branch office in Kentucky. The company has been working with the laboratory of University of Louisville computer vision specialist Aly Farag, and the contract was steered to the firm by an earmark request in a 2010 appropriations bill by Senate Minority Leader Mitch McConnell, R-Ky.
Significant progress is already being made in automated face recognition, using photographs taken under ideal conditions, such as passport pictures and mug shots. The FBI is spending $1 billion to roll out a "Next Generation Identification" system that will provide a national mug shot database to help local police verify identities.
But surveillance of crowds from a distance -- in which lighting and shadows vary, and faces tend to be partially obscured or pointed in random directions -- is still not reliable or fast enough. The BOSS research is intended to overcome those challenges by generating far more information for computers to analyze.
The system consists of two towers bearing "robotic camera structures" with infrared and distance sensors. They take pictures of the same subject from slightly different angles. A computer then processes the images into a "3-D signature" built from data such as the ratios between various points on someone's face, to be compared against data about faces stored in a watch-list database, the documents show.
The Homeland Security Department hired the Pacific Northwest National Laboratory to test the BOSS system at an arena in Kennewick, Wash. The plan, according to a "privacy impact assessment," was to use 30 volunteers whose facial data would be mingled in a database among 1,000 mug shots to see whether the system could reliably recognize when any of the volunteers were present.
Ed Tivol of Electronic Warfare Associates said the goal was to provide a match with an 80 to 90 percent certainty from a range of as far as 100 meters, just more than a football field, something "that has never been done."