WASHINGTON -- The U.S. government suspects that individuals with connections to al-Qaida and other hostile groups have repeatedly sought to obtain jobs in the intelligence community, and it reinvestigates thousands of employees a year to reduce the threat that one of its own may be trying to compromise closely held secrets, according to a classified budget document.
The CIA found that among a subset of job seekers whose backgrounds raised questions, roughly 1 out of every 5 had "significant terrorist and/or hostile intelligence connections," according to the document, which was provided to The Washington Post by former National Security Agency contractor Edward Snowden.
The groups cited most often were Hamas, Hezbollah, and al-Qaida and its affiliates, but the nature of the connections was not described in the document.
So sharp is the fear of threats from within that last year the NSA planned to launch at least 4,000 probes of potentially suspicious or abnormal staff activity after scrutinizing trillions of employee keystrokes at work. The anomalous behavior that sent up red flags could include staffers downloading multiple documents or accessing classified databases they do not normally use for their work, said two people familiar with the software used to monitor employee activity.
This shrouded, multimillion-dollar hunt for insider threats has suffered from critical delays in recent years and uneven implementation across agencies, the budget records show. And the spy agencies' detection systems never noticed that Mr. Snowden was copying highly classified documents from different parts of the NSA's networks.
He subsequently fled to Hong Kong and then Moscow, where he remains after being granted temporary asylum.
Contractors like Mr. Snowden, an NSA spokeswoman said, were not included in the plans to reinvestigate 4,000 security clearances.
CIA officials said the number of applicants ultimately tied to terrorist networks or hostile foreign governments was "small" but declined to provide an exact number or the reasons the broader group of applicants initially raised concerns.
"Over the last several years, a small subset of CIA's total job applicants were flagged due to various problems or issues," one official said in response to questions. "During this period, 1 in 5 of that small subset were found to have significant connections to hostile intelligence services and or terrorist groups." The official, like others interviewed for this article, spoke on the condition of anonymity to discuss classified material.
The intelligence community's dramatic emphasis on insider threats came in the wake of disclosures by WikiLeaks in 2010. The anti-secrecy group received hundreds of thousands of military and diplomatic documents from Army Pfc. Bradley Manning, now known as Chelsea Manning.
Congress made security a top priority and in 2011 ordered the director of national intelligence, James Clapper, to set up "an effective automated insider threat detection program" to guard against similar security failures. The program was supposed to flag possible abuses, identify double agents and prevent leaks.
The project was delayed several times because the intelligence community was preoccupied with handling the fallout from Manning's leaks, the budget documents show.
Congress gave Mr. Clapper an October 2012 deadline to install the automation system and until October 2013 to have it fully operating. At the Obama administration's request, the deadlines were each pushed back a year.
Steven Aftergood, a government secrecy expert at the Federation of American Scientists, which analyzes national security policy, said he suspects the agency may respond to a lot of "false positives" -- alerts for activity that is actually innocent and work-related.
"If the 4,000 cases turn up only two or three actual threats, they need to adjust their detection threshold or they'll be using a lot of resources for no purpose," he said.
An intelligence official knowledgeable about insider-threat programs said government agencies may need to better calibrate their software and reassess their criteria for what constitutes a realistic or likely threat.
But he added, "Insider-threat detection is an important security tool that needs to be put in place."
NSA spokeswoman Vanee Vines said the investigations are part of an effort to try to reduce risks, not investigate known threats.
President Barack Obama issued a national insider-threat policy in November 2012 that defines the problem as any risk that insiders will use their access to government secrets, knowingly or unknowingly, in a way that hurts U.S. security. The damage can come through "espionage, terrorism, [or] unauthorized disclosure of national security information, or through the loss or degradation of departmental resources or capabilities."
The newly obtained budget records also reveal what some consider a new security risk in the making. The NSA is creating a massive new database, code-named WILDSAGE, to facilitate the use of sensitive intelligence. The system "provides a mechanism for cybersecurity centers to share signatures at the SECRET classification level," the budget document said.
The NSA had not fully implemented WILDSAGE, according to the budget document. But the intelligence community also reported that it "made significant progress in FY 2011 in increasing collaboration" across cybersecurity centers and in creating architecture to share important information on risks."
Such shared databases give government employees information they may need to track and disrupt terrorist or cyber plots, officials said.
The investigation of a 2009 plot by al-Qaida's affiliate in Yemen to bring down a civilian aircraft over Detroit found that government agencies failed to share information that could have flagged the would-be bomber and put him on a terrorist watch list.
But the new NSA system has stoked fears about insiders' access after Manning downloaded the entire contents of a similarly shared State Department database.
Manning was convicted last month of leaking classified information and subsequently sentenced to 35 years in prison.