Thousands of technology, finance and manufacturing companies are working with US national security agencies, providing sensitive information and in return receiving benefits that include access to classified intelligence, said four people familiar with the process.
These programmes, whose participants are known as trusted partners, extend far beyond what was revealed by Edward Snowden, a computer technician who did work for the National Security Agency. The role of private companies has come under intense scrutiny since his disclosure that the NSA is collecting millions of US residents' telephone records and the computer communications of foreigners from Google and other internet companies under court order.
Many of these companies voluntarily provide US intelligence organisations with additional data, such as equipment specifications, that don't involve private communications of their customers, the four people said.
Makers of hardware and software, banks, internet security providers, satellite telecommunications firms and many other companies also participate in the programmes. In some cases, the information gathered may be used not just to defend the nation but to help infiltrate computers of its adversaries.
Along with the NSA, the Central Intelligence Agency, the Federal Bureau of Investigation and branches of the US military have agreements with such companies to gather data that might seem innocuous but could be highly useful in the hands of US intelligence or cyberwarfare units, according to the people, who have either worked for the government or in companies that have these accords.
Microsoft, the world's largest software firm, provides intelligence agencies with information about bugs in its software before it releases a fix, according to two people familiar with the process. That information can be used to protect government computers and to access the computers of terrorists or military foes.
Microsoft and other companies knew that this type of early alert allowed the US to exploit vulnerabilities in software sold to foreign governments, according to two US officials. Microsoft does not ask and cannot be told how the government uses such tip-offs, said the officials.
Frank Shaw, a Microsoft spokesman, said those releases were designed to give the government an early start on risk assessment and mitigation.
Shaw says there are several programmes through which such information is passed to the government.
Some US telecommunications companies willingly provide intelligence agencies with access to facilities and data offshore that would require a judge's order if it were done in the US, one person said.
In these cases, no oversight is necessary under the Foreign Intelligence Surveillance Act.
The extensive co-operation between commercial companies and intelligence agencies is legal and reaches deeply into many aspects of everyday life, though little of it is scrutinised by more than a small number of lawyers, company leaders and spies. Company executives are motivated by a desire to help national defence as well as to help their own firms, said the people, who are familiar with the agreements.
Most of the arrangements are so sensitive that only a handful of people in a company know of them, and they are sometimes brokered directly between CEOs and the heads of spy agencies, the people said.
Former NSA and CIA chief Michael Hayden described the attention paid to important company partners: "If I were the director and had a relationship with a company who was doing things that were not just directed by law but were also valuable to the defence of the republic, I would go out of my way to thank them and give them a sense as to why this is necessary and useful.
"You would keep it closely held within the company and there would be very few cleared individuals," Hayden said.
Co-operation between nine US internet companies and the NSA's Special Source Operations unit came to light along with a secret programme called Prism. According to a Power Point presentation provided by Snowden, the programme gathers e-mails, videos, and other private data of foreign surveillance targets through arrangements that vary by company, overseen by a secret panel of judges.
US intelligence agencies have grown far more dependent on such arrangements as the flow of much of the world's information has grown exponentially through switches, cables and other network equipment maintained by US companies. In addition to private communications, information about equipment specifications and data needed for the internet to work - much of which is not subject to oversight because it does not involve private communications - is valuable to intelligence, US law-enforcement officials and the military.
Typically, a key executive at a company and a small number of technical people co-operate with agencies, according to the four people. If necessary, an executive, known as a "committing officer", is given documents that guarantee immunity from civil action. The firms are provided with regular updates, which may include the broad parameters of how that information is used.
Intel's McAfee unit, which makes internet security software, regularly co-operates with the NSA, FBI and the CIA, for example, and is a valuable partner because of its broad view of malicious internet traffic, including espionage operations by foreign powers.
Such a relationship would start with an approach to McAfee's chief executive, who would then clear specific individuals to work with investigators or provide the requested data, the person said. The public would be surprised at how much help the government seeks, the person said. McAfee's data and analysis doesn't include information on individuals, said Michael Fey, the company's worldwide chief technology officer.
In exchange, leaders of companies are showered with attention and information by the agencies to help maintain the relationship, the person said.
In other cases, companies are given quick warnings about threats that could affect their bottom line, including serious internet attacks and who is behind them.