Big Brother Watching You II

Big Brother.

US and British intelligence agencies have successfully cracked much of the online encryption relied upon by hundreds of millions of people to protect the privacy of their personal data, online transactions and emails, according to top-secret documents revealed by former contractor Edward Snowden.

The files show that the National Security Agency

and its UK counterpart GCHQ

have broadly compromised the guarantees that internet companies have given consumers to reassure them that their communications, online banking and medical records would be indecipherable to criminals or governments.

To criminals or governments...

Those methods include covert measures to ensure NSA control over setting of international encryption standards, the use of supercomputers to break encryption with "brute force", and – the most closely guarded secret of all – collaboration with technology companies and internet service providers themselves.

Through these covert partnerships, the agencies have inserted secret vulnerabilities – known as backdoors or trapdoors – into commercial encryption software.

The files, from both the NSA and GCHQ, were obtained by the Guardian, and the details are being published today in partnership with the New York Times and ProPublica. They reveal:

• A 10-year NSA program against encryption technologies made a breakthrough in 2010 which made "vast amounts" of data collected through internet cable taps newly "exploitable".

• The NSA spends $250m a year on a program which, among other goals, works with technology companies to "covertly influence" their product designs.

• The secrecy of their capabilities against encryption is closely guarded, with analysts warned: "Do not ask about or speculate on sources or methods."

• The NSA describes strong decryption programs as the "price of admission for the US to maintain unrestricted access to and use of cyberspace".

• A GCHQ team has been working to develop ways into encrypted traffic on the "big four" service providers, named as Hotmail, Google, Yahoo and Facebook.

The breakthrough, which was not described in detail in the documents, meant the intelligence agencies were able to monitor "large amounts" of data flowing through the world's fibre-optic cables and break its encryption, despite assurances from internet company executives that this data was beyond the reach of government.

The key component of the NSA's battle against encryption, its collaboration with technology companies, is detailed in the US intelligence community's top-secret 2013 budget request under the heading "Sigint [signals intelligence] enabling".

Funding for the program – $254.9m for this year – dwarfs that of the Prism program, which operates at a cost of $20m a year, according to previous NSA documents. Since 2011, the total spending on Sigint enabling has topped $800m. The program "actively engages US and foreign IT industries to covertly influence and/or overtly leverage their commercial products' designs", the document states. None of the companies involved in such partnerships are named; these details are guarded by still higher levels of classification.

Among other things, the program is designed to "insert vulnerabilities into commercial encryption systems". These would be known to the NSA, but to no one else, including ordinary customers, who are tellingly referred to in the document as "adversaries".

"These design changes make the systems in question exploitable through Sigint collection … with foreknowledge of the modification. To the consumer and other adversaries, however, the systems' security remains intact."

The document sets out in clear terms the program's broad aims, including making commercial encryption software "more tractable" to NSA attacks by "shaping" the worldwide marketplace and continuing efforts to break into the encryption used by the next generation of 4G phones.

The Guardian.

The New York Times.

ProPublica.