Whistle-Blower's Evidence, Uncut
Former AT&T technician Mark Klein is the key witness in the Electronic
Frontier Foundation's class-action lawsuit against the
telecommunications company, which alleges that AT&T cooperated in an
illegal National Security Agency domestic surveillance program.
Inside the Secret Room
A federal judge refuses to give AT&T back its internal documents, but
orders the EFF not to give them out.
[TELECOM Digest Editor's Note: However, nothing was said about other
publications (net or printed) not giving them out. PAT]
Years before the NSA's warrantless surveillance program made national
headlines, then-AT&T technician Mark Klein suspected his company was
colluding with the government to spy on Americans.
The Ultimate Net Monitoring Tool
A little-known company called Narus makes the packet-inspection
technology said to be the basis of the NSA's internet
surveillance. Here's how it works.
In a public statement Klein issued last month, he described the NSA's
visit to an AT&T office. In an older, less-public statement recently
acquired by Wired News, Klein goes into additional details of his
discovery of an alleged surveillance operation in an AT&T building in
Klein supports his claim by attaching excerpts of three internal
company documents: a Dec. 10, 2002, manual titled "Study Group 3,
LGX/Splitter Wiring, San Francisco," a Jan. 13, 2003, document titled
"SIMS, Splitter Cut-In and Test Procedure" and a second "Cut-In and
Test Procedure" dated Jan. 24, 2003.
Here we present Klein's statement in its entirety. This, and other
documents were filed under seal in federal court in San Francisco.
AT&T's Implementation of NSA Spying on American Citizens
31 December 2005
I wrote the following document in 2004 when it became clear to me that
AT&T, at the behest of the National Security Agency, had illegally
installed secret computer gear designed to spy on internet traffic. At
the time I thought this was an outgrowth of the notorious Total
Information Awareness program, which was attacked by defenders of
civil liberties. But now it's been revealed by The New York Times that
the spying program is vastly bigger and was directly authorized by
President Bush, as he himself has now admitted, in flagrant violation
of specific statutes and constitutional protections for civil
liberties. I am presenting this information to facilitate the
dismantling of this dangerous Orwellian project.
AT&T Deploys Government Spy Gear on WorldNet Network
-- 16 January, 2004
In 2003 AT&T built "secret rooms" hidden deep in the bowels of its
central offices in various cities, housing computer gear for a
government spy operation which taps into the company's popular
WorldNet service and the entire internet. These installations enable
the government to look at every individual message on the internet and
analyze exactly what people are doing. Documents showing the hardwire
installation in San Francisco suggest that there are similar locations
being installed in numerous other cities.
The physical arrangement, the timing of its construction, the
government-imposed secrecy surrounding it and other factors all
strongly suggest that its origins are rooted in the Defense
Department's Total Information Awareness (TIA) program which brought
forth vigorous protests from defenders of constitutionally protected
civil liberties last year:
"As the director of the effort, Vice Adm. John M. Poindexter, has
described the system in Pentagon documents and in speeches, it will
provide intelligence analysts and law enforcement officials with
instant access to information from internet mail and calling records
to credit card and banking transactions and travel documents, without
a search warrant." The New York Times, 9 November 2002 To mollify
critics, the Defense Advanced Research Projects Agency (Darpa)
spokesmen have repeatedly asserted that they are only conducting
"research" using "artificial synthetic data" or information from
"normal DOD intelligence channels" and hence there are "no
U.S. citizen privacy implications" (Department of Defense, Office of
the Inspector General report on TIA, December 12, 2003). They also
changed the name of the program to "Terrorism Information Awareness"
to make it more politically palatable. But feeling the heat, Congress
made a big show of allegedly cutting off funding for TIA in late 2003,
and the political fallout resulted in Adm. Poindexter's abrupt
resignation last August. However, the fine print reveals that Congress
eliminated funding only for "the majority of the TIA components,"
allowing several "components" to continue (DOD, ibid). The essential
hardware elements of a TIA-type spy program are being surreptitiously
slipped into "real world" telecommunications offices.
In San Francisco the "secret room" is Room 641A at 611 Folsom Street,
the site of a large SBC phone building, three floors of which are
occupied by AT&T. High-speed fiber-optic circuits come in on the 8th
floor and run down to the 7th floor where they connect to routers for
AT&T's WorldNet service, part of the latter's vital "Common Backbone."
In order to snoop on these circuits, a special cabinet was installed
and cabled to the "secret room" on the 6th floor to monitor the
information going through the circuits. (The location code of the
cabinet is 070177.04, which denotes the 7th floor, aisle 177 and bay
04.) The "secret room" itself is roughly 24-by-48 feet, containing
perhaps a dozen cabinets including such equipment as Sun servers and
two Juniper routers, plus an industrial-size air conditioner.
The normal work force of unionized technicians in the office are
forbidden to enter the "secret room," which has a special combination
lock on the main door. The telltale sign of an illicit government spy
operation is the fact that only people with security clearance from
the National Security Agency can enter this room. In practice this has
meant that only one management-level technician works in
there. Ironically, the one who set up the room was laid off in late
2003 in one of the company's endless "downsizings," but he was quickly
replaced by another.
Plans for the "secret room" were fully drawn up by December 2002,
curiously only four months after Darpa started awarding contracts for
TIA. One 60-page document, identified as coming from "AT&T Labs
Connectivity & Net Services" and authored by the labs' consultant
Mathew F. Casamassima, is titled Study Group 3, LGX/Splitter Wiring,
San Francisco and dated 12/10/02. This document addresses the special
problem of trying to spy on fiber-optic circuits. Unlike copper wire
circuits which emit electromagnetic fields that can be tapped into
without disturbing the circuits, fiber-optic circuits do not "leak"
their light signals. In order to monitor such communications, one has
to physically cut into the fiber somehow and divert a portion of the
light signal to see the information.
This problem is solved with "splitters" which literally split off a
percentage of the light signal so it can be examined. This is the
purpose of the special cabinet referred to above: Circuits are
connected into it, the light signal is split into two signals, one of
which is diverted to the "secret room." The cabinet is totally
unnecessary for the circuit to perform -- in fact it introduces
problems since the signal level is reduced by the splitter -- its only
purpose is to enable a third party to examine the data flowing between
sender and recipient on the internet.
The above-referenced document includes a diagram showing the splitting
of the light signal, a portion of which is diverted to "SG3 Secure
Room," i.e., the so-called "Study Group" spy room. Another page
headlined "Cabinet Naming" lists not only the "splitter" cabinet but
also the equipment installed in the "SG3" room, including various Sun
devices, and Juniper M40e and M160 "backbone" routers. PDF file 4
shows one of many tables detailing the connections between the
"splitter" cabinet on the 7th floor (location 070177.04) and a cabinet
in the "secret room" on the 6th floor (location 060903.01). Since the
San Francisco "secret room" is numbered 3, the implication is that
there are at least several more in other cities (Seattle, San Jose,
Los Angeles and San Diego are some of the rumored locations), which
likely are spread across the United States.
One of the devices in the "Cabinet Naming" list is particularly
revealing as to the purpose of the "secret room": a Narus STA
6400. Narus is a 7-year-old company which, because of its particular
niche, appeals not only to businessmen (it is backed by AT&T, JP
Morgan and Intel, among others) but also to police, military and
intelligence officials. Last November 13-14, for instance, Narus was
the "Lead Sponsor" for a technical conference held in McLean,
Virginia, titled "Intelligence Support Systems for Lawful Interception
and Internet Surveillance." Police officials, FBI and DEA agents, and
major telecommunications companies eager to cash in on the "war on
terror" had gathered in the hometown of the CIA to discuss their
special problems. Among the attendees were AT&T, BellSouth, MCI,
Sprint and Verizon. Narus founder, Dr. Ori Cohen, gave a keynote
speech. So what does the Narus STA 6400 do?
"The (Narus) STA Platform consists of standalone traffic analyzers
that collect network and customer usage information in real time
directly from the message.... These analyzers sit on the message pipe
into the ISP (internet service provider) cloud rather than tap into
each router or ISP device" (Telecommunications magazine, April
2000). A Narus press release (1 Dec., 1999) also boasts that its
Semantic Traffic Analysis (STA) technology "captures comprehensive
customer usage data ... and transforms it into actionable
information.... (It) is the only technology that provides complete
visibility for all internet applications."
To implement this scheme, WorldNet's high-speed data circuits already
in service had to be rerouted to go through the special "splitter"
cabinet. This was addressed in another document of 44 pages from AT&T
Labs, titled SIMS, Splitter Cut-In and Test Procedure, dated
01/13/03. "SIMS" is an unexplained reference to the secret room. Part
of this reads as follows:
"A WMS (work) Ticket will be issued by the AT&T Bridgeton Network
Operation Center (NOC) to charge time for performing the work described in
this procedure document....
"This procedure covers the steps required to insert optical splitters into
select live Common Backbone (CBB) OC3, OC12 and OC48 optical circuits."
The NOC referred to is in Bridgeton, Missouri, and controls WorldNet
operations. (As a sign that government spying goes hand-in-hand with
union-busting, the entire (Communication Workers of America) Local 6377
which had jurisdiction over the Bridgeton NOC was wiped out in early 2002
when AT&T fired the union work force and later rehired them as nonunion
"management" employees.) The cut-in work was performed in 2003, and since
then new circuits are connected through the "splitter" cabinet.
Another Cut-In and Test Procedure document dated January 24, 2003,
provides diagrams of how AT&T Core Network circuits were to be run
through the "splitter" cabinet. One page lists the circuit IDs of key
Peering Links which were "cut-in" in February 2003, including ConXion,
Verio, XO, Genuity, Qwest, PAIX, Allegiance, AboveNet, Global
Crossing, C&W, UUNET, Level 3, Sprint, Telia, PSINet and Mae West. By
the way, Mae West is one of two key internet nodal points in the
United States (the other, Mae East, is in Vienna, Virginia). It's not
just WorldNet customers who are being spied on -- it's the entire
The next logical question is, what central command is collecting the
data sent by the various "secret rooms"? One can only make educated
guesses, but perhaps the answer was inadvertently given in the DOD
Inspector General's report (cited above):
"For testing TIA capabilities, Darpa and the U.S. Army Intelligence and
Security Command (INSCOM) created an operational research and development
environment that uses real-time feedback. The main node of TIA is located at
INSCOM (in Fort Belvoir, Virginia).."
Among the agencies participating or planning to participate in the
INSCOM "testing" are the "National Security Agency, the Defense
Intelligence Agency, the Central Intelligence Agency, the DOD
Counterintelligence Field Activity, the U.S. Strategic Command, the
Special Operations Command, the Joint Forces Command and the Joint
Warfare Analysis Center." There are also "discussions" going on to
bring in "non-DOD federal agencies" such as the FBI.
This is the infrastructure for an Orwellian police state. It must be shut
By Ryan Singel
13:30 PM May, 26, 2006
Formerly sealed documents from a lawsuit against AT&T for allegedly
helping the National Security Agency spy on Americans' communications
without a warrant were released in redacted form Thursday, and confirm
the legitimacy of documents published earlier by Wired News.
The papers, which were obtained by Wired News through an anonymous
source, included a declaration written by Mark Klein, several
snapshots of a secret room in an AT&T facility in San Francisco that
Klein alleges is used to spy on a wide swath of domestic internet
traffic, and eight pages of wiring diagrams marked "AT&T Proprietary."
In order to help the Electronic Frontier Foundation's class-action
lawsuit, Klein provided the online civil liberties advocacy group with
a sworn affidavit and three documents, totaling more than 140
pages. The EFF filed these, along with a motion asking for a
preliminary injunction that would stop the alleged spying and an
evaluation of Klein's evidence by a former FCC internet expert, under
seal with the court.
While the judge in the case initially declined to unseal the documents
last week, he ordered AT&T and EFF to jointly redact Klein's statement
and the preliminary injunction motion and make them public.
Much of the wording in the redacted text of Klein's affidavit (see
http://Wirednews.com for details and .pdf),
which was published in the court docket Thursday afternoon, matches
language in the statement published Monday by Wired News.
Technical details in the newly released documents also mesh with the
documents published by Wired News. Additionally, both sets of
documents refer to an employee who was cleared by the NSA to work in
the room, but who was later laid off by AT&T as part of a
downsizing. This shared detail, along with others, was not part of
Klein's only previous public statement, which was released by his
lawyer in early April and printed in full by Wired News.
The proposed preliminary injunction (.pdf) filed by the EFF also
referred to a declaration by J. Scott Marcus, a former senior
technical adviser for internet technology for the FCC.
Marcus found that the surveillance room described in documents
provided by Klein is "consistent with the media reports describing
telecommunication companies' assistance with the program, and
illustrates an infrastructure built and designed by AT&T Corp. to
conduct large-scale covert collection and intensive analysis of
substantial amounts of both international and domestic communications
carried by AT&T Corp.'s network, including domestic communications of
AT&T WorldNet internet service customers such as the plaintiffs."
Additionally, the preliminary injunction argues that the secret room
was connected to an "additional, parallel backbone network that would
be unnecessary if AT&T Corp. were merely using the Surveillance
Configuration for ordinary business purposes, because such analytical
results could, and logically would, be transmitted over the common
The presence of extra routers on the list of equipment in the
documents published by Wired News suggests intercepted traffic is
being forwarded somewhere, according to Columbia University computer
science professor Steven Bellovin.
But Bellovin points out the system described could not forward all the
internet data flowing into the room.
"An OC-3 network (150 Mbit/s) can't possibly carry all of the traffic
from multiple OC-3, -12 (621 Mbit/s), and -48 (2488 Mbit/s) networks
back to Ft. Meade (NSA headquarters)," Bellovin said in an
e-mail. "They'd have to do some filtering there." But the other
equipment in the room, including a Narus packet inspection tool, is
well suited to pick out traffic of interest and forward it along.
Bellovin was one of the technical experts who looked at the documents
for Wired News. He suggested that the room might be part of a
network-management system or used to comply with lawful wiretap
But Klein's statements about the room being off-limits to anyone not
cleared by the NSA make Bellovin suspicious.
"The taps are oddly located for the alleged purpose, but the
restricted-access room is unusual," Bellovin said.
AT&T filed a redacted brief in its defense this week, but carried out
the redaction in a sloppy manner that makes it easy to remove the
The poorly redacted text included the statement, "Although the
plaintiffs ominously refer to the equipment as the 'Surveillance
Configuration,' the same physical equipment could be utilized
exclusively for other surveillance in full compliance with" the
Foreign Intelligence Surveillance Act, according to CNET News.com.
That brief, along with one filed by the federal government, argued
that the judge should promptly look at secret documents prepared for
him by the government.
Those documents, which include sworn statements by the head of the NSA
and the director of national intelligence, constitute the government's
argument as to why the case should be dismissed on national security
U.S. District Court Judge Vaughn Walker will review the government and
AT&T's motions to dismiss the case June 23.
Copyright 2006 Wired News
[TELECOM Digest Editor's Note: That rule about 'forbidden to enter the
area' (by anyone other than a couple people) was also the case in
Chicago when the ESS machinery was first installed back in the early
to middle 1970's. The very large Illinois Bell building downtown at
Congress Parkway and Clark Street (65 West Congress) was originally
full of the older style switches and frames. When they started
installing the ESS equipment, one or two floors of the building
became totally _off limits_ to even telephone workers who did not
have a special pass to be in the area. I found this out through a
discussion with an old man (in 1975) who was about to retire from
Illinois Bell. He had been an 'inside plant' worker for many years.
I asked him once, "what does that ESS stuff look like?" His reply
was, "I have never seen it up close. None of us older guys are
allowed in that room. Only a couple of the young kids who know
about computers are allowed to go anywhere near it. They told me
and the guys I work with to stay away from the area." See
http://WiredNews.com for the full report, pictures and sketches, etc. PAT]