In March of 2004, virtually every member of the Justice Department's leadership was prepared to resign over the administration's warrantless wiretapping program. Thanks to former Deputy Attorney General James Comey's testimony, that much is well known. But even as Congress has debated giving the telecoms immunity from civil suits for participation in that program, it's still not understood precisely what was going on that caused such a revolt.

In his new book on the Justice Department under George Bush, New York Times reporter Eric Lichtblau, who, along with James Risen, broke the warrantless wiretapping story, sheds more light on this episode (while leaving plenty unexplained). But putting Lichtblau's telling together with recently reported details about the National Security Agency's data driftnet, it's possible to come to a fuller understanding of what was going on.

Shortly after 9/11, the administration began a much more aggressive surveillance program, a program that had two main components: an aggressive data mining program led by the National Security Agency and wiretapping done without a warrant that stemmed from leads generated by the data mining.

While that program was ongoing, the FISA court was still approving warrants submitted by the Justice Department like normal. But sometimes the two processes crossed. Sometimes the Justice Department would want a wiretap for someone who had already been wiretapped without a warrant. Lichtblau reports that "some 10 to 20 percent of all court warrants fell into this area of 'double coverage.'" Department lawyers had a clear reason for wanting to go through the court if possible: they could not build a legal case against a suspect if the surveillance was illegal.

In that case, the Department, with the consent of the chief judge of the FISA court, set up a system where only the chief judge would handle such applications -- since none of the other judges were privy to the program. It was a dicey process. James Baker, the Justice Department official in charge of intelligence issues, set up a process where any Department lawyers working under him could "opt out," if he or she didn't feel comfortable forwarding such a wiretap application to the court, Lichtblau reports. "Several lawyers" took him up on that offer.

But about that data mining. The NSA's driftnet spat out leads that FBI agents found largely useless. Lichtblau explains:

[F]or every real lead with a plausible terror connection, there were many others that washed out, the trail gone dead, the connection uncertain. NSA officials saw their surveillance and data-mining work as vital to stopping another attack, but at the FBI, officials complained privately that they were forced to chase many hundreds of NSA-generated leads that inevitably arrived at dead ends. The “Pizza Hut leads,” FBI officials began calling them derisively, because in the six-degrees-of-separation world of tracing who had called whom and whom that person had called and whom those people had called, the NSA leads often seemed to wind up with the pizza delivery man.

For all its useless leads, the data mining part of the operation was more controversial within the Justice Department, as Comey's testimony last summer made clear. All of this might have gone on indefinitely if Jack Goldsmith had not been tapped to lead the Justice Department's Office of Legal Counsel in 2003. Backed by Comey, his concerns about the program's legality finally pushed the Department into conflict with the administration over the program. You know that story -- the showdown between Department officials and Alberto Gonzales and Andy Card in John Ashcroft's hospital room.

The president backed down and told the Justice Department to make the necessary changes to the program. What those changes are, we still don't know exactly. But Lichtblau gives a good idea of the contours:

A detailed checklist containing some twenty items would be developed to determine how exactly the NSA went about determining who was a suspected member of al Qaeda when it started eavesdropping. An audit of past wiretaps would be conducted, with the Justice Department’s active involvement.... And most critically, several specific aspects of the program would be shut down altogether. What exactly the NSA shut down in order to keep the program alive remains unclear even today, but according to intelligence officials, it appears to involve several distinct and technical aspects of the operations that relied on the use of data-mining to trace communications patterns across U.S. borders, elements that officials at the Justice Department thought were particularly problematic in a program already rife with legal complications. “Imagine you’re doing ten things one day, and the next day you’re only doing eight of them,” one official said. “That’s basically what happened here.”