TPM Muckraker

« previous | MUCK HOME | next »

Expert: White House Had "Primitive" Email Setup

From the AP:

A computer expert who worked at the White House provided the first inside look at its e-mail system Tuesday, calling it a "primitive" setup that created a "high" risk that data would be lost.

Steven McDevitt's written statements placed on the public record at a congressional hearing asserted that a study by White House technical staff in October 2005 turned up an estimated 1,000 days on which e-mail was missing....

McDevitt's statements detailed shortcomings that he said have plagued the White House e-mail system for six years. He declared that:

_The White House had no complete inventory of e-mail files.

_There was no automatic system to ensure that e-mails were archived and preserved.

_Until mid-2005 the e-mail system had serious security flaws, in which "everyone" on the White House computer network had access to e-mail. McDevitt wrote that the "potential impact" of the security flaw was that there was no way to verify that retained data had not been modified.

You can see McDevitt's full answers here (pdf).

Remember that as The Washington Post outlined last month, the Bush Administration managed to dismantle the Clinton Administration's email archive system without replacing it with anything at all.


30 Comments

| Leave a comment

Here's a bombshell for you, Paul:

Cunningham briber Mitch Wade's MZM somehow connected to White House e-mail?

Remember MZM, the government body shop whose founder and CEO Mitchell Wade was charged as a co-conspirator in the Duke Cunningham bribery scandal? Well, MZM is among the handful of contractors identified in writing by a White House technology official at a Waxman hearing today as having been hired to deal with White House email - millions of which have gone missing. See page 4/question 5 of thie questionnaire (.pdf) filled out by Mr. Steve McDevitt, a senior manager in office of the chief information office of the Executive Office of the President administration office from 2002 through 2006.

Article is here.
Questionnaire here.

Remember it was MZM that landed a curious contract for "furtniture" at the White House in 2002 for $140,000. MZM, an intelligence services company, never had a federal contract before that one.

And it was in 2002 that the White House inexplicably turned off the Clinton-era automatic e-mail archiving system. And replaced it with nothing.

Someone had to turn it off. MZM?

4 of 26 for a list of contractors supporting the Enterprise Architecture [EA] mission. Look at the similarity between the names between the NSA missions, DoD, NSA support for telecoms, and rendition:

Boeing
Booz Allen Hamilton
Lockheed Martin
MZM
Systems Management and Engineering, Inc
TKC Communications
Unisys

Issues:
- No-bid contracts
- Alleged Contractor interest in suppressing public knowledge of alleged war crimes evidence (rendition: Boeing subsidiary alleged involvement)
- Alleged contractor interest in suppressing information to Congress about alleged NSA illegal activivity, FISA violations, and impeachable offenses (Lockeed Martin alleged involvement)


The shared information fails to explain why the original data, in the original format before transfer/migration, is not available.

7 of 26: The plan did not include destroying data before it has been saved to a backup loaction [text added]: "The iniital email retention process involved a manual process of copying messages [not destroying, not reformatting] from the Exchange journals to .pst files for storage and retention."

Refs
5 of 26: "Detailed plans were created to support this migration." These plans were started well before the migration in 2002; yet, we're in 2008, and they're still attempting to "figure" this out. This does not address the question: Where are the original files? They were not destroyed, but copied.

- Are we to believe that the EOP, without outside questions by the court, would not have taken any effort to understand why the data was getting lost?

Timeline Questions

Page 16 of 26, question 23 discusses emails the IT-transfer team sent. Questions about the IT timeline arise: If there was a massive loss of data "during" the transition, then what indications would the IT department have before deleting the backup files? It would be helpful to understand details about the data migration:
- Why did they permit massive loss of archived data before making backups;
- Why did the transition plan get approved if it did not ensure, prior to data-access, that the original email was preserved in a stable format?

Note, staff reactions to lost email are not addressed in the responses; nor is a timeline of reconstitution provided.

- If the email truly went missing during a controlled transfer, then why no record outside EOP requesting a resend of emails to EOP?
- How long was it, after first 'data lost' during the transfer, did it take for the IT team to discover tere was a problem?
- How soon after e-mails went missing, did EOP staff realize, "The loss of data wasn't related to a nuclear event, but because of the IT department transition effort."

7 of 26: Note, the automated systems didn't write over, but copied:

"The Mail Attender utility was used to automatically copy email message from the journals to the .pst files on a regular basis."

- What happened to the original journals; why aren't they available for review?

13 of 26: This wasn't a random set of events, but well planned:

"Each step of the process was discussed and documented. The team met on a daily basis to plan activities and to report on actions that had been completed."

This sugests a second level of paperwork:
- Checklists to confirm "ready"
- Checksheets to monitor events
- Lists of indicators/warnings
- Options to face the known risk areas [See 8 of 26, question 11]

The known risks should have been, before proceeding, matched with technical performance measures; and managed along agreed-to criteria. For every risk, there should have been a mitigation plan, with prepared actions in the event of occurrence.

Yet, we're asked to believe something "out of the blue" occurred that "nobody thought about". Yet, the original data failes -- before transfer/copying -- "can't be found". This defies reason.

12 of 26 focuses on the .pst files, not the orignal files. What happened to the originals? It doesn't matter that there was "confusion" about what was going on with the .pst. Thats unrelated to the original data sources.

Stunning incompetence. There are COTS programs that archive all mail passing through an Exchange server (which is apparently what they're using). This should not have been rocket science; in fact, considering they had a working system in place from the prior administration, it shouldn't have even been an issue.

Did the Administration not trust the Clinton-era architecture because Clinton had it installed, or was there a deeper reason?

13 of 26, question 18: Note, when responding to the questions about the interim actions, they avoid the original journal files.

- Why wasn't there any mention of the journal failes in this list on page 13?
- Are we to believe that the journal files -- the original ones -- were never touched, and there was no problem?
- Where are these journal files?
- Where were the journal files during this analysis, and discussed on page 13?

See page 7 of 26, question 10:

"I also did not have any operational responsibility for the archiving of the mail in either the Notes or Exchange environments."- Who did have the job/responsibility to safeguard the original journal files, not the .pst files, regardless whether they were or were not transferred/migrated from/to any format?

Why does it appear as though there was no independent person/entity, outside EOP and unrelated to any NSA-telecom contractors, who would conduct an independent review of the planned data migration? An independent team would be one that was not involved with the original planning, not involved with any activity, merely there as an outsider to trouble shoot with a "black hat" to ensure to the leadership that the proposed plan was workable, and didn't miss anything.

See 12 of 26, question 16

This is what the outside reviewer, unerlated to the EOP or transfer team would most likely have caught before first approval to start:

"Some of the issues that became known include:"

Bullet 4: No audit controls
Bullet 5: No well documentation process

- How did this "plan" to transfer get approved?
- Who signed off on "this plan" and gave the go ahead to proceed?
- Were the contractors self-certifying "their inputs" were valid despite Boeing Lockeed Martin knowing full well about independent testing in DoD? For LM and Boeing to be involved at any capacity, and apparently "nobody" though to run "the plan" by someone that would ensure there was adquate documentation boggles the mind.

- Where are the memos, and private notes from teh Boeing-LM personnel who were assocaited with this?
- When will DoD IG demand from the Defense Plan Rep Offices that someone at LM/Boeing explain what their software engineers have been doing?
- Is there a problem we need to examine in re Boeing/LM in re the DoD Software compliance contracts?

This is absurd!

There are some spelling errors in the responses provided to Congress:

14 of 26: "It took a couple week" could be "It took a couple of weeks" or "It took a complete week"

14 of 26: "An independent verification and validation was also perform" could be "An independent verification and validation was also
performed"

- Which?

Issue: Sometimes if someone is rushed, they'll miss things. Suggests this person providing responses to Congress is working alone.

Where was the IV&V on the original plan before "go ahead" OKd, not just the subsequent analysis?

17 of 26: IV&V

As stated in a previous response, an independent verification and validation was performed by a separate set of contractors who were not members of the team that was perforrning the analysis effort.

IV&V indicates this was a major effort, not something someone just flips a switch.

18 of 26: Disclosures that by-name WH counsel attended specific meetings. Issue isn't the content, but that there's a specific meeting that the WH counsel was there.

They cannot claim "privilege" on these meetings now that they're disclosed.

- Where is Fielding on this?
- Where is Miers in responding to this disclosed conversation?

Here's the problem for the legal office:

Page 16 of 26:
- The comments specifically mention "counsel" and "power point" slides: These are now subject to review; and the exitence of the other contractors means non-EOP methods can be used to look at the Boeing, Lockeed Martin data.



23. Was this chart the only result of your analysÍs of messages from the EOP Exchange
environment? If not, did you produce any other briefing maferials or documents that
explained your methodology or findings? \ilhat were those documents?

Answer: Relates not to the EOP emails, but the emails the IT-related team were sending. These are outside the "orignal" emails which went missing

There were numerous documentso PowerPoint presentations and other memoranda that described the analysis that was performed, the actions taken to correct the process and the recommendations to improve the processing of .pst files. The team documented the details of each action taken to clean up and correct the identified issues.

There must be thousands of email messages between the team members that describe the actions
of the team, the completion of specific tasks, analysis of issues and to provide status to OCIO
management, OA Counsel and OA management.

- What was Miers specific comments on these issues?
- How did OA counsel coordinate with Miers on this as the PPT slides ere transferred?
- What comments did Miers make on any summary memos, charts or other things?

Makes you wonder what Miers knows that she would just thumb her nose in the direction of a citation of contempt.

Actually, I think the question should be reworded as what Miers doesn't know. Based off of her performance during the SCOTUS nomination, I'm inclined to think she's the biggest hack in history.

18 of 26: More disclosures of WH Counsel involvement:

Outside of OA, there were other meetings with the purpose of presenting these issues to White House Management and White House Counsel and their staff.

- WH counsel and their staff were involved!

Cannot claim privildge on the meetings now that this meeting has been disclosed, and details of that meeting provided.

- What were the specific staff concerns about this "planned effort" to 'fully comply' with the "records retention act"?

- How can any of the WH counsel say, "The President doens't have to comply," yet the record shows the WH counsel was involved?

- Why bother having WH counsel involved with somethign that "doesn't fall under" the records act?

If the law, as the EOP/OVP wants us to believe, "didn't apply," then there whould have been no reason for WH counsel to instent upon, an dbe present during any of these planning actions. None.

32 CFR 2800

Take a look at 32 CFR 2800. The fact that WH counsel was present at these meetings destroys Addington's last credibiltiy in avoiding OVP compliance in re OVP Office Security Compilance requirements.

POW Working Group Meetings

The same issue occurred with the POW working group meetings: Same players were involved with meetings on the POWs, but they feigned ignorance later. WH counsel, OVP, and DOJ cannot argue, "We had no idea about these records," when they were involved with the meetings: Rendition, POW abuse. This looks like WH Counsel should have known that the records, as they allegedly related to war crimes evidence, was not meeting either the US requirements or Geneva obligations.

20 of 26: More WH Counsel involvement

These reviews include White House and OA Counsel, White House Office of Records Management and OA Records Management.

This is a COVER STORY put forth to show how they just didn't have a very good email system, damn those problems.

The Cover Story is to provide a logical reason for the 10 million destroyed emails.

They destroyed evidence, a violation of record keeping laws... and this well planted story is their way out of that violation of federal law.

Aw gee did we forget to archive?

Keep those "perception management" stories flowing....

Just like those 18,000 missing votes -- we just didn't have a couple of procedures in place... damn... sorry bout that...

CROCK OF S**T

Although this administration is rife with a mind boggling amount of corruption, I still cannot believe those e-mails disappeared without a trace.

It's insinuating that their email system is worse than AOL 2.0.

One of the interesting aspects here is that I think the Clinton Administration said they *did* have an e-mail system that appropriately retained and managed e-mails covered under the Presidential Records Act. If the Clinton Administration could do it in the 90s, what did the Bush Administration do to roll back the clock, so to speak?

Remember, Bush called it 'The Google'.

That ought to be representative of his knowledge of e-mail and computers. Unlike his brother Jeb who suffers from crackberry addiction, it was painfully obvious that he was not well versed in computer usage; furthermore, this would indicate, at least to me, that the inner workings and gears of the executive branch were of little interest to curious George, and that the way things were lost were often lost on him.

Well, sure, but a President doesn't make these decisions himself in the White House. And, if indeed a workable system was already in place under Clinton, was it dismantled? Why? And further, for what reasons was the new system chosen?

If we go with the MZM/Mitchell Wade theory, we need to figure out for what purpose that system was turned off in 2002, which Bush himself had to have been certainly aware of.

What events in 2002 could have precipitated the shutoff to have occurred? Could this have been part of a campaign to help bring us to war in Iraq?

Here's the hypothetical I need an answer to, because I'm not too keen on computer science:

Karl Rove did use gwb43.com for official e-mails to evade the archives and the .gov email system. If a site were to have shared the same IP address as one another, is it possible for them to have been able to gain access to the gwb43.com e-mail system and snoop around?

What a load of crap! "Primitive" email system. Ever hear of Microsoft Outlook?

Look, if these are the same guys that can hook up a machine in a room and vacuum up EVERY bit of information on the internet, then hold that information for later data mining, then why in the world would be believe they could NOT monitor the email coming in and out of their building....UNLESS THEY DID NOT WANT TOO.

Doesn't pass the smell test.

I christened this "The Incompetence Theory". When people are SOOO incompetent, at some point you have to realize they are not incompetent, but doing exactly what they want to do.....

user-pic

I agree with you.

This administration has heavily relied on incompetency as a defense. It's painful to watch department heads at congressional hearings 'forget' or be unaware of everything that goes on in their departments. Either they think we are stupid or they just don't care what we think.

If there's one thing I've learned from this administration is that while there are hordes of incompetent bureaucrats, most of them are puppets of lobbyists and have had them installed in order to promote their agenda.

On the face, the administration has had staggering and often criminal incompetence; Alberto Gonzales, Miers, Monica Goodling and Lurita Doan all come to mind. In every instance of incompetence however, it always served the purpose of someone else.

The e-mail system fiasco, in my mind, cannot be any different. If it was indeed incompetence that gutted the system, then why wasn't it repaired? Who's purpose did it serve, and who decided it would be in their best interest to not have it repaired?

user-pic

Send a supeona to the NSA and AT&T. The Chimp and the Dark Sithlord had their illegal wire-tap/data dump up and running. They are bond to have copies.

Cull the servers at all the agencies (especially the DoJ) and the RNC.

This pdf is a post-changeover sales pitch for the micrsoftafcation of the white house systems.

This guy is the shill. He was hired to know nothing about the change over that was decided by Rove and company 2 years prior.

No mention of contracts with microsoft or RIM yet to start to convert the email system they needed a server with licensed windows server, MS Exchange and a RIM contract.

The first 2 years i believe were contracted out to RNC shell companies.

Is it a coincedence that Ms. Theresa Payton was previously involved in the loss of government data while she was at Bank of America?

Bank of America loses a million customer records

February 25, 2005
A “small” number of backup tapes with records detailing the financial information of government employees were lost in shipment to a backup center, Bank of America said on Friday.

The tapes contained information on the customers and accounts of the U.S. government’s SmartPay charge card program, which has more than 2.1 million members and annual transactions totaling more than $21 billion, according to the General Services Administration. Reports have pegged the number of cards affected at 1.2 million.

At the time of the data loss, she was CIO at Bank of America. Sounds like a perfect Admin hire!


Leave a comment

Advertisement
Please disable your adblocker!
Ads are how we pay the bills!

Subscribe
Tip Line

Josh
Marshall

Bio

Zachary
Roth

Bio

Advertise Liberally
Share
Close Social Web Email

"To" Email Address

Your Name

Your Email Address