NAO have released details of their Email correspondence with HMRC leading up to the HMRC data breach, and answers a couple more questions I had with incident.
Click Here for NAO Emails
From the NAO Emails it is very clear to understand the HMRC data was zipped (compressed to make the data files smaller), likely with an application called Winzip. The so called password protection of CD we are told about is just a Winzip password, which wouldn't be very hard to defeat. See http://www.zipcure.com/ for instance.
On analysing what was said in the Emails further and ignoring the political spin about them...
NAO rep. states "I do not need address, bank or parent details in the download - are these removable to make the file smaller?" - Clearly NAO were not asking for the removal of the sensitive data for security, it appears the NAO wanted to receive a smaller database on the grounds of it being easier to manage on a single CD, i.e. a single zip file. This is contrary to the media reports which state NAO advised HMRC not to send sensitive information on security grounds.
So the NAO wanted the data to fit zipped on a single CD-R, in response this request the HMRC rep. states "I must stress we must make use of data we hold and not over burden the business by asking them to run additional data scans/filters that may incur a cost to the department."
In my view I think this an attempt to fob off NAO, rather than for a genuine financial reason. As running a report to filter out the unnecessary data doesn't have too much cost associated with it, it just takes a little time to organise. So I am guessing the HMRC rep. knew this and didn't want to go through the hassle of extracting the information out the HMRC IT systems again. Sure I could be wrong in assumption, I'm just going from pass experiences with requesting stuff from busy IT bods.
These are my own views on reading the Emails, please let me know your views, and of course the content of these Emails makes absolutely no excuse for HMRC failing millions of people in not protecting our private information.
8 comments:
Not entirely true. If Winzip v9 or later was used, there is a a mucch more secure password/encryption scheme used involving an RFC2898 password to key derviation function and AES encryption.
If they used this and a non-trivial short password then crack programs would struggle to reveal the password used.
See http://www.winzip.com/aes_info.htm
and
http://www.lastbit.com/zippsw/default.asp
Sure I'm guessing which zip application was used, perhaps it's a free one either way you can still brute force / dictionary attack against a zip file password until the cows come home no matter what encryption is applied, sure it does take longer against a WinZip 9, but if you were determined enough could find ways to speed up the process.
One brute force Zip file cracker, "ZipCure" claims it can crack 90% of passwords within an hour with the right settings, including WinZip 9 files. Perhaps I should run some experiments with it and find out for sure.
It would be really interesting to find out the strength of the password HMRC used, but you know that will never come out. I really want to believe they used a non-trivial long password...
If only HMRC used PGP to zip the file, providing asymmetric encryption, guaranteeing only the recipient can decrypted/unzip and read the information. It's not as though PGP costs great deal more than WinZip either.
see my other post on bruteforcing a zip password http://blog.itsecurityexpert.co.uk/2007/12/power-of-playstation.html
It was Winzip 8 if you're interested.
Thanks for the info on WinZip.
If the WinZip 8 archive has over 5 files in it, the password can be easily recovered in less then an hour on a regular PC, regardless of the password complexity and length.
It's q 389 on the link provided.
http://www.publications.parliament.uk/pa/cm200708/cmselect/cmtreasy/uc57-iii/uc5702.htm
I've been told that with winzip 8, you need to have a good sample of the text inside the document to crack the encryption. DO you have any views on the truth of that?
Q388 Mr Dunne: Can we turn to the actual data itself? Are you able to tell us, without giving away public information - either tell us now or privately - the version of software on which the data was sent on to the CD?
Mr Hartnett: I am not able to tell you that. Sarah?
Ms Walker: No.
Mr Hartnett: The only thing I can tell you - and we will write to you, Mr Dunne - is how the CD was protected, which was with Winzip 8.
Q389 Mr Dunne: Winzip 8?
Mr Hartnett: Eight not nine.
Q390 Mr Dunne: Does Winzip 8 allow for automatic encryption?
Mr Hartnett: No, I think that is nine. Winzip 8 allows for compression. I am sorry, I am not a technician, but it allows for compression and password protection is my understanding.
Q391 Mr Dunne: Are you able to tell us, again without making this easy for someone who may have these CDs, whether a dictionary password was used for password protection?
Mr Hartnett: I do not know the answer to that.
Q392 Mr Dunne: Would you be able to write to us privately - I do not know if we can keep that confidential - and, secondly, the number of symbols in the password? Would that be possible to provide confidentially or will that come out in the Kieran Review?
Mr Hartnett: It will come out in the Kieran Pointer Review."
There is a big difference between a password protected zip file created with WinZip version 8 and below, and WinZip version 9 and above, hence the question.
However even WinZip 9 passwords can be brute forced, namely every combination of letters, special characters tried until the password is found, in this scenario the longer the more complicated the password, the longer it take to crack, also the processing power of your PC effects speeds. i.e. 4 characters seconds, 6 characters an hour, 8 characters a almost a day etc
However WinZip 8 and below are vulnerable to a different password cracking method, which can be cracked in under an hour on a standard PC. The password "recovery" software I use relies on the number of files within the zip archive rather than file size or text within the zip archive, usually 5 files+ is enough to recovery any password within an hour.
I tell you what I'll do, I run some experiments with WinZip 8, find out for sure and post the results in a fresh post.
Thanks for the interesting comments
Thanks. My working assumption is that there are 2 csv files - each with 12.5 million lines of data. On each line will be a surname of every person in receipt of child benefit.
Post a Comment