Version 1.11, September 2008
This article presents a solution for situations in which an encrypted sparseimage (such as file vault) gets corrupted, and you happen to have an older backup of that same image (or have the skills to look for a lost header - see below).
Please note by "corrupt image" I don't mean necessarily "corrupt filesystem" (which may additionally be the case, but it is only indirectly handled here). The case handled here is: when the Finder or the terminal (hdiutil) say "corrupt image" - not when they say "no mountable filesystems". For the latter (whether it is an image or a real disk), there's no better tool than Disk Warrior.
Analysis: the symptoms...:
trying to mount the image (by finder or hdiutil) yields...:
[host:~] user% hdiutil attach -nomount -noverify -noautofsck -stdinpass MyImage.sparseimage
hdiutil: attach failed - corrupt image
using the -debug option in hdiutil showed some more (taken the most informative lines):
[host:~] user% hdiutil attach -nomount -noverify -noautofsck -stdinpass -debug MyImage.sparseimage
... WARNING flushing header and index nodes failed. DIResolveBackingStoreToDiskImage: unable to resolve to any disk image class. 103. ...
Just because a little header is gone all my data gone?! Without even the possibility to repair it somehow!?
Yes. Why? Because AES encryption is not just your passphrase molded into your data. Your passphrase gets thru a method called pbkdf2. This function generates the 128-bit key needed using your passphrase.
The Key, the salt, the iv (initialization vector) and other info are stored into the image header, a 4kb block, which is in turn encrypted using 3DES-EDE. Without this data, you're not going to be able to recover your stuff even if you remember the passphrase.
Last but NOT least, Apple has (by now) 2 formats for the header and 2 places for them: the old one, version 1, (concerning my case and all FileVault/encrypted images created before Mac OS X 10.4.7), was stored at the END of the image.
With version 1 of the header, at every change of the image, the "header" has to be re-appended to the end of the file. If the computer freezes, or you have a power interruption, and mac os x fails to write this down to the disk, you lose the most important piece of information.
The new format (version 2) introduced with Mac OS X 10.4.7 stores the header at the BEGINNING of the image, which is a much safer idea. But this actually happens only for new images.
So my advice is: If you have older encrypted sparse images, upgrade to Mac OS X ≥ 10.4.7, make new encrypted sparse images immediately and copy the files back onto them (the easiest way for FileVault: disable it and re-enable it in the prefpane. Make sure you click the checkbox "securely erase". Tested, it works.)!
If You still have an old backup of the same broken image, you can try the following
(after making a BACKUP of both the broken and the old image!).
(If you don't have an older backup, you have really bad luck. But see below, on how to seek your hard disk for a lost header..)
Note: I'm assuming the name "WorkingBackup.sparseimage" for the working backup image, and "Corrupt.sparseimage" for the corrupt one. Replace names in the first two lines or rename your images accordingly.
Important note (as of September 2008): As two readers have been reporting (thanx to Pietro and G.Ray), it seems that if the backup sparseimage from which you take the "header" has a virtual size lower than the one with the broken header, although you will be able to open it and see the complete contents after the following operation, you will still be unable to access the contents of files which are stored after the size of the working backup.
It looks like the v1 header contains information about the virtual size of the image as well. The solution for this is: make sure the working backup image is filled up to at least the size of the broken image, and then proceed with step 1. You can do this by copying/duplicating a big file on it or by using dd (in this example we store a file with 500mb size on the sparseimage). Or even smarter, as G.Ray pointed out: just copy the corrupt sparseimage within the mounted working backup.. :)
[host:~] user% WORKINGIMG="WorkingBackup.sparseimage"
[host:~] user% hdiutil attach $WORKINGIMG
[host:~] user% dd if=/dev/zero of=/Volumes/WorkingImageMountpoint/fillup.dat bs=1m count=500
[host:~] user% diskutil unmountDisk diskX
1) copy the header of the working backup image (the last 4kb) to the end of the broken image:
[host:~] user% WORKINGIMG="WorkingBackup.sparseimage"
[host:~] user% CORRUPTIMG="Corrupt.sparseimage"
[host:~] user% dd if=${WORKINGIMG} bs=1 skip=`ls -al ${WORKINGIMG} | awk '{ print \$5 -4096 }'` of=${CORRUPTIMG} seek=`ls -al ${CORRUPTIMG} | awk '{ print \$5 }'`
2) try to attach it, without mounting it, as the filesystem will need to be probably checked:
[host:~] user% hdiutil attach -nomount -noverify -ignorebadchecksums $CORRUPTIMG
3) if it attaches as a device, check the image filesystem with fsck (change X and Y accordingly. hdiutil tells you them in step 2):
[host:~] user% fsck_hfs -f /dev/diskXsY
if it is not enough, like in my case, try with DiskWarrior 4 (having the image attached but not mounted, as shown above).
4) After the check, I suggest to detach the whole disk image (change X accordingly..):
[host:~] user% hdiutil detach diskX
5) and mount it again (or double-click in it in the Finder..):
[host:~] user% hdiutil attach $CORRUPTIMG
NOTE: I've seen that sometimes, Mac OS X actually mounts an image but doesn't show the volume in the Finder or on the desktop (don't know why..). Usually this is the case when you doubleclick/hdiutil an image, enter the passphrase, and there are no errors but nothing "appears".
To check it: If You have "my computer" icon in the Finder prefs activated, you will find it there. If not, select Go -> Go To Folder -> /Volumes and see if it is actually mounted. Another good source of information on mounted disks is Disk Utility.
Alternatively, in the Terminal:
with this command you can try to "wake up" the disappeared image:
[host:~] user% disktool -r
with this commands you can see the list of mounted disks/volumes:
[host:~] user% diskutil list [host:~] user% mount
Read http://crypto.nsa.org/vilefault/23C3-VileFault.pdf -
THANKS to the guys at nsa.org! THEY did the real in-depth study to make this possible!
I just put together the results for the purpose of recovering my stuff (and hopefully, that of others too).
Useful decryption tool (included in http://crypto.nsa.org/vilefault/vilefault-23c3.tar.gz):
I used the source of vfdecrypt, vfdecrypt.c/.h, to better understand what's happening and the binary to obtain an unencrypted version of the image, which I then attached with hdiutil. Might be useful for You, too:
[host:~] user% curl -O 'http://crypto.nsa.org/vilefault/vilefault-23c3.tar.gz'
[host:~] user% tar xzf vilefault-23c3.tar.gz
[host:~] user% cd vilefault/vfdecrypt
[host:~] user% BROKENIMG="MyBrokenImage.sparseimage"
[host:~] user% PASSPHRASE="secret passphrase"
[host:~] user% gcc -lcrypto util.c vfdecrypt.c -o vfdecrypt
[host:~] user% ./vfdecrypt -v -i $BROKENIMG -o ${BROKENIMG}_decrypted.sparseimage -p "${PASSPHRASE}"
I'm posting here also the binaries (ppc and intel) for vfdecrypt, in case you don't have gcc installed.
They are compiled as stated above, from the original sources, without any modification:
vfdecrypt-ppc (tgz) - vfdecrypt-intel (tgz)
If You made a new filevault before 10.4.7, it is highly advisable to make a fresh one. This will reduce the risk of corruption dramatically. There is an easy way to check if Your image has the header at the beginning or at the end:
[host:~] user% IMG2TEST="/Users/.username/username.sparseimage"
[host:~] user% dd if=${IMG2TEST} bs=1 count=8 2>/dev/null | grep -c encrcdsa
If the result is "1" then you have a version 2 header, which is at the beginning. That's good :)
If it is 0, then you have the old format, version 1, which places it at the end. You can counter-Check it with the following:
[host:~] user% IMG2TEST="/Users/.username/username.sparseimage"
[host:~] user% dd if=${IMG2TEST} bs=1 skip=`ls -al ${IMG2TEST} | awk '{ print \$5 -8 }'` 2>/dev/null | grep -c cdsaencr
If you have a version 1 header (bad! risky!), the latter line will output 1.
As You can see from the above, both headers have a string to recognize them: version 1 headers END with the string "cdsaencr" , while version 2 headers BEGIN with the string "encrcdsa". If you have no backup image from which to restore the header, there is some chance to find these on the free space of your hard disk. To do this, the best thing is to write a script in perl, php, or a program in C, which reads your hard drive partition device (the one containing the broken image, e.g. /dev/disk0s3), searching for one of those strings (presumably "cdsaencr", as version 1 headers are the ones that break more often). If you find it, try to copy that block back to a file (best on another device, to avoid overwriting it). Be sure to seek to the position where you found the string, minus 4088. The inverse is true for "encrcdsa", version 2, i.e., position of the "e" + 4095.
In fact, I believe that if the header of a version 2 image has been corrupted or deleted, most probably you'll also have to reconstruct more of the image, that is, the partition map for example.. so I'm quite sure that just taking the first 4kb won't be enough. Didn't have this case and I hope to never have it... :)
Good Luck!
DISCLAIMER: This info is provided as-is, with no warranties of any kind.
©2007 Lorenzo Perone (email)
Thanks to all people that mailed about this article and helped making it better / clarifying things :)
To macosxhints.com admins: it is OK to copy the contents of this page into a hint
(provided that link to it and to author are kept, if You're on a good mood) ;)