Wednesday, February 13, 2013

Dealing with Data: A Hard Drivin' Tale of Woe

My hard drive crashed last week, taking all of my data and digital life with it. I've written and rhymed about this already, but I thought I'd jot down some notes about how it happened and how I got my life back in case it helps anyone else in the same boat.

I have an iMac at home, running Snow Leopard, purchased about 3 years ago. It's been reasonably solid; we haven't had any hardware issues with it until this doozy.

Part I: The Symptoms
My wife complained that the computer had gotten painfully slow, so I took a look at it. Indeed, it was horrible, taking tens of seconds to perform simple tasks and minutes to do something really complicated like launch an application.
I ran Activity Monitor to see what was causing the problem and ... nothing. The CPU was nearly idle, memory usage was well below the physical limits, and the disk didn't seem to be thrashing.
On a hunch, I thought I'd check the disk, so I ran Disk Utility.

(Note to self: This would have been a good time to back up the data on the disk.)

Part II: The Botched Repair Job
"Verify Disk" in the Disk Utility app told me that the disk was not healthy and that I should repair it. It wouldn't let me do this on the boot disk, and told me to boot from the OS installation disk and try it again.

(Note to self: This was another great opportunity for me to back up the disk.)

Eventually (after I asked my wife) I was able to locate the install disk and booted from it (shut the computer down, power it on, hold down "c" after you hear the chime). I ran Disk Utility again, only this time the "Repair Disk" button was not dimmed out, so I clicked it, hoping that it would magically fix whatever had happened.

Repair Disk ran for a minute and then stopped and said that it was unable to repair the disk. It suggested that I back up the disk, format it, and re-install the OS.

(Note to self: This was a good time to regret having not backed up the data already.)

I could not get to the disk at this point, so I rebooted the machine normally (not from the install disk). This caused the machine to boot into the grey booting screen... and then shut off. I tried this a couple of times, and then booted with the install disk again. Once more, I was able to get into the OS, but I was not able to get to the disk (either through the UI or through the shell in a Terminal window). Moreover, the "Repair Disk" button (which had worked so beautifully the first time around) was now dimmed out again. In fact, the drive itself was dimmed out, like the system could see that there was something there, but couldn't actually mount it.

I tried running a Terminal window and getting to the disk, but it was not mounted (nothing in /Volumes), so I couldn't get there and couldn't copy the data.

(Note to self: This was when I realized that I really needed to have backed up everything several steps before.)

Part III: Go to Apple

I took the machine into the Apple Store, hoping that they'd have some advice for me.

The main thing they said was that they don't deal with personal data; as soon as the guy realized that there was data loss involved, he said there was nothing he could do. He gave me the card of a company two hours away and said that they might be able to retrieve the data for me for anywhere from a few hundred to several thousand dollars.

I love my family. And I love the digital pictures of my family that I'd apparently just lost. But I don't love them that much.

He said there was one other thing I could try, although it could risk further corruption that would make retrieval more difficult. But in case I wanted another option: boot the machine as a "target disk."

Part IV: Recovery

Booting as target disk means that you're running the computer as just an incredibly expensive external drive for some other computer. You power it on while holding the "t" key. Meanwhile, you've connected another computer to this machine via a cable that you almost certainly don't have, and the machine looks like just another disk drive to that other computer.

At this point, my bluetooth keyboard flaked out, in a combination of dying batteries and possibly confusion over bluetooth connection to a machine that didn't have a brain anymore. Fortunately, even a PC USB keyboard has a "t" key, so I was able to use that keyboard to boot into target mode.

I didn't have the cable I needed for this experiment (FireWire). Moreover, none of my other computers even have a FireWire port (newer Macs have Thunderbolt ports instead. I have a theory that they invent new interface standards every couple of years to keep cable companies in business). I ended up ordering a FireWire to Thunderbolt converter and a FireWire cable, which plugged together to connect a MacBook Pro to the dead iMac.

And ... voilĂ ! This actually worked. I plugged it all together, booted the dead machine as a target disk, and was able to see that disk on my other computer. I quickly (okay, it took hours for this much data to travel over FireWire) copied everything I could see on that drive over to my other system and to every other disk I could lay my hands on. I hired monks to transcribe it. I transmitted the bits as smoke signals. I tattooed the binary on my body. I had my children memorize long strings of hex.

Data recovered.

The weird thing here, to me (arguably not the most knowledgeable of people about disk drive technology) is that the disk was clearly screwed up enough that it would not mount when the machine was booted with the install disk, but it was good enough to be visible across FireWire.

Whatever.  I had my data back.

(Note to self: At this point, I felt like I'd cheated the system, since by rights I should not have been allowed to get my data back. But I'm okay with this. Crime may not pay, but cheating's not exactly a crime.)

Part V: Replacement

I searched for information on replacing the internal drive on an iMac and discovered that my machine/drive was actually under recall because apparently these particular Seagate drives are known to fail. So on the positive side, at least the replacement was easy and free. It's something I'd rather have known before the drive failed (apparently they do notify customers of these recalls; I may have missed that crucial memo). But at least I got the new hardware I needed, along with a clean install of my original OS.

Part VI: Back Up Your Damn Data

Now I have a working machine/drive again, and I have all (I hope) of my data on other drives. I am now ready to copy everything back and start using the machine again. But first, I'm going to figure out a backup plan. Because I know me, and if I just start using assuming I'm going to get to it soon, I'll be writing another piece in a couple of years about how I lost everything. Again.

I've polled several friends about what they do and the options boil down to these:
- Local backup
- Cloud backup
- Local + Cloud backup (for the truly paranoid, in which crowd I might now count myself)

Local backup:
There are a few options here, including:
- USB drive connected to a computer you want to back up, or a network drive connected to the network that the system is on (or at least can reach). Copy stuff manually (probably not a good idea, unless you're way better at doing this than I am), or set up some kind of automated backup solution, including the Mac's Time Machine. Note that you probably want to ensure that the backups actually happen; I had actually set up Time Machine in the past for the machine that died, but the drive decided to go to sleep and never wake up. So that didn't help.
- RAID array: This is an option on some desktop systems, where you can have disks that are essentially copies of each other, so if one goes out, the data is still alive on the other one. Swap out the bad disk for a good one and you're back in parallelizing business. This is not an obvious solution for an iMac since it only has the one internal drive.

Cloud backup:
The options that I've heard about most frequently included:
- DropBox and Google Drive: I've used the free version of these applications for a while, as a convenient way to exchange large files between computers. Recently, I've also started to use DropBox as a synchronization mechanism for some software that I use. And I use Google Drive for my various Google Docs. They both offer a small amount of storage for free (2GB on DropBox, 5GB on Google Drive). If you pay, they'll provide more storage (DropBox currently offers 100GB for about $10/month, Google Drive is about $5/month for the same 100GB).
The way these services work as a backup solution is that applications create a folder on your computer that is linked to their cloud service. You then create/copy/move anything you want backed up into that folder and it automatically syncs to the servers of these apps in the cloud. Essentially, you start using that cloud folder for all of you documents that you want to back up, and the service takes care of uploading it for you automatically.
- CrashPlan: This application is a bit more flexible; it allows you to specify various parts of your system that you want backed up and it automatically copies those items to its servers and keeps them sync'd over time. Also, it apparently encrypts the data that it uploads, so that you can feel a bit easier about your secure files that you're tossing up into the cloud. The prices vary, depending on how much storage you want, how many computers you want to back up, and how long a term you want to pre-pay.
- Carbonite: Like CrashPlan, Carbonite backs up everything you choose on a single system, encrypts it, and sends it off to the company's servers. You pay an annual fee ($59), or less per year for longer terms.

Local + Cloud
Some people are doing a combination of the above, where they have multiple local copies, but they also keep some of their core data backed up using one of the cloud services above.

I'm sure I haven't covered the full spectrum of backup possibilities here, but these are the main ones that have come up in my recent discussions with colleagues, friends, family pets... anyone that would listen to my tale of woe and help me figure out how to not fall into this pit of data Hell again.

Part VII: Conclusion

See Part VI: Back up your damn data.
Watch my video on the subject if you need more motivation.


Andrew said...

Oh fun, that brings back memories of 15 years ago when I used to make pocket money recovering my Mac-using neighbour's data using a box of disk scanning and recovery tools. I must have done it twenty times; for each neighbour about 3 times. Back then Macs were a tad more susceptible to corrupting their disks. HFS (and HFS+) is a phenomenally weird and wonderfully format. :-) Norton Utilities 3.X used to be able to bring back just about any disk from the dead. Disk first aid (precursor to disk utility) was never that great and sometimes corrupted the drive in its attempt to fix the damage.

The reason you couldn't boot was because something in the long chain of special flags and boot sector(s) went bad. All that stuff is independent of the data structures required to read the disk. I used to see disks with nothing wrong with them but the thing didn't boot. Just drag the finder out of the system folder and then drag it back in! Just kidding. That doesn't work anymore.


Stefan said...

I use CrashPlan (cloud and to a MacMini in the closet), Time Machine, and Super Duper. I am certifiably paranoid about data loss.

Unknown said...

When it comes to backing up in the cloud, does anyone have experience or an opinion regarding Backblaze? For less per month than your 100GB Google Drive, you're supposed to be allowed to back-up unlimited data. This combined with this blog post certainly makes me want to move this up on my to-do list.

TK said...

You should check out my tutorial on backing up with Amazon Glacier. It was written with Linux in mind, but should work just fine with OSX or even Windows. At $0.01 per GB per month, it's dirt cheap and offers great peace of mind.

Apart from that, I make local backups to a RAID-6 array as well as attempt to keep important files like music, pictures, and videos synced across all machines.

p.l said...
This comment has been removed by the author.
p.l said...

>- RAID array: This is an option on some desktop systems, where you can have disks that are essentially copies of each other, so if one goes out, (...) This is not an obvious solution for an iMac since it only has the one internal drive.<

2 bay NAS box with RAID1 setup, this seems to be the best solution.
Most of stuff you don't use daily keep only there (with network drive mounted), the rest keep on internal hardrive and sync to network drive with any utility, even bare rsync.
Plain NAS box without disks is about 150$ (like d-link dns-320 but there are plenty of others).
What is more, you can setup an automated backup of your NAS content to any off-site storage like cloud or other NAS :D.

Anonymous said...

You should generally be wary of trying to recover from install media since that media will be out of date, and fixes will have been made since it was released. That is the likely reason why the install media failed to recover but a different live (and up to date) system was just fine.

You could also have made a recovery disk (including on a USB stick) which would have saved you much time and hassle! It also has the advantage of being more up to date than install media.

hard drive data recovery said...

This is exactly what I am facing and going through right now!! Symptoms are there, problem and case scenario is almost the same as you've written. Let see if I can beat and over come the situation. Fingers crossed xx