SSD Drive - Backup & Failure

[Arjan]

Data backup solutions in the field of Federal Information Systems Management Act or FISMA, Sarbanes-Oxley, or SOX, Basel II, the Federal Information System Controls Audit Manual, or FISCAM, Health Insurance Portability and Accountability Act, the Data Protection Act 1998, the Gramm-Leach-Bliley Act or GLBA and Foreign Corrupt Practices Act or FCPA in 1977.

Why is it that forum spammers always seem to use girls' names? In a forum like this where the population is probably >99% male it makes them stand out even more

[Arjan]

Thus, I expect the SSD drive to last several years.

Expect it to fail when you least expect it and and when you need it most: just before a deadline with data on it that represents days, weeks or months of work with no recent backup.

I have a SSD in my desktop PC and I love it (super fast and completely quiet) but I don't trust any storage media. I have boxes full of failed hard drives and have seen plenty of CD-R and DVD-R's fail. They all fail at some point.

[synthjoe]

Expect it to fail when you least expect it and and when you need it most: just before a deadline with data on it that represents days, weeks or months of work with no recent backup.

+1

As an IT professional I can only confirm that. I have 30 years old storage mediums working perfectly with worthless data on them and failed ones less than just a couple of years old with irreplaceable data. And anything inbetween, by truckloads.

Wear levelling and spare blocks are a good thing but my XP desktop writes several GB's on disk every day (registry, logs, temporary files, etc., you can display the column in task manager), so I don't expect SSD's to last too long, unfortunately... I'm using one in my laptop, though.

[donjuancarlos]

Just to clarify, SSDs have limited write/erase cycles, not read cycles. The Kronos mostly reads data. With wear leveling, your Kronos would have to be running like an enterprise server (constantly writing) to wear the drive out. So there is no need to worry there.

After approximately 10 years, however, NAND flash cells lose their charge to the point that data loss begins to occur. That's when and why you'll have to consider swapping out the drive. There should be some sweet tech to replace it with by then...

[shap]

No doubt there will also be an app for disk imaging the drive (unix?)

I'm not encouraging any of you to go out and void your warranty, but here's one way to handle backup that will work for people who run Linux.

The goal (for us) of this procedure is to preserve a pristine "factory install" image before we power up a new laptop. We image every new machine to our storage server, and then make a copy of the image to an external drive that is kept offline. This lets us restore a factory-shipped disk image (which incidentally wipes out our proprietary data) when we later re-sell the laptop.

We want to capture the state of the disk before any "first boot" configuration happens, so we run this procedure before powering on the new machine for the first time.

1. Remove the drive from the new machine and stick it in an external drive enclosure. Walk it over to your Linux PC. If you don't have a Linux PC, you can do this with a live Linux image without damaging your existing windows install.

2. Use dd to image the drive:

dd bs=1M if=/dev/sd? | bzip - > /path/to/outfile.img

you can also use gzip. We're paranoid, so we then use sha256sum to confirm that we have an accurate copy.

What this gets you is a block level backup of the entire drive, including the boot block.

Once you have this baseline, it is usually more efficient to make further backups by mounting the filesystems under linux and using the cpio or star commands (note: not tar - that doesn't save some of the newer kinds of file metadata). For portability reasons, cpio is preferred.

This is a UNIX Geek's approach. If all of this is greek to you, it's probably not an approach that is likely to work well for you.

Note, however, that the corresponding restore procedure only works if you are restoring to an identical drive - same model, not just same size.

[shap]

After approximately 10 years, however, NAND flash cells lose their charge to the point that data loss begins to occur.

Remind me about this in six years or so, and I'll post a drive refresh script. Or better still a drive upgrade script.

[shap]

According to Toshiba, you'd need to write 11GB of data every day for five years to reach the "endurance limit" of the media.

Given the write-fatigue specs on current drives, that's a conservative number. Practically speaking it provides a good working intuition. Unless you're deleting and re-loading large sample libraries every day, write fatigue will not be the thing that gets you. Better still: SSD's are much more robust than conventional drives with respect to vibration, acceleration (i.e. dropping them), and heat.

Our company has large (multi-terabyte) production read/write data sets have remained live for more than a decade. In any given year about 10% of the data set gets modified. So long as the disk utilization didn't exceed 85% (which is also the guideline for conventional drives, though for different reasons), I wouldn't hesitate to put those on SSD - the SSD would fail for heat reasons a lot sooner than it would fail for reasons of write fatigue. The only reason we haven't moved yet is the cost of SSDs - it's still cheaper for us to swap hard drives as they die on us.

The type of thing that shouldn't go on an SSD is something like a paging or swap file area, which is continuously overwritten at a very high rate.

[donjuancarlos]

I'll probably just pull my drive out after a couple of years and image it to some other hard disk I have lying around and then pop it back in...

[shap]

I'll probably just pull my drive out after a couple of years and image it to some other hard disk I have lying around and then pop it back in...

If the goal is to refresh the NAND state, the thing to do is to image it to another drive and then copy the (unchanged) image back. Compressing the image, by the way, is still a good idea, Writing multiple gigabytes to a hard disk can take a while.

Realistically, by the time is an issue you'll want a bigger SSD, so what you'll more likely do is buy a new one, and do a somewhat more complicated process to re-establish a bootable image on the new SSD.

Speaking of which, somebody needs to stick a PC keyboard into the USB jack and let us know whether any of the usual choices (esc, del, F2, F8, F10) brings up a BIOS boot menu for us.

[madbeatzyo111]

After approximately 10 years, however, NAND flash cells lose their charge to the point that data loss begins to occur. That's when and why you'll have to consider swapping out the drive. There should be some sweet tech to replace it with by then...

This property is known as data retention; it's different from drive endurance. It means that once data is written it should be 10 years before any loss occurs (due to charge dissipation). As shap mentioned all you need to do is refresh (ie re-write) it before that happens, thus resetting the 10 year clock. No need to replace the drive. Incidentally RAM data retention is effectively zero as it requires power to keep data loaded. I'd hate to have to replace my computer's RAM every time I shut it down

[donjuancarlos]

Yeah, cough, that's what I meant.

[synthjoe]

Just to clarify, SSDs have limited write/erase cycles, not read cycles.

While this is true, remember that a simple modification involves:
1. writing the file to a new area of the disk
2. modifying pointers of the filesystem
3. modifying file header information (last modified, etc.)

Additionally, FLASH memory (usually, in such applications) can only be written/erased in blocks, so modifying a single byte means erasing / rewriting a block of 512 bytes or multiples thereof.

Of course, the Kronos might write a lot less data than an average PC or a storage server, but an OS boot can often result in a lot of write operations - and I have no idea what the embedded linux does when 'idling' on the Kronos.

So, while I'm hearing what enthousiasts say, I'll stay in the sceptical camp until I see a 10 years old SSD working in a regular application throughout that time in practice.

[madbeatzyo111]

I don't think there's anything to worry about. The 10 year number that everyone throws around has to do with how long your data will remain intact (without refreshing), not how long the drive will last physically.

In other words, if you wrote some data and then turned off the Kronos for 10 years, your data might be gone, but the drive would still be good as new.

However, if you re-wrote the entire contents of your SSD more than a million times, then you will probably start to see some degradation. Assuming it takes 30min to re-write the entire contents of the drive, this would take you about 55 years. Hopefully you'll be using the Kronos XI by then

[synthjoe]

However, if you re-wrote the entire contents of your SSD more than a million times...

Unfortunately flash endurance is in the 10/100 thousand write/erase cycles rather than in the millions... Which brings your 55 years down, radically. Again (as someone has already correctly stated) retention and endurance are two different things, and while I can live with the retention figure, I'm worried about endurance in some of the applications. Maybe Kronos is not one of those, though.

[madbeatzyo111]

Ok let's use more realistic numbers then. Let's say 50000 cycles and 5 full disk re-writes per day. That's still 27 years.

Considering the usage model in the Kronos--tons of reading of the HD for live streaming, etc. with relatively much less writing, I would submit that a magnetic drive capable of live streaming (ie, 7-10krpm) substituted in place of the SSD would fail much earlier.

My company is in the process of replacing all of our laptops (tens of thousands) with SSDs because magnetics fail so regularly every 3-4 years. In this economy, taking such a big financial hit is not taken lightly and SSDs must have demonstrated some real lifetime advantage.