Wednesday, June 12, 2013

DiskFresh stops data fading away

My development server has two huge 2TB drives in it, which makes backups a nightmare. But what has worried me even more is that these drives are too big to do proper disk maintenance. No drive is "too big to fail", and the more data it contains, the bigger the disaster when it does fail. It's just a matter of time.
All magnetic disks suffer from the same problem: the analogue magnetic signal decays over time. If the file is being regularly updated, then this isn't a problem, because each update rewrites the analogue signal. But what about those old music files that you haven't listened to in 2 years? How do they get updated? When was the last time your hard drive even read those files to see if they are OK or not?
Until now I have used Spinrite to maintain my hard drives. On my laptop this isn't a hassle, because the drive is only 160GB big, and I can run Spinrite overnight in "refresh mode", where it reads and rewrites every sector on the disk, whether it has been used or not. If any sectors are damaged or faulty, Spinrite will attempt to recover the data. It usually succeeds.
In addition, I use Carbonite to make an automatic backup to "the cloud" of all the files I need on a regular basis. This includes all the programming projects I am working on. So if I file gets damaged or corrupted, or I just screw up and need an older copy, I can get it back in a few minutes. For bigger files like my two 4GB Truecrypt volumes, I use SyncToy to make copies to the server, and to an external USB hard drive.
But this is where things start getting difficult. The 500GB USB drives are slow, and it can take ages to run Spinrite on them. Each of the 2TB drives takes Spinrite nearly two weeks to check thoroughly (level 4), but I simply cannot expect my server to be offline when this happens, and Spinrite only works in offline mode. Gotcha. So I have had several hard drives lying around the office with important data on them, and no way of ensuring that the integrity of the data is being maintained. I can't afford to wait for the drive to fail or the data to be lost. That's a luxury for big companies like Google.
Google have so many hard drives in their data centres that they are always removing and replacing faulty hard drives. They have special machines (shown above) to destroy drives once they fail. Watch from about 3 minutes into this YouTube video.
A few weeks ago, the author of Puran Defrag and the Puran Utilities, Vishal Gupta, wrote to me to say he had started working on a program to solve this problem. I gladly agreed to do some testing, and make suggestions. I made a complete image backup of my test machine, and set to work.
DiskFresh is the result of this collaboration. Vishal did all the engineering, wrote all the code, and put up with my nitpicking, wild ideas, and occasionally useful suggestions. The result is elegantly simple, but extremely useful. You select a drive (either the entire physical disk or a drive letter) and the program methodically reads each sector, and assuming it gets no error messages from the drive, rewrites the data back to the same sector. Bingo! A fresh analogue signal on the drive, and you can rest easy that (for the next few months at least) the magnetic signal is not going to fade away and lose your data with it.
DiskFresh also has a "read-only" mode which is useful for SSD drives and USB Flash drives. Flash memory also has a "data fade" problem, but rewriting the data doesn't help, because you can only write data on a flash drive a limited number of times. Instead, you just read all the data fairly often (4 times a year) and the drive controller should pick up any problems before it is too late.
On my server's scheduler I have set DiskFresh to run every 13 weeks, on a Saturday. It will refresh all 3 physical drives, a total of 4.3TB of storage space. The server can still run while this is happening, and the DiskFresh software detects how much other hard drive activity is happening, and will scale back if the drive gets too busy. It is very well behaved, and doesn't consume lots of computer resources or disk space. On completion of the refresh, you get a text report listing useful information and any errors encountered. FWIW, each 2TB drive took about 22 hours to refresh during testing.
There is a free version that you can use on your home PC or for testing in a business environment. There is even an option to schedule it to run quarterly. The commercial version has some extra features, and the installer allows you to also install Puran Defrag, Puran File Recovery and/or Puran Duplicate File Finder. All of these are essential tools for modern Windows PCs, and it is great to have them in a single setup program. It makes life easier for the IT technician. Choose from a 5PC license for small businesses, or a site license for 20 or more PCs.
Update: For a more detailed look at why DiskFresh is so useful, see my Fact-Reviews article on DiskFresh and why you need it.
Update 4 July 2013: DiskFresh is now at version 1.1, which fixes a bug encountered when using encryption in some cases.

Full disclosure: I was not paid to do any testing, nor do I receive any money for any sales of this product. I am grateful that Vishal listened to my rantings about the hard drive signal fading away, and I am glad to be able to use this utility on my computers: Windows 2012 Server, Windows Vista (laptop) and Windows 7 Home Premium (Test PC).


Unknown said...

Thanks for the heads-up on Diskfresh.
I have used it on a Samsung 840 SSD which was slowing down.

I am using a Dropbox pro account to backup my essential data to the cloud and to my multiple PC
The rest of the data is on the workstation with a copy on a Zyxel NAS which is powered on-off once a week.

Although I could (and probably will) use diskfresh or spinrite for the workstation disk, how should I proceed for the NAS ?

Donn Edwards said...

I wish I had a good answer. If you have access to the drives in the NAS, remove each one separately and run Spinrite on them.

In my case, I can't open the NAS, so there isn't much I can do, other than try to find instructions for how to open the box.

FWIW it is fine to use Spinrite on the SSSD, but only on level 1 or 2.

Warning: The NSA and 4 million other sick weirdos with "security clearance" have intercepted this page and know that you are reading it.