Saturday, December 13, 2008

Boot Time Defragmentation

Boot time defrag capabilities have been around for years, but as Windows becomes more complex and file security gets tighter, boot time defrag becomes more useful. The defrag is done while your PC boots up (i.e. at boot time) before all the windows services get loaded, i.e. when the minimal number of files are being used.
My first recollection of a boot time defrag was in an early version of Diskeeper, sometime around 2000. When I bought PerfectDisk 7 it had a better boot time defrag, that sorted out the metadata files that you can't access once the OS is running. PerfectDisk 8 and 2008 also have this feature.
Then there is PageDefrag from Sysinternals, that defrags the registry, hibernation and page files, when it can. I haven't had much success using this on Windows Vista. Not surprising, since it was published in November 2006.
Then there is the freeware UltraDefrag, which can do a much more general job of defragmenting all kinds of files at boot time. I have had great success with this on Windows 2000 servers, as well as workstations. It doesn't manage to defrag the MFT though, but this feature is on a future "things to do" list. I have stopped using UltraDefrag for any other kind of defragmentation, because I don't like the user interface, but it's brilliant for boot time defrag. I have installed it on a few servers so that it can have a go at tidying things up when none of the server's services are running.
UltimateDefrag 2008 has finally released its own boot time defrag module, and claims
"The UltimateDefrag 2008 boot time/system file defrag module is a world first in what it enables you to be able to do when it comes to defragging and moving system files to areas on your drive that further enhance performance."
I have downloaded the new version and plan on running some tests to see how it works. It's a free upgrade for existing UD2008 owners, and until the end of the year the usual price of US$ 39.95 has been reduced to $29.96 for new customers.
The Best So Far
Last, but not least, is my current favourite defrag program, Puran Defrag 5.1. It has the best of both worlds: automatic defrag that you can set to run every now and then, and boot time defrag, which I have set to run the first time I boot up on a Wednesday. The review of this program has been ready for some time, but thanks to my legal hassles it has been delayed. The photo of my laptop screen at the top of this article shows PuranDefrag at work. It is currently selling for US$19.95, which is excellent value for a general purpose simple to use defrag utility.

Update 16 Dec 2008: See KB312067: "Shadow copies may be lost when you defragment a volume". It seems that the Shadow Copy service doesn't work terribly well on drives where defragmenting is being done. This affects any kind of defrag, not just at boot time.

16 comments:

jesse said...

I have to disagree with your assessment of Puran Defrag as the best defragger available. I agree its capabilities outstrip the rest, at least because of its ability to close file interstices (file gaps) and its very comprehensive boot defrag utility, but its unfortunate disregard for Vista's VSS data makes it nigh unusable if one wants to actually use Vista's System Restore or incremental backup feature. I ran a single boot defrag and found all of my System Restore points gone. There's a possibility I made some kind of mistake and didn't check an option which would cause Puran to respect VSS data, but that doesn't take away the fact that my System Restore points disappeared immediately after a thorough run by Puran Defrag.

For a defragger that still works well and actually respects VSS data placement, I stand by your previous recommendation of PerfectDisk 2008.

Donn Edwards said...

I must confess I have no idea what VSS is, or how it works. I find the System Restore facility painful, and have never tried to use it to actually restore something. It's also a favourite tool of virus writers to ensure the malware is unremovable, so I keep it switched off. Hence I did not pick up the System Restore issue.

I have asked the Puran Defrag guys to leave a comment on this post to respond to Jesse's observation. I'm curious to know what they have to say about it.

Anonymous said...

Any comment then about Diskeeper 2009 please!!!
I want to see comparison between Diskeeper 2009, JKDefrag and Ultimate Defrag 2008.
Your comments?

jesse said...

VSS is the Volume Shadow Copy Service in Windows Vista. As far is I know, and granted I haven't done any research to support this, Vista's System Restore feature is pretty secure. I actually emailed Puran and they suggested the loss of restore points was because the amount of data movement caused the reserved space to fill up. Note that this is only an issue on NTFS formatted drives with cluster size less than 16KB due to silliness on Microsoft's part in not designing their monitoring systems properly (whatever). But I've read that VSS data is written to disk in such a state that if it gets moved at all then VSS will invalidate it. This is what happened to me during a boot defrag--I suddenly ended up with about 15GB more space from lost restore data.

That said, apparently using a 16KB cluster size will allow defragmentation to be seen as such and VSS won't make shadow copies of files. Unfortunately, you can't have the boot partition be of 16KB clusters because the Vista bootloader requires 4KB clusters. This is all a bunch of silliness that I think Microsoft would do well to fix, but whatever.

Note: you can see VSS data if you enable System Protection and do stuff for a few days; in PerfectDisk you'll notice a large amount of excluded data toward the end of the drive: this is VSS data and both Diskeeper and PerfectDisk (and probably O&O Defrag, but again, haven't checked) respect its placement and refuse to move it.

Donn Edwards said...

I stopped using PerfectDisk on Vista precisely because there were all these files that it couldn't or wouldn't move, and it was causing the disk to become horribly fragmented with no possibility of getting sorted out, even after I had disabled System Restore.

I don't understand why the System restore files should be dependent on their *position* on the hard drive: nothing else is. To me it looks like another half-baked technology from MS where they haven't thought it through properly. If Microsoft's own defrag program can screw it up (which is what KB312067 says), then it's completely unreliable and people shouldn't be thinking of it as a safety net. Remember that in Vista the defrag utility runs automatically, so it is systematically destroying the system restore stuff.

Have a look at the Wikipedia Shadow Copy article.

@jesse: You have pointed out an important problem that ALL defrag programs need to address, and I don't think you should be too hard on PuranDefrag. They have promised to do further research and respond to this issue.

FWIW, O&O Defrag 10 does not support VSS, according to the review in Windows IT Pro magazine. I could find no mention of it on the O&O web site.

Donn Edwards said...

@anonymous: I absolutely refuse to install Diskeeper on my laptop. I will try it out on a test machine, but don't expect it to be any less hyped than before, and any more disappointing than previous versions.

FWIW I have not installed PerfectDisk on my laptop yet, even though I've had the machine for 6 weeks already. On my last laptop I installed if within 2 weeks, but that was before I bought Puran Defrag.

I will begin a new round of tests as soon as I can get hold of a decent test system. My Vista laptop is not going to be the test system any more.

jesse said...

Donn, you and Puran support have been great resources these past few days. I've pretty much decided that since I never used System Restore in the past--not because it didn't work, but because I'm pretty good at not screwing up my computer--I'm going to just turn it off and use Puran Defrag. It seems to work faster and more completely than either Diskeeper or PerfectDisk.

Speaking of which, I'm going to run a trial of Diskeeper 2009 just to see how it's doing these days. I last used 2008 and it worked, though not as well or as quickly as I'd prefer. I honestly don't think it's really going to make much of a difference, but hey, what can it hurt to check? Err . . . perhaps I shouldn't ask a question like that. It kind of seems like tempting fate. Oh well. I'll make a comment or something in a week or so, since Diskeeper has perhaps the slowest defragmentation engine around.

Unknown said...

Donn, next time you are on Vista fire up command prompt(as admin please) and type in vssadmin. That will list all the options you can do with this tool. You can see where you have those "shadows" and set the size of them.

After you've done fiddling with that fire up disk cleanup and select files from all users on this computer select the desired partition and under more options you will see System Restore and Shadow copies. Click the clean up button beneath that . ;)

BTW as far as i know O&O(ver. 11 but i am not sure about this one), Diskeeper and PerfectDisk all have a VSS compatible defrag mode. I use PD and i've encountered no problems with VSS so far. :)

If you need any more Vista tips just ask. I'll be more than happy to help.

Cheers !

Mikey

Unknown said...

Just an after thought ... It might also be a good idea to turn off the Volume Shadow copy service.

Anonymous said...

Hello,

Thank you for your patience.

We have confirmed by our tests that a defragmenter can on the max just do minimal data movement to avoid loss of Shadow Copies and this is only called Shadow Copy Aware defragmentation.

Puran Defrag can defrag this way if you simply disable Additional Operations. There is no need to exclude any file for this purpose as the files containing Shadow Copies are already not accessible. In your post at Donn Edwards blog, you referred Perfect Disk excluding some files, we are sure that they are excluded just because they are not accessible. Here is a link to PDF by Raxco which states its VSS working http://www.raxco.co.uk/file.asp?FileID=553, Please read Page 11, upto 3 paragraphs

So no defragmenter can fully optimize your disk if you need VSS saved.

To confirm it, you can create shadow copies and then defrag that drive with Puran Defrag with all additional operations disabled. You will note the difference.

NOTE: The above information is limited by our knowledge.

Thank You

Regards,
Vishal Gupta
Puran Software

Donn Edwards said...

I have posted the comments sent to me by Vishal at Puran Software.

I plan to do a bit more research on this and then write an article. I have recommended PuranDefrag 5.1 as my favourite defrag program, but I run my machine with System Restore turned off.

Anonymous said...

Not one of the available programs will defrag Vista MetaData if you have the page file on a separate drive.

Anonymous said...

Boot defrag does not work, it does not detect the custom managed pagefile on windows vista, no body else can do the job of defragmenting the custom managed pagefile on windows vista except PD.

Donn Edwards said...

@Anonymous:

Which program doesn't work? How did you create the custom pagefile? I'd like to test this for myself.

Anonymous said...

Guys,

all that jabber about fragmentation will end if you get a SSD HDD ( as I just did =).

Never ever have to defrag crap!

HEIL SSD!!!

Donn Edwards said...

@Anonymous

I envy you. I have wanted to get an SSD for my laptop for some time. If you are running Windows, make sure the automatic defrag is turned OFF. In Windows 7 it is on by default. Turning it off will improve the life of your SSD drive.

One word of caution: it is a good idea to run chkdsk /R from time to time on your drive (SDD or HDD) because it causes the entire drive to be *read*, and alerts the drive's own internal error handling to deal with any serious errors by itself.