Monday, November 26, 2007

Benchmarks*: PerfectDisk 8.0.0.67

You can find a review of the $39.99 PerfectDisk 8 program (PD8), as well as the benchmark results obtained. I was disappointed that the times were not faster than WDD, and somewhat bewildered. PD8 does a very thorough job, and can defrag drives that WDD is incapable of doing. I am at a loss to explain why the numbers are what they are, or why the times measured were so slow. My laptop never "feels" slow after using PD8, so these numbers seem to contradict my experience of using the product. Weird.
The graph shows PerfectDisk 8 (PD8) in dark blue, and the results from Windows XP in light blue. The first test is at the bottom, and shorter lines mean faster times.
  • "Basic XP" refers to the standard install, without Office 2007, so there are only 710 files to be tested.
  • "Defrag" refers to the read times of the same 710 files, after two defrag passes using the PD8 program, with no "Optimize Files" or "Optimize Boot" options enabled, allowing PD8 to manage the boot files itself (recommended). A 12% performance improvement is measured, not as good as the WDD result.
  • "Defrag+Layout.ini" refers to the read time after enabling "Optimize Files" and "Optimize Boot" functions in Windows, and the Windows LAYOUT.INI option in PD8. An improvement of 20.9% is recorded, still slower than WDD.
  • "XP+Auto" is the result obtained after enabling both "Optimize Files" and "Optimize Boot" functions in Windows, and then running WDD, i.e. the best that Windows XP can manage.
  • "Basic Office" refers to the read time of all 802 test files, where no defragmentation has been done whatsoever, after the installation of Microsoft Office 2007 Professional (Trial).
  • "Full Defrag" refers to the read time of all 802 test files after several reboots and two passes using the PD8 program, with no "Optimise Files" or "Optimize Boot" options enabled, using the "SMARTPlacement" defrag option, and allowing PD8 to manage the boot files itself (recommended). A 23.4% performance improvement has been measured, still 6.3% short of the "Office Auto" below.
  • "Full+Layout.ini" refers to the read time after enabling "Optimize Files" and "Optimize Boot" functions in Windows, and the Windows LAYOUT.INI option in PD8. This performance improvement of 26.8% is still 1.6% slower than "Office Auto".
  • "Office Auto" is the result obtained by allowing Windows XP to do its own defrag after enabling both "Optimize Files" and "Optimize Boot" functions.
The performance improvement ranges between 12% and 26%, with mixed results obtained when the built-in Windows functions are employed; "Optimize Boot" is enabled by default. Still, an average 20.8% performance improvement over no defrag at all is pretty good, better than DK2007's average and not far behind Diskeeper 2008 Pro Premier, which is $60 more expensive.
Here is a picture of the drive before any defragmentation was done. Compare this with the top picture, which is the end result of the "Full Defrag" step. PD8's main strength is its ability to organise the files using its "SMARTPlacement" method.
Even though faster read times were obtained using both the "Optimize Files" and "Optimize Boot" options available in Windows, the "Optimize Files" option causes boot files to be moved away from the start of the disk, and is therefore not recommended.
Notice how the purple startup files are scattered around the disk if you choose the "Let Windows manage the layout.ini files" option. Over time this will get worse, not better, and the small performance gain at startup will be lost.
PD8's boot time defrag is highly effective at defragmenting critical system files and metadata, but it would seem that the "SMARTPlacement" method is not as effective at fast startup read times as some of the other methods available, even if it does manage to organise the files in an effective manner. Further tests will tell how PD8 and other defraggers fare with data files and prolonged use.
I have started testing JkDefrag and will publish these results next.

9 comments:

GoOrange said...

What's going on here? Most programs tested are producing at best minimal improvements over XP and in most cases no improvement or worse results. Are commercial defrag programs really just a sham in the end? Is there really any reason for me to keep perfect disk on my system when I have the XP defrag program?

Very interesting results, although at this point, I'm still not sure what to make of it all.

Cheers.

Donn Edwards said...

I share your confusion. Bear in mind that the XP results were done on a machine that was not being used for anything - it was just "hanging around" waiting for the defrag to finish.

I don't think defrag programs are a sham, but I think that my testing methods aren't able to reproduce the situation experienced by most users.

I guess we're still looking at the difference between "lab results" and "real life".

Anonymous said...

Don't forget the OS file cache will cache most recently used (read and write) files, frequently small temp files do not even get written to disk (note they always get allocated and do show on the file system)
The cache on the actual drive will reduce head seeks when reading writing, and some of the more modern drives have intelligently queueing to group requests together.
Caching helps reduce fragmentation as files are written and also performance hit of reading a fragmented file.

Also the fact that drive access is quite random (in general use), and seldon is an entire file read end to end (again in general usage). Check out filemon from sysinternals, I was quite surprised at how often the page file is read in-between reading files.

Then finally we have the fact windows (as default) optimises boot and commonly used apps (prefetch), in general normal desktop use, it will take a long time to get. So from a boot up point of view, fragmentation should have minimal impact on boot times as the prefetch (as default) does a good job.

Anyhow this is why you can get weeks and months worth of usage and fragmentation does not increase significant AND also cause any significant fragmentation in general usage.

Keep up the good work Donn ! Nick.

Anonymous said...

Donn,
Thanks for continuing to give defragmentation what we feel is the attention it deserves. You are doing a great job.

As a representative of Raxco Software, I wanted to comment on your post. As you point out, there appears to be a difference between the lab and real life. And this is why we strongly encourage users to test products (including ours and our competitors) in their own environments, and that’s why we encourage people to use even our white papers and other literature only as a basis for their own investigations.

Regarding your tests, I wanted to make a few specific points:
• Write time is a very important piece of drive performance, and this piece is missing from your testing. Write time is affected by free space consolidation, or lack thereof, and this is why PerfectDisk places a high emphasis on it. As pointed out in a larger testing environment by Balder Technology Group (http://www.raxco.com/products/perfectdisk2k/whitepapers/FreeSpaceConsolidation.pdf), free space consolidation is just as important as file defragmentation in the overall concept of disk defragmentation. And without proper free space consolidation, write times will suffer, as pointed out in these tests by Balder. Balder is headed by David Goebel, who was part of the original team for Microsoft that architected, designed and implemented NTFS.
• We cannot tell exactly how your “open” statement actually opens files (i.e. how it caches and accesses). We did see that PreFetch.exe uses TimeGetTime, which can have a practical resolution as high as 10ms (http://support.microsoft.com/kb/172338), which could potentially degrade results to the point of being nearly random. Using non-cached I/O may also increase the contrast of the results. And, using the QueryPerformanceCounter API will give a 100ns (nano seconds) resolution, or approximately 100,000 times better.
• A benchmarking tool written on pure Win32 C would likely give more accurate results, as there is a random latency inside the script engine itself which introduces additional discrepancies.
• The number of files in your test was relatively small. In our experience, with a high volume of heavily fragmented files, PerfectDisk will outperform the built-in defragmenter.
• In an environment with a large number of large files on a very large drive with limited free space, PerfectDisk will outperform the built-in defragmenter, in our experience.
• The built-in defragmenter does not defragment all the NTFS metadata files on a drive, resulting in a less defragmented drive. Without this turning into a marketing discussion, you can see on our website as well as independent web sites like CNET and Microsoft user groups, there are countless real-world experiences from people who have not been able to achieve satisfactory performance gains from the Windows defragmenter (or other free or other commercial defragmenters for that matter), and have only achieved real, noticeable and verifiable (in their environments) gains after running PerfectDisk.

Do the built-in and freeware utilities defragment files? Yes. Do they do as complete a job, leading to significant, sustainable performance gains, in real-world scenarios, including mission-critical, high-activity servers and high-activity workstations, from a read and write standpoint? Our experience, and our millions of users around the world, says no.

We love the blog and all your efforts.

Thanks,
Joe Abusamra
Vice President of Operations
Raxco Software, Inc.
www.raxco.com
www.perfectdiskblog.com

Donn Edwards said...

Thanks for a detailed and helpful response. It is much appreciated.

I wish I was a C programmer! I plan to develop some more tests, to deal with the read/write question. I also think that PreFetch.exe should distinguish between seek time and read time.

Once all the results are in I'll examine the data more carefully and see if there are any patterns that stand out.

Thanks once again!

Anonymous said...

Hello Donn! Although I don't have a technical contribution to make to your research, I really want to thank you for all the time and energy you put into your testing. I've become an "Insights and Rants" addict and really admire the "attention to detail" you put into your work.

PerfectDisk 8.0.0.67 seems to be my choice of "defragmenters" so far, keeping my desktop and my notebook both running smoothly. Even though it does the job considerably slower than all the software I've used (and I've tried quite a selection!) it offers the best results and most relevant features. Vopt8 falls in right behind, but the program definitely seems a little "flimsy", with some energy wasted on features that I would rather see dedicated to the defragmentation process. It also crashes on my Vista machine when I attempt to resize the program window. The speed of the defragmentation process is amazing though.

Diskeeper 2008 and O&O 10 offer too much fluff for too little results. I wasn't particularly happy with either piece of software and the uninstallation process of O&O can leave one's system with some issues (amazing what a stray .dll file can do for your system!)

JKDefrag is great! I've used it via the GUI and "straight up". One can't beat the price and it comes in quite handy for a touch up. Most of the other freeware defraggers aren't worth the time, although I'm a Contig fan as well.

Well, enough babble from me! Keep up the good work and know that your efforts are greatly appreciated.

Rick Eisenhardt
rickeisenhardt@gmail.com

Anonymous said...

Donn.
Which layout.ini settings did you use ?

The default setting in PD8 and PD2008 only optimises boot files.
Changing this to the 2nd option "Let PerfectDisk manage all the layout.ini files", will optimised placement of boot and app files and should see noticeable improvement.

Anonymous said...

http://perfectdiskprousers.ning.com/forum/topic/show?id=741025%3ATopic%3A8342

Hi Donn,

I am not a specialist but just wanted to say thank you for sharing your toughs with the web community.

I have posted a reply to your comment in Pefectdisk blog, keep the digging it help us the normal user to have a better experience.

Michel

Anonymous said...

http://www.maximumpc.com/article/the_disk_defrag_difference?page=0%2C2

Shame on all the 3rd party defraggers. They are actually slowing down the system.
Vista´s built-in defrag does the best job.