Friday, May 23, 2008

UltimateDefrag 2008 Debate: I

Regular readers of this blog already know that I had an unfortunate spat with DiskTrix over their marketing emails. Since it was a genuine mistake I have removed the article. In the meantime I have had an interesting conversation with Robert Ferraro about UltimateDefrag, file placement, benchmarks and the like. I will include some extracts here, and hope to add some more soon. He writes:
"I just read your quick spiel on Defraggler before. You said that you’ve never come across a program that could analyze any faster than Defraggler.
"I am sorry but this is highly untrue….
"On a 500 Gb drive with 348 Gb and 405,001 files this is the result of my analyze:
Defraggler: 160 seconds
UltimateDefrag: 37 seconds
"We analyze a drive 432% faster than defraggler or in other words in 23% of the time – less than a quarter. When you minimize our display – we actually analyze that same drive in 33 seconds.
"I can assure you that with 3 years of perfecting our analysis routine – it’s as overhead free as you could get by all coding standards.
"Please note also that with UltimateDefrag – Fragmented Files Only (FFO) …. We can get you from program start to completely defragged much faster than Defraggler since we save all that time on analysis and you don’t need to select files manually. Of course one feature of UltimateDefrag is that we can defrag individual files or just all files in a folder for that matter. So while we have all of our advanced features – the most simple of all defrags – Fragmented Files Only – is there too.
"One “secret” about our Fragmented Files Only routine is that for this most basic of all defrag routines, our algorithm functions in the favor of promoting consolidation of free space. In FFO mode we defrag files in LCN order – meaning it finds the first fragmented file towards the outer tracks and puts it in the nearest available free space. If the space is now cleared then the next fragmented file, if it fits will be placed in an earlier space – so it’s always, where it can, tending towards the outer tracks and filling free space from outer to inner.
"There’s not much we haven’t thought about with our product – any true purist will see that our product does much much more than initially meets the eye and for defragging is as close to the holy grail for defragging as you can get. Our list of to-be-included features stands at around 86 more features to be added at this point."
In fairness to Defraggler, it is the quickest utility currently installed on my laptop, and I only have two 30GB partitions. I have noticed that it is slower on other drives. However my experience of UltimateDefrag 1.52 in June last year was not good: it took over 14 minutes to analyse my drive, because there was a large (>4GB) compressed file on the drive. I wrote to Robert about this, and was told to try various options, but he was unwilling to admit that this might be a problem. I plan to do some tests to see if the problem has been fixed.

Update: I obtained a license for UltimateDefrag 2008, and tried a quick test (see comment #1) on my Iomega 500GB external USB drive: I timed the "analyse" times (I ran each one twice, and used the second result) for the defrag programs currently installed on my laptop:
Program"A""B"
Windows Disk Defragmenter (WDD) for XP SP3
20.0317.65
Defraggler 1.01.068 beta23.1736.50
UltimateDefrag 2008 ver 2.0.0.48 26.53 176.56
PerfectDisk 2008 build 5244.8134.68
JKDefrag 3.3447.3147.28
PC Magazine Defrag-A-File 1.153.5649.09
UltraDefrag gave an error, and PuranDefrag doesn't have a separate "Analyse" option.
Column "A" refers to the analyse times (in seconds) of the drive with a several large files, including a 9GB data file. Column "B" refer to the analyse times with the same files, but an additional copy of the 9GB data file, this time stored as a compressed file, containing 123235 fragments.
This confirms the behaviour mentioned in the UD1.52 review, but not nearly as bad as before: 3 minutes compared to 14 minutes.

Update 2: I noticed during the tests above that some programs took longer to load than others, so I did some timing (see comment 1).
Programload time (sec)
Windows Disk Defragmenter (WDD) for XP SP3
<3
Defraggler 1.01.068 beta<3
UltimateDefrag 2008 ver 2.0.0.4810
PerfectDisk 2008 build 529
JKDefrag 3.34N/A
PC Magazine Defrag-A-File 1.1<3
Paint Shop Pro 8.108
Microsoft Word 20023
Microsoft Access 975
Microsoft Excel 20024
Microsoft Visual Basic Professional 69
These numbers are not particularly scientific, since it isn't easy to measure with any degree of accuracy, but it does confirm my feeling that UD2008 and PD2008 are larger, slower apps than the other smaller ones listed. Bigger is not always better.

Update 3: Rob (see comments) asked me to do the fragmented file test on my laptop drive. I couldn't manage the 9GB file because of space limitations, but did manage to create a 6GB file that was compressed, with 476,192 fragments. My timing results (not scientific) are shown here:
Program Analyse Timesmins:secs
PerfectDisk 2008 build 521:01
JkDefrag 3.342:00
Defraggler 1.01.073 BETA7:53
UltimateDefrag 2008 ver 2.0.0.4821:44

This is not a typo: UD2008 took nearly 22 minutes to analyse the drive. It takes 20 seconds to load the program on my machine, partly because it takes time to draw a picture of the drive.
As soon as I deleted the 6GB file, UD2008's load time dropped to 8 seconds, and the analyse speed dropped to 18 seconds.

4 comments:

Unknown said...

Sorry Donn but in Update 1, it's interesting how our product's result, UltimateDefrag 2008, seems to stand out like a sore thumb amongst all of those figures - almost too much of an abberation caused by just one highly fragmented 9 Gb file.

My official trained profession is that of a medical laboratory and research scientist.

When you take a scientific reading you would firstly standardize all conditions and you would then take triplicate measurements and take the average of those triplicate readings. You DO NOT take the last reading because it's merely the convenient one to take.

You also wouldn’t run such a performance test on an external drive. What you have on an external drive is a severely rate limiting throughput of the USB connection. A test like this should only be run on a regular internal hard drive to have any credibility.

Please do this – add a bunch of files to your “internal” hard drive. You can also add the highly fragmented 9 Gb file too.

If you are going to compare six products in two configurations you are going to reboot your system 36 times.

So reboot your system, preferably in safe mode, wait exactly 5 minutes for things to settle every time. Don’t do anything different at any time between each test after your system boots up and then run the tests again.

Remember triplicate readings and take the average of the readings. If one of the readings is vastly discrepant from the other two then you need to discard that result and run that test again.

I think you’ll find under normal circumstances and a properly standardized test our product will not perform anywhere near as bad as you have made it look and will actually most likely be #1 on the list.

Be careful to not publish such non-scientifically created results. Reader Beware!

Update #2.
How can a 3 Mb program take 10 seconds to load? Again something horribly “wrong” there with your system or methods. UltimateDefrag, when you launch it, pauses for 1 second, to allow for checking of updates which it does every 7 days. Only 1 second. I just loaded the program now – and it took just 1 and a half seconds.

If it's taking you 10 seconds to load are you sure you're not running it on an old 286!?

Donn Edwards said...

I chose the external drive precisely because it is slower and would show up any speed differences more easily.

I chose the second result because I wanted to "equalise out" any caching effect caused by running the previous application.

I did say that the results are not scientific and I am not trying to publish definitive results, but thanks for the reminders on accurate scientific methods.

My laptop is an Acer TravelMate 2451WLMi, Intel Celeron M 410 (1.46GHz, 533MHz FSB, 1MB L2 cache), 60GB HDD 5400rpm, 512MB DDR2 RAM.

If you would like to do the tests on a standardised lab machine and send me the results when you eventually get them, kindly forward me the results and I'll publish them with pleasure.

Anonymous said...

Hello,

I am one of your avid reader. To be honest, I love your articles. All of them.

I had tried perfect disk, diskeeper 2008, contig, jkdefrag, auslogic, ultimate defrag (previous edition) and ultimate defrag 2008. Thanks to your articles which spark my interest to test them all.

However, base on my personal experience, Ultimate Defrag 2008 load way faster than yours. My PC spec just average, Core 2 Duo, 2 GB of Rams, 500 GB hard disk (with several partition ; the biggest one is 203 GB which I store 'raw' software, movie, games images etc).

UltimateDefrag2008 load on my system less than 2 seconds. And it can analyze the whole 203 GB partition in about 3 seconds only. So, when it take more than 20 minutes on your system, I was called to give you this feedback.

And regarding that ultimate defrag has some difficulties to handle multi giga files, I have to agree on previous version of UltimateDefrag. But on 2008 version, seem they manage to fix that. As you may know, mostly current game images as big as 7 GBs nowadays, and UltimateDefrag could defrag it without any much problems on my system. I can't commend how it tax my system while defrag those images because most of the time, when I defrag (manually), I leave my PC (dinner, lunch or something).

So, base on my observation, UltimateDefrag (both version) loading time just about the same with other defrag programs loading time plus minus 1 second. And analyze time required, as long as it below 5 seconds, I consider it just as fast or slow as the rest.

I hope you could try again on your internal hdd and share your finding with the rest of us.

Note : If you want me to take 'fraps' video regarding loading time, and analyze time, I will do it for you.

Just sharing my experience.
PS : Test bad OS = Windows XP Pro SP3.

Anonymous said...

to expertester

Hi,

we are still. speaking about compressed file, as i understand. How many of your files are compressed this way?