Saturday, December 27, 2008

Benchmarking and Software Decisions

I have been setting up FRAGG, my benchmarking PC, with software and data in preparation for "". That's the ZAPS logo on my desktop. I added it because I was so impressed with their prices and service. So far I have managed to fill up my 149GB hard drive with a lot of software, and test data. But I am still mystified about which benchmark software, if any, is capable of measuring the effects of file fragmentation on the system in a meaningful way.
Do I have to do "The Great Benchmark Shootout" in order to do "The 2009 Defrag Shootout"? I hope not. But I have a nasty suspicion that I'm going to have to learn a lot about how these benchmarks work before I can get anything meaningful out of them.
One thing I have noticed about Vista Home Basic: the hard drive seems to have a life of its own, partly because the page file size keeps changing. I guess this tells me that 1GB RAM is not enough for many of these benchmarks. I have installed the very useful HardDriveIndicator utility to show whether the hard drive is busy, because the HDD light isn't easy to see from my desk.
Another change to my normal behaviour is that I have forced myself NOT to empty the Recycle Bin or clear any browsing caches. So far I have accumulated 8,515MB of deletable files. It is going to be interesting to see what the various defrag programs make of this. I have also created some compressed folders, and copied several large files, greater than 4GB, to the disk, some compressed, some not.
The data files are a collection of archived podcasts, audio books, Word and Excel documents, PDF files, scanned images, photographs and recordings, made by my wife and I over the last few years. There are recent files and old files, and sizes ranging across the board, in various folder structures.
At the moment the system is a mess, and it sometimes takes a several seconds for the "Programs and Features" list to open (see below). I'm trying to get it to look like a typical home user's PC, or a badly maintained office PC. Normally I use NOD32 as my anti-virus program, but this time I'm using the freeware version of AVG, because it is extremely common, and because the automatic scan of all the files takes over an hour. I figure this may become a useful benchmark all by itself.
For fun, I installed all the major browsers (IE7, Firefox, Chrome, Safari, Opera), and all the Windows Live and Live Mesh stuff, except Messenger. Then I added Microsoft Office 2007, 3, Windows Media Player 11, RealPlayer 11, iTunes 8, Quicktime, etc. I also added some extra fonts, the SysInternals Tools, DirectX, and a bunch of other stuff. I used Firefox's DownThemAll add-on to download most of the software, and copied other files onto a shared folder while this was happening, to attempt to get further hard drive fragmentation.
I have purchased PCMark05 (WinXP, Vista) and PCMark Vantage (Vista only), since they claim to be able to benchmark the system's hard drive, and these utilities were used to trash the performance of several defrag programs by MaximumPC. These utilities claim to be able to determine the boot time of WinXP and Vista, although they don't actually reboot the PC. Benchmarks used by the 3DProfessor for their Diskeeper propaganda include SpecViewPerf 10 and SiSoftware Sandra 2009 SP2.
Further research on MajorGeeks and have led me to numerous other utilities for benchmarking, including the PassMark Performance Test 6.1 and IOzone 3.311. Here is the completed software list:
Now I have to back up the disk image with Acronis TrueImage and then start all over again with WinXP.


Anonymous said...

Hello Donn! I have just finished reading your article.

I was thinking that it would be nice to include the startup time of some common applications, as well as time it takes to open large and fragmented documents (e.g. doc or pdf), as part of your benchmark. What are your thought on this?


P.S. Sorry for my bad English: it is not my first language.

Donn Edwards said...

These are good suggestions and I plan to do some tests like this. I'm also going to write a Microsoft Access "Stress test" program.

I look forward to other suggestions and comments.

JSComputerTech said...

PCMark Vantage obtains the startup mb/s via a file access trace. I think there is a free application that can also record and play back traces I just can't seem to remember where to find it.

Also this application looks interesting for benchmarking file system performance.