Saturday, November 17, 2007

The Great Defrag Shootout Round 2 - Benchmarks

The image above shows a snapshot of a newly-installed version of Windows XP SR2, with Internet Explorer 7 and .NET Framework 1.1 and 2.0 installed. There are plenty of fragmented files, many of which are caused by the downloaded security updates and patches. The yellow files are fragmented, and the start of the hard drive is the bottom left. The pink area is the reserved space for the Master File Table (MFT).
Here is a more familiar picture of the drive, using WDD. The red files are fragmented, and the MFT reserved space is just shown as empty space.
I have set up my test laptop, an IBM ThinkPad R31 with a 20GB hard drive, and using Acronis True Image Home 11 recovery CD I have made a protected partition for backups, and a main NTFS partition (C:) of 8,497 MB for Windows. There is a smaller partition of around 1GB just for storing results. The C: partition is backed up and restored using a "sector-by-sector" method, so that I can reproduce the messy arrangement of the drive whenever I want to run a new set of tests.
I have installed the following software on the basic test machine:
The idea is to create a "typical" workstation that reflects currently available software. Some of the utilities are used to take measurements, and others were used to set up the workstation the way I normally set up computers I install software on.

Testing the system

The data collection process works like this:
  1. Restore the full drive image, including all fragments.
  2. Install and updated version of prefetch.exe with the correct testing files.
  3. Install the required test defrag program.
  4. Reboot and run the first round of tests.
  5. Allow the defrag program to do its best to defrag the drive. This may take several attempts and several reboots.
  6. Reboot and run the second round of tests.
  7. Install Microsoft Office 2007 Professional trial edition.
  8. Reboot and run the third round of tests.
  9. Open a test document as part of "normal usage"
  10. Allow the defrag program to do its best to defrag the drive. This may take several attempts and several reboots.
  11. Reboot and run the fourth round of tests.
  12. Run HD Tune to test the hard drive.
  13. Copy all the test results off the drive and analyse them.
Each "round of tests" consists of running Prefetch.exe, using this layout.txt file. This is followed by JkDefrag in "Analyse" mode, and storing a screen shot of the drive image, as well as the JkDefrag log file.
The idea is to emulate the normal process of software installation and use. I chose Office 2007 Professional (Trial Edition) because it is a large, legal install, creating numerous new folders and adding a lot of stuff to the system as a whole, including DLLs, fonts, and so on. The defrag program should be able to cope with these changes.
The current Prefetch.exe install also includes an installation of my "Delay Launch" program. A shortcut is placed in the Startup menu, and 240 seconds (4 minutes) after the system reboots it loads the "prefetch.exe" program. The program attempts to measure the read time of 842 files, 39 belonging to Office 2007, and the remaining 802 belonging to Windows and the other preinstalled software.
The benchmark is run at the same time after each reboot, to ensure consistency. If a program like Diskeeper is being tested, the Diskeeper service is disabled before the reboot, so the measurements are not affected by the software being tested.
I used HD Tune to test the speed of the drive under normal conditions after a reboot, and the graph is shown here. I will post the results as they become available, complete with the raw data in spreadsheet form, so others can examine the data and provide comments.

No comments: