South African Google Earth

(Click for a larger picture)

South Africans have developed the ability to laugh at their problems, even under the most adverse circumstances. Our crime problems are being surpassed only by the random power blackouts that have been going on for months and are likely to continue for the next 8 years or so.
Today I visited my friend Dennis who is recuperating from gunshot wounds he received from 3 armed men, while out walking his dogs with his wife on Boxing day. One bullet narrowly missed his heart, went through his colon and liver. The second bullet injured his left leg, before lodging behind the knee of his right leg. Three weeks later he can barely walk, so I popped round to his house to help him set up some computers in his wife's study.
We couldn't test everything immediately, because of the 3 hour power blackout (Eskom calls it "Load Shedding") from 4pm to 7pm (their second one today) so I did a lot of running around with torches and candles first. I also sat through 90 minutes of traffic congestion to get there, caused by the power outages and rainy weather.
In case your're wondering what "Google-letu" is all about, Guguletu is a well known "township" area.

Power Cuts are a National Disgrace | Fifteen Years Too Late

Flash Drive Defrag Warning

I admit I've done it myself without giving it a second thought, but defragging your flash drive makes no sense at all. What's more, it will shorten the life of your drive.
Fragmentation is an issue for hard drives because they are mechanical devices with seek times and (to a lesser extent) varying read speeds. If you could reduce the seek time of a hard drive to zero, there would be no need to defrag the drive, because every sector of the drive would take just as long to read, so the performance improvement of rearranging the sectors would be zero. In a mechanical drive this will never happen, simply because the laws of physics tell us that any object with mass and velocity requires time to reach its destination, and time to change course.
In any RAM-based system, the rules change. It takes just as long to read one memory address as another. If it didn't, then our software would behave very differently. So if you want to read 2k from memory address A00000 it will take as long as if you wanted to read it from address B00000 or 000000. There is no "seek" time, just a "read" time. Similarly, reading 4k from two different addresses (2 x 2k) will take as long as reading them from adjacent addresses.
I have ignored the effects of caching for the purposes of this example, because I am assuming the worst case, where no caching has occurred, either for the hard drive or the RAM drive.
So why did I defrag it the first time round? I guess I associate "defrag" with "tidy up". There used to be a Norton Utility called DS for Directory Sort, and on slow DOS systems with a lot of file names in a single folder, DS helped quite a bit. Nowadays Windows does the sorting for you, so provided the directories are not too long, you don't even notice the sort order of a folder.
Now for the fun part: Flash Drives have a limited lifespan: typically 100,000-300,000 writes. There is no restriction on the number of times you can read the drive, just write it. On a well-designed drive, this means you would have to write new information to the entire drive 27 times a day for 10-30 years before the drive is exhausted. On a cheap or badly-designed drive, the story is different. Any change to the size (in clusters) or location of the file affects the File Allocation Table (FAT), so the FAT gets rewritten each time such a change is made. On a badly-designed flash drive, the FAT is rewritten to the same memory location each time, so that individual memory area will fail after 100000 writes, resulting in data loss. Most flash drives use the FAT32 file system.
Bear in mind that a defrag program changes the FAT after each cluster (or group of clusters) is moved, so that defragmenting a file with 27 fragments with resut in 27 changes to the FAT, and therefore 27 writes to the same area of memory. This is done to prevent data loss due to a reset or power failure. Defrag programs typically remove hundreds if not thousands of fragments, so a defrag pass has the potential of creating thousands of write requests to the FAT, as well as all the other reads and writes going on in other areas of the drive.
If your defrag program removes 100000 fragments, it has the potential to permanently damage the flash drive! Without wanting to sound alarmist, it is something to consider, and since you have no way of knowing whether your flash drive is well designed or not, it isn't worth the risk. Why waste time defragging to gain 0% performance improvement in exchange for the risk of 100% data loss? The technique for avoiding this situation is called "wear levelling", but not all manufacturers use it. I found the following description

Longevity/Lifespan
Unlike DRAM, flash memory chips have a limited lifespan. Further, different flash chips have a different number of write cycles before errors start to occur. Flash chips with 300,000 write cycles are common, and currently the best flash chips are rated at 1,000,000 write cycles per block (with 8,000 blocks per chip). Now, just because a flash chip has a given write cycle rating, it doesn't mean that the chip will self-destruct as soon as that threshold is reached. It means that a flash chip with a 1 million Erase/Write endurance threshold limit will have only 0.02 percent of the sample population turn into a bad block when the write threshold is reached for that block.
The better flash solid state flash drive manufacturers have two ways to increase the longevity of the drives: First, a "balancing" algorithm is used. This monitors how many times each disk block has been written. This will greatly extend the life of the drive. The better manufacturers have "wear-leveling" algorithms that balance the data intelligently, avoiding both exacerbating the wearing of the blocks and "thrashing" of the disk: When a given block has been written above a certain percentage threshold, the solid state flash drive will (in the background, avoiding performance decreases) swap the data in that block with the data in a block that has exhibited a "read-only-like" characteristic.
Second, should bad blocks occur, they are mapped out as they would be on a rotating disk. With usage patterns of writing gigabytes per day, each flash-based solid state flash drive should last hundreds of years, depending on capacity. If it has a DRAM cache, it'll last even longer.

This quote sounds optimistic, because it is written by a flash drive manufacturer. I've used several USB Flash drives, (and lost 2 of them) and the chance of data loss by misplacing the entire device is higher than wearing it out, but the point I'm trying to make is that flash drives are not designed or intended to be defragmented.
Hard drives wear out too, but defragging can help extend the life of a mechanical hard drive, not to mention improve performance. The same is not true for flash drives. You have been warned!

Merry Christmas from Musica - NOT!

This is the first and last time I ever try to buy anything from Musica.co.za. I ordered Johnny Clegg's "One Life" CD and a live DVD from them on 22nd December, hoping to have it delivered to my brother in Port Elizabeth for Christmas. Granted, it was a last minute purchase, but after 2 weeks they have still not delivered anything. Thanks for screwing up a Christmas gift for me, Musica! This is the way to grow your business and build customer loyalty!
By contrast, I had ordered the same CD from Amazon.co.uk, had it gift wrapped and delivered to my other brother in the UK. Both orders were placed within 30 minutes of one another. The Amazon web site told me there were only 2-3 items in stock, and promised delivery before 1pm on 24th December. Since it was Christmas Eve I thought this was a little optimistic, but it arrived, gift wrapped with a card, on 27th December (the next working day). Pretty darn good for an order placed on another side of the globe during the Christmas rush. Dozens of things could have gone wrong; but Amazon got it right.
That reminds me of kalahari.net, our local bookstore and music outlet. We have bought several books from them, and their service is fantastic. I looked first on their site, but they didn't have the Johnny Clegg albums. I also tried the Look & Listen site, but it told me they didn't have stock, and their aren't any outlets in PE.
I used the Musica site to find shops in the PE area and phoned them to find out if they had stock: they didn't. That's when I made the mistake of ordering from their web site. I assumed that since it took my order that they had stock and would be able to deliver. How wrong I was.
After placing the order I got an order confirmation on Saturday 22nd at 9am, stating

"Your order has been received and is being processed. Your gift recipient's parcel will be shipped within 1-2 WORKING DAYS depending on availability."

On Monday at 1pm (their first working day) I got another email that said

" ... our online warehouse is unfortunately out of stock of both these titles.
"We have placed an order to obtain the stock directly from our stores thus there may be a delay in the shipping of your order."

Since then I have heard nothing from them. I called on Thursday 3rd January (their 6th working day) and was told that the DVD was available but the CD was out of stock, and they offered to send the DVD in the meantime. I decided to wait and see if I could get the CD myself.
I looked on the Look & Listen site and found they still didn't have stock, so I phoned our local outlet and then their Hyde Park store. They had one CD left, which they kept aside for me until I could pop in and buy it.
This morning I phoned Musica Customer Services (0860-687422) again to find out the status of order number 50070. The lady who answered the phone was much more helpful and explained that the order is automatically cancelled after 7 days, but they had had the CD in stock for a few days but the DVD was unavailable until their suppliers reopened. I asked why I hadn't been told any of this and all she could do was apologise. Their was no explanation for why they hadn't bothered to contact me to let me know that the order had expired, or that they had located the CD.
I asked her to get the Managing Director to call me. She said he was still on holiday but their IT Director would call me on Monday; she said she was the Customer Services Manager and that customer feedback was important to them. She also said they had a banner on their web site advising customers to place their Christmas orders before 16th December, to give them 5 working days to deliver everything. There isn't much point telling me that on 5th January!
I count several system failures:

• The web site didn't/couldn't tell me that there was no stock. The best it can manage is a hugely over-optimistic "Usually shipped within 1-2 working days". Don't believe it.
  
• The second email didn't warn me that there may be a much longer delay in obtaining stock. "There may be a delay" says nothing.
  
• I was not told that my order had expired after 7 days
• I was not told that the CD had been located and was available for shipping.
  
• I was incorrectly told that the DVD was available, and was not warned that the order had already expired. I only discovered this by calling them.
  
• There was no record of my first call when I made the second call.
  

These guys clearly have no idea how to run a web site. Rather use kalahari.net (which is brilliant) or try Look & Listen. You have been warned!
Update Mon 7th January: The MD contacted me, and promised to send me his contact details and I asked him to post a comment on this blog once the system has been improved. Stand by.

Electricity Situation Gets Worse

Eskom asked for an 18% price increase, and has been given 14%. So much for the government protecting the consumer or caring about business.
How we have to put up with "load shedding" (i.e the power goes off for 2 hours in randomy selected suburbs) and free electricity to Zimbabwe as well as higher prices and general technical incompetence. Fires at generating plants and substations are usually caused by poor maintenance or obsolete equipment. One fire resulted in the loss of 500MW generating capacity. Then there was the Koeberg debacle which saw rolling blackouts happening for months.
How can this problem be solved? Some people blame it on bad decisions, others blame it on affirmative action leading to the employment of inexperienced engineers. I say that whatever the cause, our current leadership will blame it on Apartheid and do nothing else. It's a failure of power, plain and simple. That's why I think it is appropriate to drop the "e" from their logo. It better reflects what the public feels about them shedding their responsibilities.

Charismatic Dimwit

It seems to me that our country is generally run by a*s*holes, and this has been going on for decades, with one notable exception: Nelson Mandela. All the other presidents in my lifetime have been a national embarrassment. I include in this lineup notable disasters like BJ Vorster, PW Botha (the absolute worst that Africa can offer), FW de Klerk and Thabo Mbeki.
If PW and FW had their way, I would have spent time in gaol. So much for them and their "volk and vaderland" which never included white South Africans who didn't speak Afrikaans, and it was always debatable how many Afrikaners it included anyway. I still have difficulty singing "Die Stem" as part of the national anthem, but it's something I'll get over.
Mandela was a great president, even if he did allow thousands to die of AIDS while he was President, and largely ignored the question of crime. At least he did something about AIDS later, hence 46664. His successor, Thabo Mbeki, made all the right noises, changed the way government is structured, but never got a grip on the real issues. Instead he surrounded himself with incompetent dolts like the Minister of Health, who succeeded in screwing up the health system pretty successfully. I always thought she was just holding out for a bribe, but it turned out to be far worse.
So when the ABM movement started (this time it stood for Anything But Mbeki, not Anything But Microsoft) it wasn't because of outrage over health or electricity, but because a charismatic dimwit wasn't properly prosecuted and was allowed to engage in unprotected sex while being the head of the "Moral Regeneration Movement" [cough!]. Or something. As someone once said, "the rabble wants to be roused".
I guess poor people are sick of being given promises that aren't met. So they elected someone who made even bigger promises, while building a big mansion and accepting financial contributions. Go figure. I guess this just proves what American politics has taught us for some time: democracy is where the majority gets what the majority deserves. America got 8 years of George Bush. We've got Mbeki and now we're getting Zuma. Nothing changes, really.

Benchmarks*: Puran Defrag 4.0

Puran Defrag 4.0 was released recently. You can find a review of version 3, and the test results are shown below. It's results are impressive, and much better than I expected, and better than version 3.
The graph shows Puran Defrag 4 (PD4) in dark blue, and the results from Windows XP in light blue. The first test is at the bottom, and shorter lines mean faster times.

• "Basic XP" refers to the standard install, without Office 2007, so there are only 710 files to be tested.
• "Defrag" refers to the read times of the same 710 files, after two defrag passes using the full PD4 boot time defrag, with no "Optimize Files" or "Optimize Boot" options enabled. A 17.3% performance improvement is measured.
• "Defrag+Auto" refers to the read time after enabling both the "Optimize Boot" function in Windows, and then allowing Windows to optimise the placement of system files. After this a further boot time defrag is done, with no additional options. An improvement of 32.6% is recorded, 8.1% faster than WDD.
• "XP+Auto" is the result obtained after enabling both "Optimize Files" and "Optimize Boot" functions in Windows, and then running WDD, i.e. the best that Windows XP can manage.
  
• "Basic Office" refers to the read time of all 802 test files, where no defragmentation has been done whatsoever, after the installation of Microsoft Office 2007 Professional (Trial).
• "Full Defrag" refers to the read times of the same 802 files, after a full boot time defrag with all options enabled, with no "Optimize Files" or "Optimize Boot" options enabled. A 27.6% performance improvement is measured, as good as the Office Auto result below..
• "Full Defrag+Auto" refers to the read time after enabling both the "Optimize Boot" function in Windows, and then allowing Windows to move the system files around. After that a simple boot time defrag, with no additional options, was run. An improvement of 32.3% is recorded, 6.2% faster than WDD.
• "Office Auto" is the result obtained by allowing Windows XP to do its own defrag after enabling both "Optimize Files" and "Optimize Boot" functions.

The performance improvement ranges between 17% and 32%, with an improvement when the boot optimize information is used.
An average 27.4% performance improvement over no defrag at all is excellent for a $24.95/$14.95 (special offer) commercial package, beating all the other "heavy hitter" commercial programs tested so far. In most cases it works as well as or better than the built-in windows defrag. It also has an optional automatic defrag that is really not intrusive at all, and some other useful features. The only thing "missing" is a file placement display. The picture at the top of this article is a composite of two "Analyse" screen shots, one taken after installing Office 2007, and the other taken after the final test, effectively showing before and after situations.
The next test candidate is UltraDefrag 1.2.3

Puran Defrag 4.0 special offer

The guys at Puran Software are celebrating the release of Puran Defrag 4.0 by dropping the price for a limited time: US$ 14.95, a saving of $10 on the regular price. Even at the higher price it's worth it; this is a bargain!
I have been testing the new version for two weeks, and its brilliant. The boot time defrag goes a good job of tidying up the drive and consolidating free space, and the automatic defrag works pretty well. There are a bunch of small features that I like, such as the ability to do a defrag of a particular file or folder by right-clicking on it. Also, you can do a complete defrag followed by a shutdown, which is useful in an office environment. Just start the defrag and go home. It even runs chkdsk first, as a safety measure. This product is well engineered and the help file is clear and informative.
I plan to publish the benchmarks shortly: most of the delays are out of the way now. I suspect that the price drop has something to do with the outstanding success of their cricket team, but I'm only guessing. ;-)
Update 1 Jan 2008: the special offer price is now $19.95

Review: 3.0 | Benchmark

Benchmarks*: Norton Utilities Speed Disk 19.0.1.8

It's sad to see an old faithful program come to such a tragic end. Version 19 of Speed Disk is part of a humongous bloatware package from Symantec that requires around 300MB of free space just for the install files, unless you do the install directly from the CD. I tried the trial version of the Basic Edition of Norton SystemWorks (ironic, the name) that costs $49.95, and includes Norton Antivirus (NAV) and a whole load of other stuff. I avoided installing the NAV software because I didn't want it to slow the system down. It has been known to turn perfectly respectable machines into clunkers in a matter of minutes. Even the budget version of this package is more expensive than Diskeeper Professional. Sadly, it offers very little in terms of performance. The final defrag test ended up being slower than no defrag at all, and the average improvement was only 4% faster than no defrag, hardly justifying the expense.
The graph shows Norton Speed Disk (NSD) in dark blue, and the results from Windows XP in light blue. The first test is at the bottom, and shorter lines mean faster times.

• "Basic XP" refers to the standard install, without Office 2007, so there are only 710 files to be tested.
• "Defrag" refers to the read times of the same 710 files, after two defrag passes using NSD, with no "Optimize Files" or "Optimize Boot" options enabled. No performance improvement is measured at all.
• "Auto+Layout.ini" refers to the read time after enabling the "Optimize Boot" function in Windows, and then running NSD after Windows had done is standard boot optimise. An improvement of 14.1% is recorded, still slower than WDD.
• "XP+Auto" is the result obtained after enabling both "Optimize Files" and "Optimize Boot" functions in Windows, and then running WDD, i.e. the best that Windows XP can manage.
  
• "Basic Office" refers to the read time of all 802 test files, where no defragmentation has been done whatsoever, after the installation of Microsoft Office 2007 Professional (Trial).
• "Full Defrag" refers to the read times of the same 802 files, after two defrag passes using NSD, with no "Optimize Files" or "Optimize Boot" options enabled. A 17.2% performance improvement is measured, the best result obtained.
• "Full+Layout.ini" refers to the read time after enabling the "Optimize Boot" function in Windows, and then running NSD several times. An improvement of -14.2% is recorded, much slower than WDD.
• "Office Auto" is the result obtained by allowing Windows XP to do its own defrag after enabling both "Optimize Files" and "Optimize Boot" functions.

The performance improvement ranges between -14% and 17%, with mixed improvement when the layout.ini information is used. NSD has no way of enabling or disabling this feature in Windows. Judging by the drive image at the top of this entry, the program is unable to move numerous files (notice all the grey blocks at the bottom), and the pink area at the top of the image is supposed to be the MFT reserved space, but isn't. It's a mess, and it takes forever to run the defrag.
An average 4.3% performance improvement over no defrag at all is extremely disappointing for a $49.99 commercial package, lagging way behind all other programs tested. The program seems to be unaware of the built-in Windows optimisation, or the placement of the MFT reserved space. Frankly, it's a disaster and an embarrassment to Symantec. Judging by the marketing information associated with Norton SystemWorks they already know this, and refer to defragmentation only in passing. Sad.

*See: Benchmarks*: Why the Asterisk?