Bend in the Weather

I think we can all attest to the phenomenon known as the Teddy Bear troubleshooting.

I think we all probably need our own Teddy Bears in each of our human office box cubicles or work desks.

So next time you need to do some serious troubleshooting or some heavy lifting when debugging - try pulling out the Teddy Bear.
Even better, you can hug something after it's solved!

Even as a Linux desktop user, I still have a need to run various pieces of legacy software under Windows.
Ideally, I would love to find a Linux alternative, but whilst the itch remains, I still need to scratch it.

As such, I run various Virtualbox Virtual Machines running Windows XP.

This gives me the added benefit of still being able to run my preferred operating system as my main desktop (Linux), whilst still using some legacy applications under Windows.

This is nothing new, and most people do this all the time. With most modern CPU chipsets from both AMD (AMD-V) and Intel (VT-x) providing hardware assisted virtualization solutions - it has become a simple task for many people.

One thing that has annoyed me, and any user that has ever had to reinstall a Windows machine (it happens to the best of us -- Windows often decides to eat it's own registry), is that whilst you might install straight from CD, it also means you have to then spend a good few hours applying all the service packs and then all the updates from the Microsoft website (and sit back and watch a dozen or so reboots between multiple updates). It's enough to send any sane person around the bend.

Once you've done it - you vow never to do it again, yet as Murphy would have it... it's bound to happen again. This is partly due to the fact that Windows Deteriorates over time, until one day the performance is so shoddy you can swear you saw a snail push the cursor on your screen. Once it comes to that, it's time to reinstall again.

Microsoft also recognised it's a problem, particularly in corporate environments that run many desktop PCs. As such they released WSUS. That's fine and well -- if you run a Microsoft Windows 2003/2008 Server. Unfortunately most home users don't.

Don't dispair! Some geeky German's have released a GPLv3 licensed utility that will pretty much undertake the task for you without the need for an expensive Microsoft Windows Server anywhere in sight. c't Offline Update, allows you to use either Linux or Windows to download all the updates for both Microsoft Office and Microsoft Windows platforms. In addition it can handle different language versions, and even build a nice ISO off all the updates so you can take it on the road, hand it to friends, etc.

Why on earth would they code it to download the Windows Updates on Linux? Well, it makes it easier to install in a central location -- say on a Samba server so that all your desktop machines can then install the updates from a central location over your LAN. (Call it a poor man's WSUS!)

To run under Linux it couldn't be easier. Once you have downloaded c't Offline Update and extracted it... change into the sh subdirectory. It's then just a matter of running the DownloadUpdates.sh shell script with the right arguments:

$ ./DownloadUpdates.sh -help

**********************************************************
*** c't Offline Update Downloader ***
*** for Linux Systems ***
*** ***
*** http://www.heise.de/ct/projekte/offlineupdate/ ***
*** Authors: Tobias Breitling, Stefan Joehnke ***
**********************************************************

Usage: ./DownloadUpdates.sh [system] [language] [parameter]

Supported systems:
w2k, wxp, wxp-x64, w2k3, w2k3-x64, w60, w60-x64, oxp, o2k, o2k3, o2k7, all-x64, all-x86

Supported languages:
enu, deu, nld, esn, fra, ptg, ptb, ita, rus, plk, ell, csy
dan, nor, sve, fin, jpn, kor, chs, cht, hun, trk, ara, heb

Parameter:
/excludesp - do not download servicepacks
/makeiso - create ISO-Image
/dotnet - download .NET-Framework
/nocleanup - do not cleanup client directory
/proxy - define proxyserver( /proxy http://[username:password@]:)

Example: ./DownloadUpdates.sh wxp deu /dotnet /makeiso

$

c't Update also provides a simple Windows GUI to automate the task:

Installing the updates on the client.
Once you've downloaded the updates and either burnt them to a CD/DVD (for easy portability) or made them available on a network share (Tip: share the sub-directory called client) Windows users can then easily apply the updates.

A user can run UpdateInstaller.exe (in the root directory of the CD or the share). The UpdateInstaller will determine what you have installed already (and grey these options out). It's then just a case of selecting the options you want.Now for those family members who are in dire need of a 'computer fix' and need me to reinstall Windows for them, they'll be given burnt copies of the latest ISOs, so they can in future undertake the task themselves. I would rather educate someone how to undertake the rebuild themselves for future use. (That, and I really don't have the spare time/desire to rebuild dozens of Windows machines a year! )

Finally it's also worthwhile downloading and running the free Microsoft Baseline Security Analyzer which will ensure you have all the updates applied (and inform you if you are missing any). It's also a good idea to disable any non-essential services the tool recommends.

Hopefully this tip will speed up the time it takes to fully patch a Windows desktop after a reinstall.

OpenOffice 3.1 was recently released. It brings a range of new features and bugfixes.

Those of you on Ubuntu desktops who wish to upgrade can do so quite easily. It's been built for Jaunty, Intrepid and Hardy.

You can use the following apt repos to pick it up:

Jaunty:
deb http://ppa.launchpad.net/openoffice-pkgs/ppa/ubuntu jaunty main
deb-src http://ppa.launchpad.net/openoffice-pkgs/ppa/ubuntu jaunty main

Intrepid:
deb http://ppa.launchpad.net/openoffice-pkgs/ppa/ubuntu intrepid main
deb-src http://ppa.launchpad.net/openoffice-pkgs/ppa/ubuntu intrepid main

Hardy:
deb http://ppa.launchpad.net/openoffice-pkgs/ppa/ubuntu hardy main
deb-src http://ppa.launchpad.net/openoffice-pkgs/ppa/ubuntu hardy main

Once that's done, just add the signing key:
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com D2BB86E0EBD0F0A43D4DB3A760D11217247D1CFF

Now it's a case of just running an update and you'll automatically be upgraded from OO.org 3.0 to OO.org 3.1.

sudo apt-get update && sudo apt-get upgrade

Happy Open-Officing!

Ahh the good ol' days.

I distinctly remember the days when you would stick a printer on the network and then wrestle with the sucker getting the right settings so your printouts didn't look like Egyptian hieroglyphics.
Installing custom filters and weird control files were all the norm. Not so much now.

Thanks to Michael Fox advice, I purchased a Brother Mono Laser printer this week (that handles Duplex/Networking as well). The Brother HL-5250DN works 100% under Linux and just required installing the PPDs off LinuxPrinting.org (though it was working fine prior to that just via CUPS and the included driver). What's even more surprising is that they have a Linux section on the Brother website.

I had the printer uninstalled from the box, read the 'setup instructions' included (plug this, remove that, etc), and had it printing duplex in < 10 minutes!

Before Michael's advice -- I had avoided Brother printers -- mainly due to having a PoS Brother plain-paper Fax machine. It seems their printers are much better than their Faxes.

So a big thanks to Michael.

So I ask Linux developers... stop making it so easy to install hardware. Otherwise everyone will be running Linux before too long!

Now -- I only need a bookcase in my office, and I've completed my new office setup!

In a not to distant past Russell Coker wrote about RAID Issues and referred in part to a report containing data from 1,530,000 disks running at NetApp customer sites. (also available in PDF or Postscript)

Interesting reading, for sure - particularly if you run any large dataset and want to ensure it stays intact!

It prompted in the recesses of my memory a report compiled in February 2007 by Google. The report, 'Failure Trends in a Large Disk Drive Population' was a report presented at the USENIX (FAST '07) Conference.

The Google report looked at actual hardware failures of disk Google saw over several years. The numbers were crunched (Is there anything else Google does besides crunch large datasets?). Some interesting results popped out:

• The disks studied where either SATA or PATA consumer-grade disks that were either 5400RPM or 7200RPM ranging in size from 80G to 400G and been commissioned from any time after 2001. - Interesting this is the same disks that many of us will find in our own machines. No Enterprise disks, SCSI or SAS disks in the study.
• HDDs had a higher tendancy to fail at the start of their life or anything beyond >3 years of use.
• Low or Heavy Utilization of the HDD resulted in greater loss then 'Medium use'.
• Disks that had surface scan errors had a greater result of failure over the next 60 days.
• HDDs in operating in cool temperature (15-30^oC) had much greater failure rate in the first 3 years. Whilst disks > 3 years had a greater failure rate with the higher the operating temperature. The ideal rate for running disks to minimise failure rate was 30-35^oC.
• A disk that spends more than 50% of it's powered on time > 40^oC is a good indication of a possible problem.
• SMART data analysis revealed that it is not a reliable way to determine if a disk is about to fail. 36% of all disk failure had no SMART errors. The disks that had SMART errors the majority where seek errors (~72%). So basically, expect to see seek errors, beyond that you appear to be running blind with SMART.

Looking at both the Google Report and the NetApp Storage Report some 'best practices' become apparent to ensure you minimise your data loss:

• HDDs are mechanical devices. Expect failure and plan for it.
• Attempt to operate disks in the 30 - 35^oC temperature range to extend their life.
• Monitor disk temperatures. Extended periods where a disk temperature rises unexpectedly (ie: not under any additional load than normal), is often an early sign that failure is close at hand.
• Attempt to purchase disks not from the same batch. This will avoid a common manufacturing fault taking your disks out at the same time. (Many storage companies will do that for you as part of their service.)
• HDD failure follows the Bathtub curve. The 'right side' of the bath kicks in around 3 years. Getting life out of your HDDs beyond that is a bonus, treat it as such!
• HDDs are cheap these days. Don't be cheap -- implement RAID-6 over RAID-5 as a matter of course. Ensure you use Double-parity on your RAID-6 implementation. You'll find most recent versions of RAID-6 implement double-parity as 'standard'. (NB: If using NetApp - it's coined as RAID-DP.) Some vendors even allow you to upgrade the storage firmware online if using RAID-6 with Double Parity (NetApp for example has this feature).
• Hot-Spares in your RAID-6 array is a very good idea. For the cost of the array consider it an insurance policy against the dreaded multiple disk failure which could potentially toast your array.
• How important is your data? Can you put a cost on it? If the cost of replacement is extreme, consider redundancy options. This could include: Implementing RAID-60 (or RAID-6+0), archive/backup solutions, or even a total Disaster Recovery (DR) solution.
• Air flow around an array unit is critical. Don't cram your arrays in a fully populated rack, as minimal air-flow will ensure. This will add to the HDD temperatures and general storage enclosure. Remember you're aiming to keep your disks at 30-35^oC.
• Keep a logbook of when each drive was added/replaced. You know that 18 month disk then is less likely to fail than that disk that has been whirring away for 5 years. Record size, manufacturer and model/run. You may see some 'patterns' emerge in your own failure rates that will help with additional purchase decisions (ie: particular makes/models to avoid!)
• Perform regular 'scans' or 'checks' of your HDD health, knowing the current state of disks allows you to plan for the inevitable failure.

Other tips I've picked up over the years looking after Enterprise systems attached to large storage arrays:

• There is a reason that the ASX demands that publicly listed companies with 'mission critical' services for the public (items such as water supply, electricity/gas, and telecommunications) MUST ensure their 'mission criticial' applications have full Disaster Recovery operation on hot-standy.
• Don't cut corners and not implement items like HotSpares and Double Parity. When you have a drive failure (not if), you'll be glade you spent that little bit extra on it.
• A well planned storage solution should have drive failures, it shouldn't have storage failures. (Don't tell Sun/Internode that!)
• Ensure you have clean-filtered power. Don't assume it. Power voltage fluctuations and disk writes don't play nicely.
• Don't put all your eggs in one basket. Bad things do happen (I won't mention Internode again). Spreading your data out over multiple storage arrays helps add another level of redundancy. This is a GoodThing^TM
• It's never a bad idea to have spare disks in storage waiting for that inevitable failure, rather than relying on a vendor to have your disk capacity/model available. Every day that dead disk isn't replaced you are putting your array at risk.
• Look after your disks and arrays, and they generally will look after you (ie: ensuring you don't spend countless hours in the early mornings attempting to recover the unrecoverable!)

Above all, may your next disk failure not be a complete failure.

I'm interested to hear others stories, experiences and ideas that they have put in place to keep their data nice and cosy and their HDDs whirring in a constant and reliable state. Feel free to drop me a comment.

As a side note: I've been putting RAID-1 in place on my desktop machines as disk prices are so cheap now over the past 18 months. For the price of a few hundred dollars why bother the stress of a hard-disk failure? Having said that, it's no replacement for backing up your 'important' information. RAID-1 is still susceptible to the dreaded multiple disk failure issue. Having said that, I've been lucky at that hasn't occurred. (Touch wood!)

This week saw me head to the Victorian regional city of Ballarat for a few days to undertake some RHEL5 x86_64 builds for work.

Whilst it's not as exotic a destination as an unnamed colleague visited nor did it see me returning without my pants or shoes ( sorry Jon, I couldn't resist! ), it was a productive visit.

Traditionally we undertake builds remotely, however these servers are likely not to appear on our network or even administered by ourselves, so it made sense to go onsite to undertake the work.

I stayed in the Doherty Ballarat Hotel and Convention Centre which literally is on the opposite side of the road to Sovereign Hill (and no I didn't have time to go there!)

View Larger Map

Whilst I spent most of my time in the office battling with hardware that had been incorrectly ordered (ie: missing hardware and/or non-compatible hardware (ie: 3.5" SATA drives ordered instead of 2.5" SAS drives for the internal enclosure)), I did venture back to the hotel in the evenings for some R&R.

     

The hotel room was quite pleasant, and what something a corporate traveller expects. There was no surprises out of the ordinary (which you really don't need when travelling), though I did have two strange nights of sleep due to external factors:

1. 1^st night: Apparantly it's Koala mating season. Staying in a hotel room surrounded by forest is lovely to look at during the day, but it's also the playground for randy koalas of a night. So thumbs up to the 'big boy' who kept me up for several hours in the early hours whilst he ravaged the local female population. Koalas aren't dying out in this region whilst this bloke is still kicking! Seriously, I thought those types of noises I may have heard from other hotel rooms, not above my hotel room.


2. 2^nd night: Local Football season must have finished. Seems some of the local players didn't head over to Bali to celebrate but decided our hotel was a great place to party.
   
   In depth conversations at yelling level could be heard at around 3am. Answers to lifes biggest questions such as "Can you scull beer or bourbon faster?" were asked repeatedly. Finally after about an hour when it took that long for the gorillas to realise that they couldn't order a pizza to their room (and yes their was going to be surcharge for anything delivered to their room), the group of 6 decided that bundling into a car and heading down to the local McDonalds whilst tanked was the sensible thing to do.
   
   I passed out after that, though it appears they were successful (if not foolish) as their car was there in the morning without any visible signs of damage. It seems the drink and driving campaigns don't apply to footballers.
   
   I made sure I made enough noise as I passed their room at 7am heading off for breakfast, however in their drunken state the night previously, I doubt I would of woken them. It did bring me mild relief to my simmering frustration/anger from the previous nights interruption however.

The crazy and overpriced Internet costs in the room ($0.20/1MB with both uploads/downloads charged or $0.15/min) ensured I had a relatively Internet free week night. I really didn't miss being connected, though it did remind me how often I relied on just 'quickly looking up something'. Items like news, weather and even a TV guide via the Internet seems to have become the 'standard' method of information retrieval for me. It became more of an inconvenience than anything. I realised then that there was indeed a TV guide in the room. I gathered a quick round-up of news/weather using local radio. So this is what life was like prior to the Internet? (It seems so long ago now!)

Traditionally I use Google Reader to keep abreast of my feeds (as it allows me to bookmark, tag and share feeds easily), but being disconnected from the Internet was going to make that impossible.
I did however utilise a RSS reader, and was pleasantly surprised how easily it actually integrated into my Gnome desktop (particularly as it's a KDE app). The tray-icon and minimize/restore from tray all worked flawlessly (mostly due to the fact it's probably coded using FreeDesktop specs). I went with Akregator over Liferea after hearing reports that recent versions were prone to 100% CPU usage.

One thing I did notice, is that current RSS readers do allow for 'offline' mode - though it would be nice if it gave you the option of a 'full download' of certain feeds. Offline mode that only sucks down text is quite frustrating at times, as the associated image (being discussed in the article) isn't available. It feels like over-hearing a discussion you weren't meant to hear!

Whilst it's probably prudent allowing it to be set as a 'per feed' setting as I suspect many people may not want to download a full page for every article on every feed. (Particularly those feeds you only 'scan', and read a small amount of, or those that are overly heavy with graphics/flash, and embedded video.)

Since returning I did notice that Gnome also has the Straw Reader. Hopefully that comes along nicely as it appears to be in early development.
Ideally I would love to see an RSS reader that integrated WebKit with a good OPML import/export feature, as well as good support for RSS (0.91, 1.0 and 2.0), RDF as well as ATOM formats. Having a feature that allowed you to selectively do a full download of certain feeds for offline mode would be fantastic. I'll whack that down on my ever growing todo list (I think I'll need to live to 800 to ever complete it!)

One nice feature I discovered is the room had direct dial-in, and due to poor mobile reception in the room it was much more cost effective to have Pauline call me direct over VOIP. It also meant we could speak for much longer.

The staff I met at the Ballarat office were extremely friendly and actually quite enjoyable to be around. ( Sorry MrPointy, they were more cheery then you! ) They appreciated the effort I put in whilst down there (it's always nice to get some recognition) and appreciated the attention to detail. It was quite good timing as I will relocate offices to this one once I move into our new home (probably 4-6 weeks away). Heading into the Ballarat office is much closer than the Melbourne offices from Bannockburn. I've already made some good inroads into getting to know people there. I'll be the only member from my team at this office location, so it makes sense to build some bridges.

I did add two things to my 'travel-list' when travelling for work:

• A cheap mini 10/100 switch.
  This is invaluable when you're stuck in a room with one Ethernet port attempting multiple server builds, and also wishing to have your laptop online at the same time. I headed out the first day a picked up a cheapie TP-Link 5-port 10/100 Switch. Cost me $20 from a local computer store, but saved hours of frustration.


• A decent power board with surge protection
  In hindsight I should have also taken/purchased a decent power board. The room only had 4 power outlets which meant I could only have one server up at a time. It would have been nice that I could have multiple machines up at the same time, as these were identical builds (besides their hostnames), I could have used something like clusterssh to undertake the same tasks across all the machines at the same time -- ensuring I could have finished the job in a fraction of the time.

We live and learn.

Wine is a way to run native Windows applications without Windows. The Wine project have been able to replicate the APIs required to run quite a number of applications under their environment. It's one way you can run your legacy Windows applications under your Linux desktop. I daily run Wine for a number of applications for business:

• Windows Lotus Notes for my office Email - it's not as crappy then as the native Linux one then. :-P
• Internet Explorer 6 for those pesky websites that refuse to run without IE (many customer client websites).
• Microsoft Office for some complicated files that get garbled by OpenOffice (but that number is decreasing with every release!)

Other good news about Wine is that 1.0 is just about out the door. (Don't be fooled by the low number - these guys are perfectionists, and it's been a long, long running project!) Here's the blurb off their website:

Friends, vintners, penguins, lend me your ears: We come not to praise Windows, but to celebrate our increasing independence from it. The final release candidate for Wine 1.0 will appear this Friday (13th), and barring catastrophe, Wine 1.0 itself will be released a few days later.

I recently blogged about how hard it is for many small businesses to move off Windows. One way they can (particularly those that are on pre-Vista desktops and don't wish to go to Vista) is to utilise Wine for those legacy business Windows applications that need to run. 8-) This way they can keep their functionality, but move to a more secure, less restrictive (licensing) operating system and a better and more richer computing experience in my opinion. :-) I recently attempted to see if I could run MYOB under Wine. Currently Pauline is stuck on Windows, due to her requirement to run MYOB to do our company books. MYOB ran under Wine without a hitch. :-D Well... until I went to print. It went off to the dark printing gods and spat out an empty plain page -- no matter what I printed. And thus is the reason for this blog post. If anyone does business accounting, you realise printing is a key feature. Many hectares of forests are destroyed each year keeping up with the legal requirement of keeping records accounting records. (Personally we print a lot to PDF -- but still, under Wine this was failing for us!). Scroll through many Linux forums/help sites and you'll hear people whining and complaining about printing under Wine. It seems to have also increased with the introduction of CUPS. From what I could see people where having problems printing to a physical printer most of the time, even though their CUPS printers correctly are represented and available in Wine. (This is exactly what I was seeing -- just blank pages printed). Unfortunately this was going to be a show-stopper. Without a way to print (paper or electronically) for MYOB the idea was dead in the water. I didn't want to run Windows XP in a virtual machine (more on that in my next post over the following days). I decided to sit down and take a bit of a fiddle (that's a technical term!) :-P Really -- if CUPS-PDF didn't work, and my native printers attached, even a Postscript file would be sufficient (I could then feed that into ps2pdf or something similar if I really wanted it in PDF format for Emailing, or just print the Postscript file directly to the printer. I found that within the printing application I could create a virtual printer that printed a postscript file directly to a file in Linux. Okay, that works fine under Linux... Now the big test, did it print fine with MYOB running under Wine? I could see the printer fine (as I could with all the other CUPS printers previously). I bit my bottom lip and hit print. Low and behold... a working report! 8-) Even better I could link the Postscript printer to the CUPS-PDF printer and it would convert it auto-magically into a PDF for me. I since tested a range of applications under Wine and they all seem to be printing. So if you need to print in Wine and don't want a headache, try the local Postscript virtual printer hack. It works well. :-D For those that want to replicate it, here is a series of pictures showing the Postscript printer properties. And if you still don't believe.... here is a sample print job... one in Windows with MYOB and one under Linux

(06:34) Someone set the alarm far too early. Jumped out of bed in a panic & twisted ankle. #

(06:44) Jumped in the shower. I missed, landed on the soap. Other ankle now throbbing. #

(07:09) Walked out front door to grab morning newspaper. #

(07:10) Eyelid meet bee. Eye now swelling. #

(07:11) Bastard paperboy has got the paper in the damn bird bath again. #

(07:15) The same someone who fiddled with the alarm clock has now set the toaster to the 'burn like a crisp' setting. #

(07:16) Forgot to buy coffee, will have to settle for water. #

(07:20) Toothbrush is now black from the charcoaled toast. #

(07:25) Where the hell are my keys? #

(07:29) Tackled dog rugby style after chasing him around the coffee table. Have managed to recover them, albeit full of dog saliva. #

(07:30) Apparently keyless entry into your car doesn't work after being chewed into a black ball of plastic. #

(07:31) Hit head when entering car. Maybe this is why keyless entry was invented? #

(07:43) If you are going to have a road-rage episode, choose wisely. Now sporting a growing fat lip to match my bug bee-eye. Note to self: Don't abuse a boxer. #

(08:17) All car-parks now taken due to the time lost with the boxer. Have to park 4 blocks further away from the office. A great workout for sore and swollen ankles. #

(08:33) Remember the umbrella even if the weather report says there is little chance of rain. #

(08:53) Finally in at work. #

(08:56) The company provided International Roast is a poor substitute for coffee. #

(09:05) Coffeeeeee annnndddd keeeyboardddddds donnnnn't mix. #

We can all dream!

How many times do you get sent MS-Word, MS-Excel or MS-Powerpoint files?
Whilst OpenOffice does an excellent job opening them all, I would prefer them all in an open format natively.

It appears the Dutch may come to the rescue and make that dream a reality.

Reading through IDABC website it appears that the Dutch Council of State are going to open-source an application they have written that allows you to convert flawlessly between Microsoft Office <-> Open Office. It can also convert -> PDF.

At present, the only minor hiccup is images generated to PDF are a little 'darker'.

Really, I am happy to just get MS Office -> OpenOffice.

Really the European approach to OSS is a beacon of light that hopefully is adopted across all regions. The days of proprietry file formats are quickly disappearing. For future generations and long term storage Open Standards in file formats in mandatory.

History will look back at the stand the EU has made against Microsoft and show that their stance was the under swell that pushed many in that direction.

It's great to see South Africa has also filed their appeal against the OOXML becoming an ISO standard. It does show there are some people left within ISO that still have a back-bone and conscience.

South Africa challenges the validity of the final vote that we contend was based upon inadequate information resulting from poorly conducted BRM. Moreover, we challenge the validity of a process that, from beginning to end, required all parties involved to analyze far too much information in far too little time, involved a BRM that did not remotely provide enough time to perform the appointed purpose of that procedure, and for which an arbitrary time limitation was imposed to discuss and resolve a significant number of substantial responses, despite the Directives for not requiring any such limitation as to duration.

It is our opinion that the process followed during all stages of this fast track has harmed the reputations of both ISO and the IEC and brought the processes enshrined in the Directives into disrepute, and that this negative publicity has, in turn, also harmed the reputations of all member bodies of ISO and the IEC.

Let's hope this appeal isn't squashed in a face-saving exercise by ISO. The entire ISO process surrounding the adoption was flawed from the start with many unethical breaches of conduct undertaken during the whole voting process.