Wednesday, July 9. 2008
Time for an Australian OSS based Accounting & Payroll program
Ask any small business owner in Australia what drives them craziest running their business.
More often then not the answer will be their accounting software.
Why? It's not because they are unable to understand the principles of double-entry accounting or even the tax system imposed on them.
No, its the lack of a decent, flexible and affordable solution.
The two most popular business accounting packages for the SME is either MYOB and QuickBooks. Both have huge hurdles, and it's not uncommon for users to find that they spend more time attempting to work the software than the software working for them.
I have previously blogged about the issue when talking about how Linux is difficult for many Aussie SMEs to take up. (Mainly due to their critical accounting software only available under Windows, though it is possible to run them under WINE if required). Though getting these Windows programs running under Linux doesn't actually solve the problem, it just migrates it. You are still stuck attempting to drive these frustrating accounting packages.
The Australian market is crying out for an alternative! It's something I've pondered over for some time.
An OSS based accounting and payroll system that put the source code directly in the hands of the users would be appealing to many.
I like the idea of a browser-based accounting system. It means end-users aren't restricted to one operating system platform, and in theory it doesn't need to be restricted to just a computer. (3G Accounting anyone?).
Accounting isn't fun, in fact it's dead-pan boring. It is however a required evil if you wish to be paid and be compliant with the taxation law. (Particularly GST, BAS reporting and PAYG).
I've looked at programs like SQL-Ledger and LedgerSMB, though both didn't 'fit right' for an Australian environment.
SYMBOL was promising. But unfortunately has been abandoned. fiveDASH, I played with for several hours, though I found it quite primitive and in it's infancy... it's one that at least has some promise, but it will need to progress quite some way to make it in any real production system for a business.
As such, I've decided to look at writing my own. This is purely selfish, as I would like to ditch MYOB come the 2009-2010 financial year. That gives me just under a year to get something up and running.
I'll run a dummy book up til the time, by running MYOB in parallel to ensure all is working as required. The ATO also has some excellent test suites/scenarios to assist in testing an application.
The big choice now is from what components should I build it?
I think PostgreSQL is probably the most solid OSS database to use, though writing something that can also talk to MySQL (due to it's popularity) is probably also wise. End-users can then elect to run whichever they feel most comfortable for their backend.
I considered many languages, but I'm leaning towards PHP for a few reasons:
I'm interested to hear people's feedback with their thoughts. Do you think a PHP application (backended by your choice of MySQL or PostgreSQL) for your business accounts is feasible? (Obviously if 'approved' by the ATO as compliant).
Do SMB owners prefer the option of paid software with support packs? (like offered by MYOB and Quicken) they can fall back on? Would you not trust your business books to an OSS based accounting package? What other hurdles do you envisage if you used OSS accounting software?
More often then not the answer will be their accounting software.
Why? It's not because they are unable to understand the principles of double-entry accounting or even the tax system imposed on them.
No, its the lack of a decent, flexible and affordable solution.
The two most popular business accounting packages for the SME is either MYOB and QuickBooks. Both have huge hurdles, and it's not uncommon for users to find that they spend more time attempting to work the software than the software working for them.
I have previously blogged about the issue when talking about how Linux is difficult for many Aussie SMEs to take up. (Mainly due to their critical accounting software only available under Windows, though it is possible to run them under WINE if required). Though getting these Windows programs running under Linux doesn't actually solve the problem, it just migrates it. You are still stuck attempting to drive these frustrating accounting packages.
The Australian market is crying out for an alternative! It's something I've pondered over for some time.
An OSS based accounting and payroll system that put the source code directly in the hands of the users would be appealing to many.
I like the idea of a browser-based accounting system. It means end-users aren't restricted to one operating system platform, and in theory it doesn't need to be restricted to just a computer. (3G Accounting anyone?).
Accounting isn't fun, in fact it's dead-pan boring.
It is however a required evil if you wish to be paid and be compliant with the taxation law. (Particularly GST, BAS reporting and PAYG).I've looked at programs like SQL-Ledger and LedgerSMB, though both didn't 'fit right' for an Australian environment.
SYMBOL was promising. But unfortunately has been abandoned.
fiveDASH, I played with for several hours, though I found it quite primitive and in it's infancy... it's one that at least has some promise, but it will need to progress quite some way to make it in any real production system for a business.As such, I've decided to look at writing my own. This is purely selfish, as I would like to ditch MYOB come the 2009-2010 financial year. That gives me just under a year to get something up and running.
I'll run a dummy book up til the time, by running MYOB in parallel to ensure all is working as required. The ATO also has some excellent test suites/scenarios to assist in testing an application.The big choice now is from what components should I build it?
I think PostgreSQL is probably the most solid OSS database to use, though writing something that can also talk to MySQL (due to it's popularity) is probably also wise. End-users can then elect to run whichever they feel most comfortable for their backend.
I considered many languages, but I'm leaning towards PHP for a few reasons:
- Most SMB that have a Linux server for File and Print serving also have a LAMP environment already setup.
- Writing the system using say the Zend Framework ensures a lot of the ground work is already done. Working within their coding standards also ensures the code is reasonably clean and readable (and thus maintainable).
- Python was also considered, but it's more likely a SMB has a Apache/PHP environment over an Apache/Python setup
- I already have quite a number of generic PHP libraries that will mean I can get moving much faster. My Python libraries are no-where as extensive.
- A Java-based web app would be an overkill and likely to confuse many small business owners. (ie: Install a Java Application Server, etc.)
- SugarCRM is already popular in many SMBs. It would be quite easy to have an Accounting + Payroll package integrate into SugarCRM. Sugar is written in PHP, so it would make the job much easier if they both spoke the same language.
I'm interested to hear people's feedback with their thoughts. Do you think a PHP application (backended by your choice of MySQL or PostgreSQL) for your business accounts is feasible? (Obviously if 'approved' by the ATO as compliant).
Do SMB owners prefer the option of paid software with support packs? (like offered by MYOB and Quicken) they can fall back on? Would you not trust your business books to an OSS based accounting package? What other hurdles do you envisage if you used OSS accounting software?
Wednesday, June 11. 2008
Improving the Gnome terminal
I've been hearing ramblings on a few blogs about a little app called Terminator. (No it isn't Arnie hitting ya desktop), but in fact a terminal program that can do split screens and a bunch of other funky stuff.
Here's the blurb straight from the Terminator website:
It can be found in the Ubuntu 8.04 (Hardy) repositories, and the Debian Sid repositories.
If you're running those distros, you can just click here to install terminator
Those still stuck on Ubuntu 7.10 (Gutsy) can also install it. Just add the following lines your
Here's the blurb straight from the Terminator website:
Terminator is an attempt to maximise useful space on a given desktop for terminals. I’m a sysadmin by trade, so I live in ssh sessions and thus terminals. An extra line here and there over all those hours seems like it’s worth trying for.
Much of the behaviour of Terminator is based on GNOME Terminal, and we are adding more features from that as time goes by, but we also want to extend out in different directions with useful features for sysadmins and other users. If you have any suggestions, please file wishlist bugs!
It can be found in the Ubuntu 8.04 (Hardy) repositories, and the Debian Sid repositories.
If you're running those distros, you can just click here to install terminator
Those still stuck on Ubuntu 7.10 (Gutsy) can also install it. Just add the following lines your
/etc/apt/sources.listfile:
deb http://ppa.launchpad.net/gnome-terminator/ubuntu gutsy main restricted universe multiverse
deb-src http://ppa.launchpad.net/gnome-terminator/ubuntu gutsy main restricted universe multiverse
Once done run the following:sudo apt-get update && sudo apt-get install terminator
NB: For Gutsy users, you will get a warning that there is no authentication on the package. As it's from the ppa area, that's quite normal.
The next version 0.9 is supposed to also have tabs and some other nice goodies like drag n drop... (take a peep over here)
One thing I've found is that I like the extra real-estate. No annoying menu to get in the way. I use a lot of keyboard short cuts (like Ctrl_+, Ctrl_- and Ctrl_0 to resize the font sizes). I've found most of the Gnome shortcuts work.
Even better, it seems to be less of a memory hog than what the default Gnome Terminal is.
The developer is breaking the bonds that hold it to Gnome, so in theory you should be able to use it on other Window Managers like KDE, Enlightenment, etc in the not too distant future.
And now, for the sample screen shot!
Well it appears I'm hooked. I'm looking forward to 0.90 but for mind, the current version does all I need. (Small, efficient and easier to view multiple screens in one window).
If you haven't used it, give it a try you'll be glad you did!
Sunday, June 8. 2008
The difficulty of Aussie small businesses adopting Linux
There are many reasons people are stuck on Windows.
For small businesses the main reason is their accounting software.
Like it or not, since the introduction of GST and BAS the days of the shoe-box accounting have disappeared. Every business that wants to keep accurate financial records requires an computerized accounting system.
In Australia, The introduction of GST and BAS has made it difficult for these same people to move over to Linux.
The two most popular Small Business Accounting packages are MYOB and QuickBooks. Both products support the Australian market and handle submissions to the ATO.
Many small businesses look for software that can handle base accounting (less then 25 invoices a week) and an integrated simple payroll module that can pay their employees (1-5 typically).
Both QuickBooks and MYOB operate as closed-source commercial software that runs on Windows (and on Mac - in the case of MYOB). They also integrate heavily into Microsoft Office.
Both business models expect customers to fork our between $500 - $800 for the initial purchase, and anywhere from $200- $500 annually to get 'updates'. For any small business this is a hefty investment just to keep up with the legal requirements of running their business. Calling for support can also attract additional fees on top of the yearly retainer.
As such, many are fearful to move on, but have a love-hate relationship with their accounting software provider. They feel cheated and ripped off (and rightly so). This duopoly has created a handsome cash-cow for both software makers.
The current extortion model used by MYOB and Quicken is best explained by Turbo Cash's explaination (a GPL Windows Accounting package). Unfortunately TurboCash is Windows software and contains no Australian payroll module.
Unfortunately, I am yet to see an OSS Linux accounting software that handles all the features required for an Australian business:
There is both Ledger-SQL and LedgerSMB (a fork of the former) that handle accounting and can be customized to handle GST. Yet, they don't do anything to address BAS reporting/lodging or have a payroll module. As such, both are unsuitable for the Australian SMB market without further heavy lifting.
Their is commercial software like SAGE AccPac available, yet it's quite expensive and more geared towards a medium sized business and thus out of the price range for most small businesses.
One OSS project that did look indeed promising was SYMBOL (Surf Your Money Books On Line). It was written by an accountant from Western Australia and licensed under the BSD license.
I have been in Email discussion with the developer Edward Metcalfe, but it appears he has now moved on from the project and as such the project has stalled. Looking at the CVS repository the last update was unfortunately 13 months ago.
However, the software is indeed functional and delivers on the two items listed above (Australian Taxation/Payroll). Whilst you would need to update the PAYG tax tables (no real show-stopper), it should do the job admirably. ComputerWorld actually ran an article on SYMBOL in Februrary 2007.
Those interested in SYMBOL, will find it is a web-based application so accounting can be down via a regular browser (making it suitable on either Mac/Windows/Linux clients) It runs on a PostgreSQL backend and uses Perl/CGI for the front-end. It also is multi-user based. (One thing software like MYOB and Quicken charge an arm and a leg for!)
The fear of running SYMBOL though in it's current form is that it is unmaintained. It's likely the current government or subsequent ones will indeed look at overhauling the payroll and corporate tax systems (as opposed to just an implementation of the Goods and Services Tax). This would mean that any software a small business adopts would need to address these modification.
The ATO has worked to provide great documentation on the complex tax scheme we have in Australia for software developers. Their Software Developers Website gives test case scenarios and formulas to write compliant and competing software.
I'm in two minds:
Ideally I would love to see a AJAXed based web-version of the application with a modern interface using either the Yahoo UI or the Dojo Toolkit's Dijit and DojoX interfaces.
Either solution would provide an excellent interface and provide the user with an experience that matched a local fat client. Using a web framework like Django and a PostgreSQL database underneath, would make for a solid backend.
I think there is a market for an excellent OSS product to capture the market and knock off this duopoly. However I suspect it will take some committed individuals to pull it off. From the grumblings you hear frequently from small business owners there is a ready market to ditch their over-priced software once a viable and supported solution arises.
Here are some interesting facts about the Australian Small Business Market:
I suspect a lot of OSS developers do a fair amount of moon-lighting and as such require a way to invoice for their work. (Consulting may be your day time job also to pay for the bills, and OSS coding your passion and after hours work!).
So what are your thoughts?
Is there enough interest in getting up an OSS Accounting+Payroll solution for Australian small businesses?
For small businesses the main reason is their accounting software.
Like it or not, since the introduction of GST and BAS the days of the shoe-box accounting have disappeared. Every business that wants to keep accurate financial records requires an computerized accounting system.
In Australia, The introduction of GST and BAS has made it difficult for these same people to move over to Linux.
The two most popular Small Business Accounting packages are MYOB and QuickBooks. Both products support the Australian market and handle submissions to the ATO.
Many small businesses look for software that can handle base accounting (less then 25 invoices a week) and an integrated simple payroll module that can pay their employees (1-5 typically).
Both QuickBooks and MYOB operate as closed-source commercial software that runs on Windows (and on Mac - in the case of MYOB). They also integrate heavily into Microsoft Office.
Both business models expect customers to fork our between $500 - $800 for the initial purchase, and anywhere from $200- $500 annually to get 'updates'. For any small business this is a hefty investment just to keep up with the legal requirements of running their business. Calling for support can also attract additional fees on top of the yearly retainer.
As such, many are fearful to move on, but have a love-hate relationship with their accounting software provider. They feel cheated and ripped off (and rightly so). This duopoly has created a handsome cash-cow for both software makers.
The current extortion model used by MYOB and Quicken is best explained by Turbo Cash's explaination (a GPL Windows Accounting package). Unfortunately TurboCash is Windows software and contains no Australian payroll module.
Unfortunately, I am yet to see an OSS Linux accounting software that handles all the features required for an Australian business:
- Handles Australian Taxation - including GST and BAS reporting/lodging
- Handles Australian Payroll - including PAYG and Superannuation
There is both Ledger-SQL and LedgerSMB (a fork of the former) that handle accounting and can be customized to handle GST. Yet, they don't do anything to address BAS reporting/lodging or have a payroll module. As such, both are unsuitable for the Australian SMB market without further heavy lifting.
Their is commercial software like SAGE AccPac available, yet it's quite expensive and more geared towards a medium sized business and thus out of the price range for most small businesses.
One OSS project that did look indeed promising was SYMBOL (Surf Your Money Books On Line). It was written by an accountant from Western Australia and licensed under the BSD license.
I have been in Email discussion with the developer Edward Metcalfe, but it appears he has now moved on from the project and as such the project has stalled. Looking at the CVS repository the last update was unfortunately 13 months ago.
However, the software is indeed functional and delivers on the two items listed above (Australian Taxation/Payroll). Whilst you would need to update the PAYG tax tables (no real show-stopper), it should do the job admirably. ComputerWorld actually ran an article on SYMBOL in Februrary 2007.
Those interested in SYMBOL, will find it is a web-based application so accounting can be down via a regular browser (making it suitable on either Mac/Windows/Linux clients) It runs on a PostgreSQL backend and uses Perl/CGI for the front-end. It also is multi-user based. (One thing software like MYOB and Quicken charge an arm and a leg for!)
The fear of running SYMBOL though in it's current form is that it is unmaintained. It's likely the current government or subsequent ones will indeed look at overhauling the payroll and corporate tax systems (as opposed to just an implementation of the Goods and Services Tax). This would mean that any software a small business adopts would need to address these modification.
The ATO has worked to provide great documentation on the complex tax scheme we have in Australia for software developers. Their Software Developers Website gives test case scenarios and formulas to write compliant and competing software.
I'm in two minds:
- Do I run SYMBOL and either start maintaining it or fork it?
- Do I use a lot of the program logic, and spawn a completely new project?
Ideally I would love to see a AJAXed based web-version of the application with a modern interface using either the Yahoo UI or the Dojo Toolkit's Dijit and DojoX interfaces.
Either solution would provide an excellent interface and provide the user with an experience that matched a local fat client. Using a web framework like Django and a PostgreSQL database underneath, would make for a solid backend.
I think there is a market for an excellent OSS product to capture the market and knock off this duopoly. However I suspect it will take some committed individuals to pull it off. From the grumblings you hear frequently from small business owners there is a ready market to ditch their over-priced software once a viable and supported solution arises.
Here are some interesting facts about the Australian Small Business Market:
- There are about 1.2 million small businesses in Australia employing around 3.3 million people.
- During the past decade, small businesses have created an estimated 660,000 new jobs.
- 42% of all goods exporters are small businesses.
- 25% of the Australian GDP (estimated in 2006 at $1 trillion dollars) comes from small business.
I suspect a lot of OSS developers do a fair amount of moon-lighting and as such require a way to invoice for their work. (Consulting may be your day time job also to pay for the bills, and OSS coding your passion and after hours work!).
So what are your thoughts?
Is there enough interest in getting up an OSS Accounting+Payroll solution for Australian small businesses?
Wednesday, June 4. 2008
Sourceforge mirror now on Internode
On Tuesday, Internode announced that they are now mirroring Sourceforge.
This is fantastic news for all OSS lovers down here in Australia.
Whilst we have had the up and down again service from Optus for some time, it's great to have an alternative.
(Having said that - I don't see Optus being offered on Sourceforge any more? Has Optus mirroring disappeared again?)
Even better, customers of Internode will actually get those downloads for free now. (Whilst I don't download Gb's of data each month of Sourceforge, I do grab things regularly off it! -- and every little bit helps.) More importantly, having a speedy mirror to grab code off is what counts.
It's great to see Internode -- a company that uses a lot of OSS software in their service delivery give back to the community that makes it happen.
Their range of free mirrored content is quite appealing. They run their own download mirror site and for a Linux lover there is a huge selection of distros, and updates mirrored. Even better it supports rsync so you can grab your updates quickly and easily! Add to it, now Sourceforge and MajorGeeks (for Windows software lovers). I regularly make use of their free streaming radio content. They have stuff also for gamers (who's got the time ?! ) as well as some other content. You can take a peep here.
The Sourceforge content is hosted here: http://internode.dl.sourceforge.net/ but you'll need to use the Sourceforge website to pick a download, then select Adelaide, Australia from the list of download options. (The direct link to the Internode mirror isn't browsable).
— — —
After reviewing this post, it does sound like an advert for Internode!
Having said that, I've used a bunch of ISPs over the years, and I haven't been happier than I am now with Internode.
The service is always faultless, 13-NODE (Tech support) is answered always promptly by someone who actually understands technical details (and not a messenger service for a tech team).
I've even had technical staff diagnose and fix routing issues whilst I've been on the phone with them!
So if you're in the market for a new ISP, give them a go. Sure, they aren't the cheapest ISP going around, but they are probably the best technical ISP and have some real added bonuses for OSS fans.
Tip: For those that do wish to sign up with Internode, one way to pickup an ongoing discount of 15% is to register as a SAGE-AU member. Get the benefits of being a SAGE-AU member -- and cheaper Internet access. (You will need a valid SAGE-AU membership number to be entitled to the discount).
For me, the cost of the SAGE-AU membership is less than the discount.. so I come out ahead!
This is fantastic news for all OSS lovers down here in Australia.
Whilst we have had the up and down again service from Optus for some time, it's great to have an alternative.
(Having said that - I don't see Optus being offered on Sourceforge any more? Has Optus mirroring disappeared again?)Even better, customers of Internode will actually get those downloads for free now. (Whilst I don't download Gb's of data each month of Sourceforge, I do grab things regularly off it! -- and every little bit helps.) More importantly, having a speedy mirror to grab code off is what counts.
It's great to see Internode -- a company that uses a lot of OSS software in their service delivery give back to the community that makes it happen.
Their range of free mirrored content is quite appealing. They run their own download mirror site and for a Linux lover there is a huge selection of distros, and updates mirrored. Even better it supports rsync so you can grab your updates quickly and easily! Add to it, now Sourceforge and MajorGeeks (for Windows software lovers). I regularly make use of their free streaming radio content. They have stuff also for gamers (who's got the time ?!
) as well as some other content. You can take a peep here.The Sourceforge content is hosted here: http://internode.dl.sourceforge.net/ but you'll need to use the Sourceforge website to pick a download, then select Adelaide, Australia from the list of download options. (The direct link to the Internode mirror isn't browsable).
— — —
After reviewing this post, it does sound like an advert for Internode!
Having said that, I've used a bunch of ISPs over the years, and I haven't been happier than I am now with Internode.
The service is always faultless, 13-NODE (Tech support) is answered always promptly by someone who actually understands technical details (and not a messenger service for a tech team).
I've even had technical staff diagnose and fix routing issues whilst I've been on the phone with them!
So if you're in the market for a new ISP, give them a go. Sure, they aren't the cheapest ISP going around, but they are probably the best technical ISP and have some real added bonuses for OSS fans.
Tip: For those that do wish to sign up with Internode, one way to pickup an ongoing discount of 15% is to register as a SAGE-AU member. Get the benefits of being a SAGE-AU member -- and cheaper Internet access.
(You will need a valid SAGE-AU membership number to be entitled to the discount).For me, the cost of the SAGE-AU membership is less than the discount.. so I come out ahead!
Saturday, May 24. 2008
Nagios 2-way alerting via SMS - Part 3
This is a 3 part posting that covers how you can setup 2-way Nagios alerting via SMS.
The series is broken down as such:
Review
As we saw in Part 2 we covered setting up Nagios to send out alerts.
Whilst this is great, it's not of great value if we can't do anything with these SMS messages. In addition, if no-one is sitting in front of the actual Nagios web console to acknowledge the issue, then Nagios will continue sending the SMS messages and will quickly fill up your phone's message memory. Not ideal!
The message is quite straight forward as well to read. A service problem message contains:
Here is a Host problem message for comparison:Replying to the Nagios Problem SMS
When a problem from Nagios is sent to you via SMS it's quite easy to respond.
All you need to do is reply to the message (and include the text - which most mobile phones on the market provide a feature to do).
Whilst not required, you can append a simple message to your reply. This will be used in the acknowledgment response and add a comment to the service.
Comments don't need to be long, but they do help your other team members or interested parties know what is happening with the issue.
Be careful what you write!
The response you send will also be used in a follow-up SMS to everyone notifying them that someone is looking at the issue. You don't want to send something that might not be appreciated others that recipients
Acknowledging SMS messages
Acknowledging messages from Nagios is quite simple now we have our SMS Daemon setup (see Part 2 for details)
As a result, any incoming SMS messages get stored into our MySQL database.
We just need a process now to read them from the database, interpret them and then update Nagios appropriately.
The series is broken down as such:
- Part 1: Background and Requirements
- Part 2: Sending SMS messages from Nagios
- Part 3: Acknowledging SMS messages (this article)
Review
The SMS message Nagios generates when there is a problem.
Whilst this is great, it's not of great value if we can't do anything with these SMS messages. In addition, if no-one is sitting in front of the actual Nagios web console to acknowledge the issue, then Nagios will continue sending the SMS messages and will quickly fill up your phone's message memory. Not ideal!
The message is quite straight forward as well to read. A service problem message contains:
- The type of incident: PROBLEM
- The Host affected
- The Service affected
- The Severity of the incident
- Additional Information about the incident.
- The Time of the incident
- A field to Respond/acknowledge the incident
Here is a Host problem message for comparison:
- PROBLEM
Host 'MyServer' is DOWN
I: CRITICAL - Host Unreachable (10.0.0.74)
T: 2008-05-22 17:52:33
R:
Sending a response back to Nagios via SMS.
All you need to do is reply to the message (and include the text - which most mobile phones on the market provide a feature to do).
Whilst not required, you can append a simple message to your reply. This will be used in the acknowledgment response and add a comment to the service.
Comments don't need to be long, but they do help your other team members or interested parties know what is happening with the issue.
Be careful what you write!
The response you send will also be used in a follow-up SMS to everyone notifying them that someone is looking at the issue. You don't want to send something that might not be appreciated others that recipients
Acknowledging SMS messages
Acknowledging messages from Nagios is quite simple now we have our SMS Daemon setup (see Part 2 for details)
As a result, any incoming SMS messages get stored into our MySQL database.
We just need a process now to read them from the database, interpret them and then update Nagios appropriately.
Continue reading "Nagios 2-way alerting via SMS - Part 3"
Monday, May 19. 2008
Nagios 2-way alerting via SMS - Part 1
This is a 3 part posting that covers how you can setup 2-way Nagios alerting via SMS.
The series is broken down as such:
For those that wish to setup simple, inexpensive monitoring you will find that it is simpler than first thought. Feel free to grab the code-snippets provided over the series and make use of it in your own environment.
The Nagios SMS alert system has been running here now in production for approximately 6 months and works quite well. The Nagios system manages a range of services/hosts and checks approximately 1,500 items (by no means large); it has however cut down the amount of time I require looking at Nagios alert screens.
Background and Requirements
Background
I like to dabble in web development and have now for around 10 years. As a result I undertake hosting in commercial data centers in order to ensure maximum uptime and good responsiveness. To offset the cost, I now provide hosting/email and web-development services to a number of clients.
As a result, it means one cannot easily look at the diagnostic LEDs, or even the console easily to determine when a problem has occurred. (Whilst I can make use of an IP KVM; it is normally a PITA and best avoided!)
Ideally I was after a monitoring system that alerted me prior to a problem so that I can deal with it before it became a bigger issue.
Nagios was chosen as it provided all the features we required, had support for a large number of items we wished to monitor, and was simple to extend to check custom services based on being written in Perl. Nagios comes with a swag of documentation making it easy to write simple checks or extend on as was seen fit. As I've coded a fair amount in Perl over the years for various System Administration tasks, Nagios was seen as a good fit. Being an Open-Source project ensured that it was an added bonus of both being free-to-use, but also I had access to the source code to understand better how the system worked.
I have used Nagios for years now, and historically have relied on web/email alerts. However, being human means I can't always be in front of my computer 24x7. This worked the majority of the time, though as luck would have it we ended up with a system failure that occurred early one mid-week morning. As a result it was decided we needed to put a system in place that alerted when something occurred when I wasn't in front of the computer.
We already had a working Nagios monitoring environment, so we wanted to make use of this system going forward.
Requirements
I wrote down our requirements for a paging system. I wanted to ensure whatever solution was put in place that it could grow with us, but also that we weren't locked into a solution for years or even worse, that we embarked upon a solution that would ultimately restrict any future requirements we might have.
There were several ways we could be alerted:
We ruled out a telephone, as a recorded message wouldn't provide us any real details of what the issue was, unless we actually started recording exactly what the issue was. It was beginning to look too complicated for a simple requirement. I really thought a Voice-IVR was an overkill for what we required. (Whilst it may of indeed been fun to hack/configure!)
A pager was then looked into. Whilst pagers work well in ensuring a message is delivered we saw this as a drawback, in the sense it meant carrying around another device on us. (We already carried a mobile phone and the prospect of carrying a pager didn't appeal). Also pagers traditionally send one-way messages; and we wanted a system where we could acknowledge an alert.
We settled on a alert delivery system to be used via SMS. Whilst SMS doesn't have the guaranteed delivery system of that of a pager, it did open up the possibility of being able to respond to an alert. It had the added bonus that we already carried our mobiles close by 24x7 so there was no need to carry an additional device. Using a SMS to deliver a text based message also ensured we could then place useful information in the message to assist in the problem diagnosis.
We had now determined our delivery system (SMS), it was time to find technical solutions that would fit in with the existing Nagios monitoring platform.
Nagios by default has the ability to alert via various means. Pagers, SMS and Emails are handled quite easily within the standard configuration that ships with the software.
It would make the integration of SMS quite easily. However, it was only by default a 1-way system. There was no provision to handle a 2-way message. ( Alert Message then acknowledgment message)
A 2-way SMS alert system was preferred as it could be setup to stop alerting us continually via SMS once the alert had been acknowledged. It also had the added benefit that whilst the message was sent to other parties, it could also ensure that they were aware the problem was being investigated and avoid duplication of work.
We also wanted to run the SMS alert system as close as possible to the Nagios monitoring host. This would ensure that if we had a physical network or IP link down, we could still get messages out. This cut out using a 3rd party SMS service.
Coming Up In the next part of the series...
In the next part, I'll describe the components that make up our 2-way SMS solution using OSS software and some of the scripts we hacked together to 'glue' the solution together.
It's now been running without a hitch now for about 6 months.
We also use it as a Web2SMS gateway locally, as well as being open enough that I can utilize it via the CLI to send out alerts.
Looking back the solution was quite simple, and met all our requirements.
The series is broken down as such:
- Part 1: Background and Requirements (this article)
- Part 2: Sending SMS messages from Nagios
- Part 3: Acknowledging SMS messages
For those that wish to setup simple, inexpensive monitoring you will find that it is simpler than first thought. Feel free to grab the code-snippets provided over the series and make use of it in your own environment.
The Nagios SMS alert system has been running here now in production for approximately 6 months and works quite well. The Nagios system manages a range of services/hosts and checks approximately 1,500 items (by no means large); it has however cut down the amount of time I require looking at Nagios alert screens.
Background and Requirements
Background
I like to dabble in web development and have now for around 10 years. As a result I undertake hosting in commercial data centers in order to ensure maximum uptime and good responsiveness. To offset the cost, I now provide hosting/email and web-development services to a number of clients.
As a result, it means one cannot easily look at the diagnostic LEDs, or even the console easily to determine when a problem has occurred. (Whilst I can make use of an IP KVM; it is normally a PITA and best avoided!)
Ideally I was after a monitoring system that alerted me prior to a problem so that I can deal with it before it became a bigger issue.
Nagios was chosen as it provided all the features we required, had support for a large number of items we wished to monitor, and was simple to extend to check custom services based on being written in Perl. Nagios comes with a swag of documentation making it easy to write simple checks or extend on as was seen fit. As I've coded a fair amount in Perl over the years for various System Administration tasks, Nagios was seen as a good fit. Being an Open-Source project ensured that it was an added bonus of both being free-to-use, but also I had access to the source code to understand better how the system worked.
I have used Nagios for years now, and historically have relied on web/email alerts. However, being human means I can't always be in front of my computer 24x7. This worked the majority of the time, though as luck would have it we ended up with a system failure that occurred early one mid-week morning.
As a result it was decided we needed to put a system in place that alerted when something occurred when I wasn't in front of the computer.We already had a working Nagios monitoring environment, so we wanted to make use of this system going forward.
Requirements
I wrote down our requirements for a paging system. I wanted to ensure whatever solution was put in place that it could grow with us, but also that we weren't locked into a solution for years or even worse, that we embarked upon a solution that would ultimately restrict any future requirements we might have.
There were several ways we could be alerted:
- Telephone: Have the system 'call us'. Possibly play a standard recorded message.
- Pager: Make use of an alpha-numeric pager.
- SMS: Send a text message.
We ruled out a telephone, as a recorded message wouldn't provide us any real details of what the issue was, unless we actually started recording exactly what the issue was. It was beginning to look too complicated for a simple requirement. I really thought a Voice-IVR was an overkill for what we required. (Whilst it may of indeed been fun to hack/configure!)
A pager was then looked into. Whilst pagers work well in ensuring a message is delivered we saw this as a drawback, in the sense it meant carrying around another device on us. (We already carried a mobile phone and the prospect of carrying a pager didn't appeal). Also pagers traditionally send one-way messages; and we wanted a system where we could acknowledge an alert.
We settled on a alert delivery system to be used via SMS. Whilst SMS doesn't have the guaranteed delivery system of that of a pager, it did open up the possibility of being able to respond to an alert. It had the added bonus that we already carried our mobiles close by 24x7 so there was no need to carry an additional device. Using a SMS to deliver a text based message also ensured we could then place useful information in the message to assist in the problem diagnosis.
We had now determined our delivery system (SMS), it was time to find technical solutions that would fit in with the existing Nagios monitoring platform.
Nagios by default has the ability to alert via various means. Pagers, SMS and Emails are handled quite easily within the standard configuration that ships with the software.
It would make the integration of SMS quite easily. However, it was only by default a 1-way system. There was no provision to handle a 2-way message. ( Alert Message then acknowledgment message)
A 2-way SMS alert system was preferred as it could be setup to stop alerting us continually via SMS once the alert had been acknowledged. It also had the added benefit that whilst the message was sent to other parties, it could also ensure that they were aware the problem was being investigated and avoid duplication of work.
We also wanted to run the SMS alert system as close as possible to the Nagios monitoring host. This would ensure that if we had a physical network or IP link down, we could still get messages out. This cut out using a 3rd party SMS service.
Coming Up In the next part of the series...
In the next part, I'll describe the components that make up our 2-way SMS solution using OSS software and some of the scripts we hacked together to 'glue' the solution together.
It's now been running without a hitch now for about 6 months.
We also use it as a Web2SMS gateway locally, as well as being open enough that I can utilize it via the CLI to send out alerts.
Looking back the solution was quite simple, and met all our requirements.
Friday, February 29. 2008
Just for MrPointy
MrPointy, This blog post is just for you!
Your wicked Java way,
Ensures dodgy apps are hear to stay.
We all know Java sucks,
Yet you insist we try our luck.
Process freezes and applications hung,
Dont worry, Java has just begun!
We know you're a Java nut,
But don't think we'll fall in the same rut.
Write once, run anywhere,
Sucky applications we just can't bare.
Eclipse and NetBeans; your preferred IDE,
Give me a simple text editor; and forget IcedTea!
Java certainly has it's place,
Use it in Windows with it's poison lace.
System programming requires C,
Leave Java out of it, let us be!
Show me one decent Java app,
That memory it doesn't sap.
Just one app that will not crash,
That doesn't make my system thrash.
Python, C, and Perl is what to use,
It's bound to avoid the blues,
Use them next time you need a script,
Not that silly Java dialect!
There is no more to state,
Java just isn't that great!
Avoid coding in Java at all cost,
So your reputation is not lost!
Many claim Java is here to stay,
Though we know it's had it's day.
It's time to leave the plight,
And come to see the light!
NB: For those confused, I work with MrPointy who is an avid Java fan and coder. We all take pleasure in poking him in relation to his one-eyed views on Java.
If you are wondering on his choice of alias, read up on Buffy the Vampire Slayer, and her favourite wooden stake.
Your wicked Java way,
Ensures dodgy apps are hear to stay.
We all know Java sucks,
Yet you insist we try our luck.
Process freezes and applications hung,
Dont worry, Java has just begun!
We know you're a Java nut,
But don't think we'll fall in the same rut.
Write once, run anywhere,
Sucky applications we just can't bare.
Eclipse and NetBeans; your preferred IDE,
Give me a simple text editor; and forget IcedTea!
Java certainly has it's place,
Use it in Windows with it's poison lace.
System programming requires C,
Leave Java out of it, let us be!
Show me one decent Java app,
That memory it doesn't sap.
Just one app that will not crash,
That doesn't make my system thrash.
Python, C, and Perl is what to use,
It's bound to avoid the blues,
Use them next time you need a script,
Not that silly Java dialect!
There is no more to state,
Java just isn't that great!
Avoid coding in Java at all cost,
So your reputation is not lost!
Many claim Java is here to stay,
Though we know it's had it's day.
It's time to leave the plight,
And come to see the light!
NB: For those confused, I work with MrPointy who is an avid Java fan and coder. We all take pleasure in poking him in relation to his one-eyed views on Java.
If you are wondering on his choice of alias, read up on Buffy the Vampire Slayer, and her favourite wooden stake.
Monday, February 18. 2008
Laptop clarifications
Early this morning I blogged in relation to the OLPC and my own thoughts on solutions for Labors Digital Education Revolution policy.
Russell Coker picked up my thoughts and wrote an interesting article regarding Laptop vs Book Weight and made some good points.
To continue the discussion in the Blogsphere (man I hate that phrase!), I thought I would answer some of his questions, and clarify some of my initial comments.
What OS does the ClassMate PC ship with?
The Intel Classmate PC by default ships with Microsoft Windows XP. Close inspection of the images on their website suggests so. The Interactive Manual also clearly demonstrates a Windows based OS. As such, the default shipping system does not meet my own requirements -- without the removal of pre-installed operating system and software.
What age bracket does the ClassMate PC suit?
I did state in my article to mention that "The Classmate PC looks like more of a rival to the XO laptop more than anything." and whilst not clearly stating it; I see it more focused on primary school children then actual high school students. As such it fails the Digital Education Revolution policy (Years 9-12).
As Russell correctly points out it is 'designed for primary students (ages 5-14)'. It is also important to note that Intel have also have left the OLPC project. They are now squarely focusing on the promoting the Classmate over the XO laptop.
What computer candidates for high school children then?
I pointed to the fact of the Everex Cloudbook and Asus EeePC as two possible candidates. In addition it appears that others are about to hit the market in the next few months including a model from MSI as confirmed.
I do agree with Russell that whilst the EeePC is probably the front runner (with a SSD over a traditional HDD), though the screen size (7") is far too small. I would add that the keyboard would also be annoying for any length of time. WithUpdate: potentially 6 hours a day working on this beast, it's likely to be light enough for students to throw across a class-room in frustration.
I am also of the belief that Tablet PCs could also be reworked into a small form factor and low price (sub $500-$600AUD). It allows students to either elect to use a keyboard or a pen-based solution if preferred. In addition, drawing diagrams on a tablet is much easier. TabletPC screens generally are larger as well, making it suitable for viewing for long periods.
There is some partial Linux support for Tablet computing, but traditional Linux applications are likely to require linking to the correct libraries to provide support for native 'writing'. It seems Toshiba are also thinking along the same lines about a Tablet PC rework. TabletKiosk shows it can be done, and use OpenSuSE as an option on their range of Tablets.
Russell's main point that the he disagrees with my statement "A traditional notebook is far too heavy for high-school kids to lug around". In the adventurous mode that is Russell; he went digging through his old school books, and a set of scales to prove that current high-school text books are indeed heavier.
I could have saved him the pain... they are indeed heavier.
However, I'll stress again that my personal belief is that a traditional notebook is far too heavy for high-school kids to lug around.
Let me clarify this point.
Russell Coker picked up my thoughts and wrote an interesting article regarding Laptop vs Book Weight and made some good points.
To continue the discussion in the Blogsphere (man I hate that phrase!), I thought I would answer some of his questions, and clarify some of my initial comments.
What OS does the ClassMate PC ship with?
The Intel Classmate PC by default ships with Microsoft Windows XP. Close inspection of the images on their website suggests so. The Interactive Manual also clearly demonstrates a Windows based OS. As such, the default shipping system does not meet my own requirements -- without the removal of pre-installed operating system and software.
What age bracket does the ClassMate PC suit?
I did state in my article to mention that "The Classmate PC looks like more of a rival to the XO laptop more than anything." and whilst not clearly stating it; I see it more focused on primary school children then actual high school students. As such it fails the Digital Education Revolution policy (Years 9-12).
As Russell correctly points out it is 'designed for primary students (ages 5-14)'. It is also important to note that Intel have also have left the OLPC project. They are now squarely focusing on the promoting the Classmate over the XO laptop.
What computer candidates for high school children then?
I pointed to the fact of the Everex Cloudbook and Asus EeePC as two possible candidates. In addition it appears that others are about to hit the market in the next few months including a model from MSI as confirmed.
I do agree with Russell that whilst the EeePC is probably the front runner (with a SSD over a traditional HDD), though the screen size (7") is far too small. I would add that the keyboard would also be annoying for any length of time. WithUpdate: potentially 6 hours a day working on this beast, it's likely to be light enough for students to throw across a class-room in frustration.
I am also of the belief that Tablet PCs could also be reworked into a small form factor and low price (sub $500-$600AUD). It allows students to either elect to use a keyboard or a pen-based solution if preferred. In addition, drawing diagrams on a tablet is much easier. TabletPC screens generally are larger as well, making it suitable for viewing for long periods.
There is some partial Linux support for Tablet computing, but traditional Linux applications are likely to require linking to the correct libraries to provide support for native 'writing'. It seems Toshiba are also thinking along the same lines about a Tablet PC rework. TabletKiosk shows it can be done, and use OpenSuSE as an option on their range of Tablets.
Russell's main point that the he disagrees with my statement "A traditional notebook is far too heavy for high-school kids to lug around". In the adventurous mode that is Russell; he went digging through his old school books, and a set of scales to prove that current high-school text books are indeed heavier.
I could have saved him the pain... they are indeed heavier.
However, I'll stress again that my personal belief is that a traditional notebook is far too heavy for high-school kids to lug around.
Let me clarify this point.
- Text books (those awful paper form) are currently required. Adding another 1-1.5kgs (weight of a notebook) to a schools bag in unnecessary.
- Parents are already concerned with the weight of school bags already, adding extra kilograms to the bag won't go well with them and is likely to be met with resistance.
- The Victorian government has already published details on concerns of the weight of school bags. Some interesting statistics come out of a study undertaken by the