Occasionally I need to to the same task, or similar tasks, on a group of machines all at once. I don’t have the luxury of a fully automated environment like Puppet setup, so a great tool I’ve been using is called ClusterSSH.
Seeing as I was one of the lucky few to get their hands on an Asus Transformer Prime so soon, a couple people have asked me to write up a short review of the device.
Having never possessed a tablet before I do not have any baseline for comparison so there may be some skew in my opinions due to that fact. Some of my positives and negatives may be true for all tablets — or at least those of this size/form factor.
On a mail filter I maintain, there is a site-wide bayes database that is periodically trained by hand. It sits quietly and doesn’t change much over time. That is, until the Bayes database was moved to a new server. The SpamAssassin configuration was identical between the old system and the new system, there was just one problem: On the new server, the bayes_toks file was rapidly growing until it was quite large. Huge, in fact. Its size was expanding by several gigabytes per hour.
I checked all the usual things: auto learning was off, auto expiration was off, the permissions and user were set correctly, and so on. And yet it grew, constantly and swiftly.
After hours of searching and not finding anything, and various methods of tinkering, I found the answer. I backed up and restored the bayes database like so:
sa-learn --backup > bayes_backup.txt sa-learn --restore bayes_backup.txt
After that, the toks file was once again left alone and didn’t grow. I suspect the problem was due to moving from a 32-bit platform to a 64-bit platform but that’s just speculation really, or it could be some other difference in the perl versions and libraries on the two servers.
In case you couldn’t tell, I was trying to use a bunch of different ways to word this problem, going off of the various Google searches I did trying to track it down. Hopefully others will hit this post in the future and it will save them some time. :-)
I picked up a new Brother HL-2170w last week. It’s a network-enabled (wired and wireless) black and white laser printer. At just under $100, and with $30-50 replacement toner cartridges good for thousands of pages worth of output, it was quite a steal in terms of features for the money.
I had been looking for an all-in-one color printer to replace my scanner and inkjet, but the more I looked, the more horror stories I saw. And when I didn’t see horror stories, I saw absurd ink costs on a per-page basis. The sheer number of horror stories was astounding. I have always considered printers to be evil, banes of our very existence but the tales of woe that wait on printer reviews are really like no other components.
So as inconvenient as it may be for color, it seems that doing photo prints (99% of what I want in color) is cheaper to do at an in-store lab (such as Wal-Mart) than to do it at home, plus I don’t have to worry about the crazy quality of inkjet printers, or expensive-yet-not-photo-quality color laser output. I still have my existing inkjet for the rare case where I might need a print fast, but at what seems to be about $0.50/print, it’s really not worth it.
The Brother printer was pretty easy to setup. Plug it into the network, and it pulls a DHCP address. I went to the printer’s web interface and setup a static IP, changed its name, and had all the PCs printing in no time. The Windows 7 boxes found the driver automatically, XP I had to pick the driver by hand, and my FreeBSD workstation needed the hpijs-pcl5e driver for CUPS, but needed no other special configuration.
Given all the negative experiences I’ve had with printers (and everyone else has had), I thought I’d share this positive one.
Just a quick update to let everyone know that the pfSense book which I mentioned previously is now available from Amazon.
I haven’t been posting much lately, as per usual, and also as per usual I keep thinking I’ll get around to posting more. Well, this little tidbit does deserve a new post:
These past few months I’ve been working with the great folks behind pfSense, an awesome FreeBSD-based firewall system that has really impressed me at every turn. We’ve been using it quite a bit at work over the past year, so I’ve been contributing back in the form of documentation, code, testing, and other help wherever I can.
Along the way I started working with one of the project’s co-founders, Chris Buechler, on a book for the project. It’s now available from several retailers, and more will be coming soon.
So if you are interested in pfSense, FreeBSD, firewalls, or other related concepts, you’re bound to find something useful in our book:
As I wrote about previously, I have had problems with Apache and PHP crashing due to various PHP Extensions. I have come upon another combination that triggers a problem, but after investigating it a little I see that it’s been reported before, and nobody wants to fix it. PHP blames PHP accelerator systems, and Zend claims it’s a shared memory configuration problem (it isn’t — at least on my system)
The error happens whenever attempting a graceful restart of Apache via “apachectl graceful”:
- [notice] seg fault or similar nasty error detected in the parent process
- Apache 2.2.4
- PHP 5.2.1
- Zend Optimizer 3.2.8
- Some interaction between the Zend Optimizer being loaded along with the PHP pspell module.
If I disable one or the other, the crash goes away. Since this particular installation does not require the pspell module, I disabled it and things have been stable ever since.
I did follow Zend’s recommendations for increasing certain shared memory tunables on FreeBSD, as well as trying to recompile everything involved. For more information on shared memory tuning check the FreeBSD man page tuning(7) as well as this Zend Knowledge Base article. Note that certain sysctl settings may only be modified at boot time via /boot/loader.conf and/or /etc/sysctl.conf.
More information to come if I can find anything else…
Update 11/21/2007 – I found that in more recent version of PHP (Around 5.2.4-5.2.5) having pspell.so loaded before spl.so in extensions.ini will result in crashes when an httpd process is stopped/killed. Moving pspell anywhere after spl will clear this up (so far…).
We lucky folks in Indiana have had a rough two years dealing with time. As I wrote about last year, Indiana just started observing Daylight Saving Time (DST) in 2006. Now, for 2007 we also have to change the dates on which DST starts and ends. DST now begins on the second Sunday in March, and ends on the first Sunday in November — This year it’s March 11th and Nov 4th. Why on earth we didn’t just wait to start along with the new rules is anyone’s guess. <rant>I don’t think we should be using DST at all, but that’s a story for another time</rant>
Here I was, all set for another round of server updates, reboots, etc. Turns out that I didn’t need to worry quite so much. When I updated all of the time zone files on our servers last year for Indiana’s initial DST switch, they had already made the changes with the new start and end dates for 2007 and beyond. You can confirm this on most UNIX systems as follows:
# zdump -v /etc/localtime | grep 2007 /etc/localtime Sun Mar 11 06:59:59 2007 UTC = Sun Mar 11 01:59:59 2007 EST isdst=0 gmtoff=-18000 /etc/localtime Sun Mar 11 07:00:00 2007 UTC = Sun Mar 11 03:00:00 2007 EDT isdst=1 gmtoff=-14400 /etc/localtime Sun Nov 4 05:59:59 2007 UTC = Sun Nov 4 01:59:59 2007 EDT isdst=1 gmtoff=-14400 /etc/localtime Sun Nov 4 06:00:00 2007 UTC = Sun Nov 4 01:00:00 2007 EST isdst=0 gmtoff=-18000
If it says “Mar 11” and “Nov 4” you’re good. If it says “Apr 1” and “Oct 28” you need to update your time zone definitions. On FreeBSD, this can be as simple as downloading new zoneinfo files, recompiling them, and re-selecting the timezones:
- Download: ftp://elsie.nci.nih.gov/pub/tzdata2007b.tar.gz
- Exctact the contents to /usr/src/share/zoneinfo
- cd /usr/src/share/zoneinfo; make install
- Choose your time zone again.
A reboot may be necessary to ensure that all running programs are on the same time zone. Currently running programs may not pick up the change. You could also update FreeBSD to a recent version, which includes these changes. If you choose to do the OS update, be sure to run “tzsetup” afterward to be absolutely certain that a new tz file gets installed to /etc/localtime. After you’re done, re-run the zdump command above to be check that you now have the proper DST change dates for 2007.
If you are running any Cisco gear (or other IOS-alike devices) this should work to make the change:
clock timezone EST -5 clock summer-time EDT recurring 2 Sun Mar 2:00 1 Sun Nov 2:00
Adjust the time zone to yours, of course.
There may be other programs that handle time zone data internally (such as Java and Outlook) so you’ll have to be sure there are no loose ends in that department. Those of us in Indiana have some practice with this, so at least for us it may not be that bad.
I am aware that many of these problems could be avoided by using UTC on all our server clocks. While that may be preferable, we like to have everything in local time. It’s a choice, and we deal with the consequences. One of which is we never schedule jobs to run overnight between 1-3am — they could be run twice or not at all.
UPDATE: 3/1/07: I have also been informed that you can copy the “/etc/localtime” file from an updated system to any other system that needs it. This could be especially useful if you are unable to update all of the Time Zone definitions for any particular reason.
Once again when faced with updating PHP on a few servers, I encountered my favorite of all PHP quirks: After rebuilding extensions, PHP crashes and/or takes Apache down with it. Here are the errors that tend to show up:
- exited on signal 11 (core dumped)
- exited on signal 6 (core dumped)
- seg fault or similar nasty error detected in the parent process
And my personal favorite:
- httpd in free(): error: junk pointer, too high to make sense
I have seen this on PHP4 and PHP5, and with Apache 1.3 and 2.x. I’m not sure if it’s a problem inherent to how the FreeBSD ports system builds and installs the modules or if it’s just a problem in general. I had read once upon a time that rebuilding extensions in a certain order would fix it, and it did. I never got around to figuring out why this worked. Turns out, rebuilding them doesn’t really matter, but the order of the extensions being loaded does. Rebuilding fixed it because when a php extension port is rebuilt, it gets placed at the end of extensions.ini. I solved the problem by editing /usr/local/etc/php/extensions.ini and placing the lines for mysql, imap, and sockets at the end and in that order:
I’m not sure if the conflict is only with those three, or with others as well, but that fixes it on my servers. I tried it on three different setups, and before the change they all crashed and after the change they’re all running OK.
Hopefully if anyone else runs across this, it will help. If I get more time, I’ll dig into it more later.
There has been some more discussion of this on the FreeBSD-Ports mailing list and the FreeBSD-STABLE mailing list. Apparently at least part of this is due to the PHP recode, MySQL, and IMAP extension ordering. These extensions rely on c-client libraries with different overloaded hash functions. So the “magic” order at the end of extensions.ini should be:
There is also talk of building some logic into the PHP extension ports to ensure the ordering of the extensions so as to avoid this bug. Best of luck to those working on it!
Edit 8/25/07: I wrote a very hackish shell script that gets the job done keeping the extensions in this order. It’s not pretty, but it works. It can be found here: http://www.pingle.org/files/fixphpextorder.sh. Read the full post here.
Edit 11/21/07: Lately pspell has also become picky about ordering. I recommend placing it at the end (or at least anywhere after spl.so)