Centralized Syslog With FreeBSD

It is impossible to manage log files across a large multitude of servers. To assist with this syslog is able to send all messages to a single server. This makes keeping track and filtering much easier. It also reduces the time it takes to check the logs as only one login is needed. In this article only the use of the default syslog server which comes with FreeBSD will be covered. There are other ways of centralizing, viewing, and beautifying logs such as with syslog-ng and php-syslog-ng which will not be covered here.

In this example we'll use a fake environment with only the central syslog server and a single workstation for simplicity. Our small networks syslog server will be fbsd-syslog with an ip of 192.168.0.10 and the workstation will be called fbsd-wrk with an ip of 192.168.0.11.

First we must tell the central server to listen to fbsd-wrk. To do this we must append to /etc/syslog.conf . It is always a good idea to back up the original file to something like /etc/syslog.conf.bak in case things go wrong. Add the following to the end of /etc/syslog.conf on the central server (fbsd-syslog).

!*
+192.168.0.11
*.* /var/log/messages

This is simplicity at it's finest. It allows 192.168.0.11 access to the central server to write any file named *.* to /var/log/messages. By default each install sends logs to different locations. For example, by default FreeBSD has a line showing cron.* /var/log/cron . This sends all log files called cron.* to /var/log/cron. We could separate cron messages from our workstation (fbsd-wrk) by doing the following.

!*
+192.168.0.11
cron.* /var/log/cron.fbsd-wrk
*.* /var/log/messages

There are many more variations and applications other than cron that send messages. Any of these can be sent to a different location as above. A little Googling on the net will produce more details. Continuing with our simple example of receiving all messages and putting them on /var/log/messages, we now need to tell fbsd-wrk to send it's messages to fbsd-syslog. To do this fbsd-wrk must have it's /etc/syslog.conf file modified to read like the following.

*.* @192.168.0.10

That is all that should be present in fbsd-wrk's /etc/syslog.conf file. Now simply restart syslogd on both machines by running the following as root.

# /etc/rc.d/syslogd restart

To verify that this worked log into the central syslog server, fbsd-syslog in our example, and type the following.

# tail -f /var/log/messages

In another terminal log into the workstation, fbsd-wrk in our example, and type the following.

# logger hello world

Our "hello world" message should appear on the `tail` running on the central server. If the syslog server is receiving a lot of log entries, the log files may be turned over too frequently. This can be remedied by changing the "size" column entry in /etc/newsyslog.conf . That's it, enjoy.

Distributed Command Execution with Perl

In modern workplaces, it is unreasonable to expect system administrators to manually run commands across vast numbers of servers. It is simply inneficient and unessesary. In order to resolve this many remote administration tools have been created. Tools vary from distributed shells to configuration management sofware. These tools may be overkill for smaller environments. To add to the multitude of distributed tools the following Perl script was created. Using the script is simple. First change the @bsdhosts array elements to the hostnames in your environment. Second, change the word "user" to a valid username that can ssh into the remote hosts. Finnally, change `pwd;ls` to whatever command needs to be performed on the remote hosts. To run multiple commands within one session, simply use a semicolon to seperate the commands as shown below. Passwords can be avoided by using `keygen`. There are many sites describing how to do this on the net. Here is one such location to get you started http://gentoo-wiki.com/SECURITY_SSH_without_a_password . In short, for a small environment where there is no need for a complex distributed solution a simple Perl script can step up to the task. Enjoy!

#!/usr/bin/perl

###########################################################
# Summary: An easily modified script to run remote commands.
#
# Last modified: 01/30/2008
#
# Author: Javier Prats
#
###########################################################

use warnings;
use diagnostics;
use strict;

our $counter;
our @bsdhosts=qw(hostname1 hostname2 hostname3 hostname4);

foreach $counter (@bsdhosts)
{
       my @command= ("/usr/bin/ssh user\@$counter pwd;ls");
       system(@command);
}

Checking Disk Space Remotely

Recently I have been looking for ways to monitor disk space of remote servers. We have a Nagios server that has the NRPE plugin. This works fine on Linux, but I recieved an error on FreeBSD. Creating a script to do these checks seemed like it would be quicker than configuring the NRPE plugin to work. Before reinventing the wheel I Googled around and found a very helpful bash script here which can also be found below.
-

#!/bin/bash
ADMIN="me@somewher.com"
ALERT=70
ssh user@ip-address.com df -H  > /tmp/df.out
cat /tmp/df.out | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $1 }' | while read output;
do
#echo $output
usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1  )
partition=$(echo $output | awk '{ print $2 }' )
if [ $usep -ge $ALERT ]; then
echo "Running out of space \"$partition ($usep%)\" on $(hostname) as on $(date)" |
mail -s "Alert: Almost out of disk space $usep" $ADMIN
fi
done

Although, this does what I am attempting to accomplish, we do not have Bash on our FreeBSD servers. Below you can find my Ksh93 port of the script.

1 #!/usr/local/bin/ksh
2
3 ############################################################################################
4 #
5 # Summary:
6 # This script checks disk usage on remote hosts.
7 # Based on a script by Nixcraft posted on
8 # http://nixcraft.com/shell-scripting/3238-shell-script-check-disk-space-remote-systems.html
9 # which was modified and ported to Ksh by Javier Prats
10 #
11 # Author[s]: Nixcraft, Javier Prats
12 #
13 # Last Modified: 11/28/07
14 #
15 ############################################################################################
16
17 ADMIN="user@emailaddress.com"
18 ALERT=70
19
20 typeset -i usep
21 typeset -A hostnames
22 set -A hostnames hostname1 hostname2 hostname3 hostname4
23
24 for i in ${hostnames[@]}
25 do
26         print "checking $i";
27         ssh user@${i} df -H > ~/df.out
28         cat ~/df.out | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $1 }' | while read output;
29
30         do
31                 #echo $output
32                 usep=`echo $output | awk '{print $1}' | cut -d'%' -f1`
33                 partition=`echo $output | awk '{print $2}'`
34                 if (($usep >= $ALERT))
35                 then echo "Running out of space \"$partition ($usep%)\" on ${i} as on `date +%m/%d/%Y`"|mail -s "Alert: Almost out of disk space $usep" $ADMIN
36                 fi
37         done
38 done

A few things will need to be modified. First the "ADMIN" variable on line 17 needs to be a valid email address to recieve alerts on. Second, an array was added to the original script in order to be able to check multiple servers. Change "hostname1", "hostname2", etc on line 22 to valid hostnames you would like to check. Finally, this script uses ssh, so the user on line 27 must be modified to show a real username. On line 18 there is an "ALERT" variable. This sets the threshold for email alerts. By default email alerts are sent when disk usage is above 70%. This value can be changed to whatever is deemed reasonable. Enjoy.

Bugzilla on FreeBSD

When asked to install Bugzilla at work, I wasn't concerned since FreeBSD has it in the ports tree. However, when it came to installing the necessary Perl modules I came accross a few issues. To install Bugzilla, the documentation at http://www.bugzilla.org/docs/2.22/html/index.html was being used. Three of the required modules were producing "/usr/bin/make --NOT-OK" errors. This result could not be replicated on all machines so it does not appear to be a problem with the modules themselves. I found workarounds for all three modules. Here is how to get them installed if you experience the same snags and are limited on time. First was the MIME::Parser module. MIME::Parser also had a "must use force to install" message in it's error. From the Perl CPAN shell, accessed by using `/usr/bin/perl -MCPAN -e shell`, simply type `make -f MIME::Parser`. Second was the Mail::Mailer module. After some research I found that this module is included in /usr/ports/mail/p5-Mail-Tools. I suspect there is some difference in modules as the port version installed without any complaints. Finnally was the Image::Magick module. ImageMagick's Makefile shows an option USE_PERL5=YES. In experimentation I installed ImageMagick without any other knobs. To ensure that this resolved the problem I ran Bugzilla's checksetup.pl again. That's all there was to the process. There wasn't enough time to actually research what the cause of these problems was. If anyone else has come across these please feel free to post your findings. Anyone else in a similar situation should find this to be an acceptable and fast workaround with a larger footprint being the only consequence.

Burning audio CD's in FreeBSD

There are many places which show the process of extracting tracks from a CD and then burning the audio to a backup CD. Most of these directions however are very "hands on". Below you will find a way to automate the process with a simple Perl script. It is not very intelligent (I may work on this further later) but at least it saves a lot of typing. If your CDROM device is not "acd0" make the appropriate changes. Simply copy this script into a text file, save it, and make sure it's made executable. You will probably have to run this script as root.

#!/usr/local/bin/perl

##########################################
# Summary:
# This is a script to remove some of the
# redundancy and monotony of burning an
# audio CD in FreeBSD.
#
# Dependancies:
# Perl, dd, burncd
#
# Last modified: 08/27/2007
#
##########################################

use warnings;
use diagnostics;
use strict;

my @tracks;
my $counter;
my $extension=".cdr";

# Make sure the correct files are in /dev.
# Retaste the media.

system(`dd if=/dev/acd0 of=/dev/null count=1`);

# Create a list of all tracks in /dev
system(`ls /dev|grep acd0t > tracklist.tmp`);

# Assign all the tracks in the list to
# the @tracks array.
open(TRACKS, "tracklist.tmp");
chomp(@tracks=<TRACKS>);
close (TRACKS);
system(`rm tracklist.tmp`);

# Rip each track.
foreach $counter (@tracks)
{
system(`dd if=/dev/$counter of=$counter$extension bs=2352`);
}

# Prompt for a blank CD and burn it.
print "Please enter a blank CD and press enter.";
<STDIN>;
system(`burncd -f /dev/acd0 audio *.cdr fixate`);
system(`rm *.cdr`);

Using VNC

Most of us work in hybrid environments.  In most cases end users will have Windows machines, designers will be on OS X, and the servers will be running some form of *nix.  Rdesktop is a very good solution if one just needs to RDP into a Windows box, but what if you need to get on an OS X machine or need to see Xorg on another machine.  VNC is available for all three of these platforms.  VNC performs its job well although slow.

I'll mainly be describing the FreeBSD configuration as this is a BSD site and FreeBSD it the flavor I'm most familiar with.  This process should be very similar across all the *nix.  Feel free to add OS specific instructions.  I'll gladly post them.  Use the relevant porting system or package manager in your case.  For FreeBSD it is available in the ports tree.  If the machine which will be installed on only needs the client, VNC can be compiled without it.  From /usr/ports/net/vnc type `make -DWITHOUT_SERVER install clean` and only the client will be built.  All the options can be seen in the MakeFile for ports.

If the computer is going to accept VNC clients the server needs to be started as the user that will be logging in.  For example, if user guest01 wants to vnc into a FreeBSD VNC server from a Windows machine he/she must log into the FreeBSD machine as guest01 (or `su` as guest01) and run `vncserver`.  VNCserver will ask for a password.  Whatever you provide as a password is what will be used to authenticate the client.  For simplicity I recommend using the same password that is used to login to the FreeBSD machine.  VNCserver will then show the hostname followed by :1 .  This shows what port VNCserver for user guest01 is running on.  Now user guest01 can start VNCviewer from his/her windows machine, fill the host field with "hostname:1", type password that was set, and should connect.

When VNCviewer is started for the first time it defaults to the TWM window manager.  Each user has a .vnc directory in his/her home directory containing a file called xstartup.  This file is used similarly to .xinitrc.  If one prefers to use Fluxbox for example, simply comment out or delete the line containing twm and add fluxbox&.  This will cause VNCviewer to start in Fluxbox from now on.  The file will look similar to the following.

#!/bin/sh

[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
#twm &
fluxbox&

As sessions are started by different users the port numbers will increase.  A lock file is created in /tmp for each session.  This tells the vncserver what is available for users to use.  Using the above example, there will be a file called /tmp/.X1-lock .  These lock files remain and as users forget what port they were on and start new VNCserver instances port numbers and lock files can grow out of control.  Investigate which sessions are not being used and then `kill` them.  Running `ps aux|grep vnc` returns the vnc sessions, their owner, and what port it is running on.  Finnally go into /tmp and remove the lock file for the relevant port.

VNC does not have good security.  Although it is beyond the scope of this quick how-to, it is possible to tunnel VNC over SSH.  This adds great encryption with a minimal hit on performance.  Below are two good articles explaining how tunneling can be accomplished.

http://www.vnc.com/pipermail/vnc-list/2005-October/052697.html

http://www.oreillynet.com/cs/user/view/cs_msg/24540


Although VNC is not the smoothest or most secure way to graphically connect different operating systems it is one of the most compatible and easiest to use. 

reCAPTCHA

Recently, Slashdot featured an article about a new version of CAPTCHA called reCAPTCHA.  CAPTCHA is the letters that are usually crossed out or distorted in some fashion on web pages.  Usually the user is asked to re-enter these letters for validation purposes.  CAPTCHA prevents bots and form completing software from accessing the site.  Since these letters and numbers are usually images, as opposed to actual characters, bots cannot recognize them.  Sites using CAPTCHA can now give back to the community by using reCAPTCHA.  Instead of using random numbers and letters purely for authentication, reCAPTCHA uses text from scanned books which image recognition software did not validate.  This means when a user validates he or she is actually contributing to the act of publishing one of these scanned texts to the web.  This is a fantastic idea!  Web sites continue to deny bots access while at the same time helping release new text to the community.  Implementing reCAPTCHA onto a site is not a very complicated task.  Users are required to enter a little more text then with CAPTCHA, but this is a trivial down side considering the benefits in my opinion.  It would be great to see webmasters, developers, and admins contribute the small amount of time it would take to convert their sites to reCAPTCHA.