Three tutorials ago, a walkthrough for the installation of Windows Software Update Services was detailed, allowing all client workstations to receive system updates from your server, as opposed to each having to download these patches from Microsoft.

The same thing can be done under Linux by setting up YUM (Yellow Dog Updater, Modified) repositories and configuring all clients to update from this location instead of online repositories. The primary advantage to doing this is to minimise internet data transfers, saving your quota for other things.

The only pre-requisite for setting up a YUM repository on a Red Hat-derived system is a web server — the Apache HTTP Server should already be running on your server as we set this up last month. Aside from that, keep in mind that every update released will be downloaded into your mirror — so you’ll also need to reserve at least 3GB of disk space for each architecture and each version that you intend to maintain.

All your base. . .

When creating your mirror, you are essentially creating two repositories. The first is the base OS repository, which contains all of the packages that your installation media holds. This repository will be used by clients when you install new packages via YUM — as opposed to obtaining these packages from the media or online. The second repository is for system updates.
From within your DocumentRoot, create a couple of new directories under a path that identifies which version and which architecture of CentOS you are mirroring — one for the base OS files and another for updates. For instance, the following commands should be issued if you are mirroring CentOS 5.1 on the i386 architecture:

mkdir –pv /var/www/html/centos/5.1/os/i386

mkdir –pv /var/www/html/centos/5.1/updates/i386

Next, copy the installation RPMs from your CentOS 5.1 CDs or DVD to a subdirectory under the OS directory that identifies what architecture the base installation files belongs to — this way, your mirror is capable of storing installation data for different architectures (many small business networks will often have a mix of i386 and x86_64 workstations). For example, these commands would copy across the i386 installation RPMs:

mount /mnt/cdrom

mkdir /var/www/html/centos/5.1/os/i386

cp /mnt/cdrom/CentOS/*.rpm /var/www/html/centos/5.1/os/i386

umount /mnt/cdrom

After copying the RPMs across, generate header files for each RPM (the headers for each RPM file contain the list of dependencies, version details and more, and are used by YUM to determine the requirements for installation of any package) using the yum-arch command:

cd /var/www/html/centos/5.1/os

yum-arch

After processing, a headers directory will have been created and all header information will reside in this directory.

Adding Updates

Once your base repository is populated, it should not ever require modification (though you may like to add additional data when a newer version of CentOS is released, or if you plan on supporting additional architectures). Our second repository needs to contain system updates, and the easiest way to retrieve these is to synchronise them with a trusted mirror. This guide will provide instructions on how to use rsync to synchronise with Planet Mirror’s servers in Brisbane, but many ISPs have their own unmetered mirrors which may further preserve your data quota.

To manually synchronise your updates repository, execute the following command:

rsync –avrt rsync://rsync.planetmirror.com/centos/5.1/updates/i386 –exclude=debug /var/www/html/centos/5.1/updates/i386

If you are able to successfully synchronise the repository, add the same command as an entry to /etc/crontab so that a synchronisation is regularly performed (eg. you may want a sync nightly at 2:00am). There is no need to generate the headers for this repository as we did for the OS repository because these should be synchronised along with the updates from the upstream mirror

Push the little changes

After both repositories have been populated, you then need to modify the YUM configuration on all systems so that they pull updates and base packages from your new mirror. To do so, edit the /etc/yum.repos.d/CentOS-Base.repo file and:

• comment out the mirrorlist directives;
• modify and uncomment the baseurl directives under the [base] and [updates] sections to point to your repositories. For example, your file should look similar to:
  

...

[base]

name=CentOS-$releasever – Base

#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os

baseurl=http://server/centos/$releasever/os/$basearch/

gpgcheck = 1

gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5

#released updates

[updates]

name=CentOS-$releasever – Updates

#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates

baseurl=http://server/centos/$releasever/updates/$basearch/

gpgcheck = 1

gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5

...

Once you’ve saved the changes, you can copy the file to all other workstations in your network and perform yum updates quickly, easily and locally.

Improving the Firewall

During the installation of CentOS, you would have been prompted whether you wanted to enable a kernel-based firewall after installation along with a few basic configuration options. You can reconfigure the settings for this firewall from within the System Settings. The trouble with the default firewall is that it’s not exactly flexible; specifically, it does not provide egress filtering (much like the firewall that comes shipped with Windows). APF (Advanced Policy Firewall) is a very good, broad-level Linux firewall script that utilises the same kernel module (iptables) that the default firewall uses — but it’s considerably more flexible and extendable. Download the APF tarball to root’s home directory, uncompress it and run the installation script:

cd

wget http://www.r-fx.ca/downloads/apf-current.tar.gz

tar –zxvf apf-current.tar.gz
apf-0.9.6-3/install.sh

Once installed, edit APF’s config file (at /etc/apf/conf.apf) with all of the values that are relevant to your specific environment. Some of the directives that you will want to be aware of are:

• DEVEL_MODE — when this directive is set to ‘1’, the firewall configuration will be used for five minutes, after which the configuration will be flushed from the kernel. This setting is useful to enable when you are making changes to the firewall from a remote system — that way, if your firewall configuration locks you out of the server, you can regain access a few minutes later.
• IFACE_TRUSTED — this directive should contain a list of the network interfaces on your system that should NOT have firewall rules applied. If your server has two network interfaces — one that faces your network and the other that faces the internet — you may want to add the internal interface to the trusted list.
• IG_TCP_CPORTS — this directive should contain the list of ports that you wish to allow inbound TCP connections to. In general, you’ll only want to enable the ports for services that you have running on your server that you want to be accessed via the internet — 21 (FTP), 22 (SSH), 25 (SMTP), 80 (HTTP), 110 (POP3), 143 (IMAP), 443 (HTTPS), 8080 (HTTP Tomcat) are all good candidates for our small business server.
• IG_UDP_PORTS — as above, but for UDP connections. 20 & 21 (FTP) are good candidates.
• EGF — when set to ‘1’, this directive will enable egress filtering on the firewall.
• EGF_TCP_CPORTS — the list of ports that you will allow outbound connections to be made on. At a bare minimum, you’ll want to allow your server to access ports 21 (FTP), 25 (SMTP), 53 (DNS), 80 (HTTP) and 443 (HTTPS).
• EGF_UDP_CPORTS — similar to the directive above, but for UDP Ports. You’ll definitely want to enable ports 20 & 21 (FTP) along with 53 (DNS)

There are a plethora of other options available and the config file is well commented, so take the time to go through all configuration directives. Once you’ve finished making changes, save the configuration file and start apf by running:

service apf start

Keep in mind that if you have DEVEL_MODE set to ‘1’, the firewall will be active for only five minutes.

Adding further protection

BFD (Brute Force Detection) is another package made by the same author that ties in very nicely with APF. Essentially, it parses your log files and looks for authentication failures. If too many failures occur in too short a time from one net host, the IP address of that host is then added to the list of denied hosts within APF. This prevents malicious users from performing brute-force attacks on your server. Installation of BFD is also relatively simple — download, extract and execute the installation script:

cd
wget http://r-fx.ca/downloads/bfd-current.tar.gz
tar –zxvf bfd-current.tar.gz
bfd-0.9/install.sh

 The install script copies all relevant files to /usr/local/bfd and adds a file named ‘bfd’ to /etc/
cron.d. This latter file instructs the cron daemon to execute BFD checks every eight minutes. A config file for BFD does exist under /usr/local/bfd, but it generally doesn’t require modification. If BFD detects a brute force attack, a couple of lines will be added to the /etc/apf/deny_hosts.rules file, which instruct APF to block all connections from the IP address where the attack originated.
If an address has been mistakenly blacklisted,
you can remove the entry from this file.

Automatic recovery
No application is ever perfect — services sometimes crash, even under Linux. If you have no monitoring systems, you often don’t know that a service has crashed until an end user has reported an issue to you. An application called SIM (System Integrity Monitor) can be used to regularly check that services are running, and restart them (or even restart the server) if they have stopped.
The installation of SIM is a little different to that of APF and BFD in that the installation script has a different name and is interactive — you’ll have to supply configuration details to the installation script before all files are copied:

cd
wget http://r-fx.ca/downloads/sim-current.tar.gz
tar –zxvf sim-current.tar.gz
sim-2.5.4/setup –i


The only things that you’ll need to know when running the setup script is what services are meant to be running on your system — the script should handle the rest. For instance, if you indicate that you want SIM to monitor the httpd service and restart it if it has stopped, the script should determine the path to the executable
and should detect how to restart the service automatically. However, scripts aren’t perfect, so you may have to supply these details to the script if it cannot detect appropriate values. If worse come to worse, you can always edit the configuration, which is located at /usr/lib/sim/conf.sim by default. After the setup script has been executed, SIM will perform a check of the system every five minutes based on the line inserted into /etc.crontab.

More monitoring
Two of the limitations of SIM are that it only monitors a few services and it only monitors the local host. For instance, SIM will not detect whether OpenLDAP has stopped responding
and, as a result, will not attempt to restart that particular service. One of the most popular open-source monitoring systems is Nagios — and it’s
as flexible and extensible as monitoring systems come. In fact, Nagios is so flexible, it’s a little difficult to know where to start. Installation is reasonably quick and painless, but you can spend an eternity configuring and extending it to do just about anything if you so wish.
While RPM binaries for Nagios can be found, the most reliable way to install it is to manually compile it yourself. First, ensure that all of your dependences have been installed:

yum install gcc gd gd-devel glibc glibc-common httpd


After your dependencies are installed, create a service account and service group for use by Nagios and ensure that the apache and Nagios service accounts are members of the new group:

useradd nagios

passwd nagios

groupadd nagcmd

usermod –G nagcmd nagios

usermod –G nagcmd apache

Next, download the Nagios source and Nagios Plug-ins source tarballs to the root’s home directory (this site will redirect you to your nearest SourceForge mirror for the actual download). Once downloaded, extract all files from the Nagios tarball and run the configure script:

tar –zxvf ngaios-3.0rc2.tar.gz

cd nagios-3.0rc2

./configure –-with-command-group=nagcmd

If the configure script runs successfully without error, you can now compile and install Nagios:

make all

make install

make install-init

make install-config

make install-commandmode

make instal-webconf

Before starting Nagios, you should review the base configuration. All configuration files are located under /etc/local.nagios/etc. Essentially, the main configuration file (nagios.cfg) contains all of the core configuration directives. Towards the top of this file, you’ll see that a number of other configuration files — the object definition files — are also called. A detailed guide on how to configure Nagios itself could be a multiple-part Workshop series, so there’s not the space to go into detail here. However, http://nagios.sourceforge.net/docs/3_0/configobject.html is a good place to start learning what objects can be defined within Nagios and how different types of objects relate to each other.

For the meantime, the only file that requires immediate changes is /usr/local/nagios/etc/objects/contacts.cfg — you’ll need to change the email address associated with the nagiosadmin contact definition to a valid email address. This needs to be done so that Nagios knows who to send alerts to should any of the monitored objects go into a warning or critical state.

An account for nagiosadmin then needs to be created within the Apache web server. Access to Nagios should be restricted because you probably don’t want unauthorised users to be able to view the status of services running on your network, let alone have the ability to restart them or run other commands on your server.

htpasswd –c /usr/local/nagios/etc/htpasswd.users nagiosadmin

service httpd restart

Without the base plug-ins, Nagios can’t really check the status of any service. These should be built next:

cd

tar –zxvf nagios-plugins-1.4.11.tar.gz

cd nagios-plugins-1.4.11

./configure –-with-nagios-user=nagios –-with-nagios-group=nagios

make

make install

Once your plug-ins are good to go, you can add Nagios to the services list, verify the configuration and then start Nagios (assuming that your configuration is valid):

chkconfig –-add nagios

chkconfig –-level 2345 nagios on

/usr/local/nagios/bin/nagios –f /usr/local/nagios/etc/nagios.cfg

service nagios start

Point your web browser to http://server/nagios, log in, and you should see a few basic service checks being performed on your local server. If you occasionally get ‘Internal Server Errors’ when viewing Nagios, your SELinux settings may need tweaking. To instruct SELinux to permit the execution of the CGIs needed to generate some Nagios pages, execute the following commands:

chcon –R –t http_sys_content_t /usr/local/nagios/sbin/

chcon –R –t http_sys_content_t /usr/local/nagios/share/

Once your base configuration is running, experiment and expand. With a bit of scripting nous and effort, Nagios can monitor just about anything (for instance, I personally have an instance of Nagios that checks the Bureau of Meteorology’s radar loops to determine if a severe storm is approaching my house, and sends an SMS alert to me if it detects one).

Trending

With any luck, you have a server that operates in a stable and reliable manner and you have implemented tools that should automatically restart services if they fail, or message you to indicate that something is wrong well before users report problems: it’s always a good feeling when you fix severe problems before anyone else realises that something’s amiss. If you want to go even further, have a look at RRDTool and some of the front ends to RRDTool (such as Cacti). These tools provide the ability to graph performance metrics over time, which can help identify long-term issues on your server and can help provide
a picture of server health and capacity.   

Next: The folks who use Apple Macs.