Andy's Unix FAQ

Good

This page has received an average rating of 85% from 20 readers

Be aware that most crontab commands attempt to validate the format of all lines before replacing the original crontab. The upshot of this is that if any lines in the file are incorrect the whole file will be rejected, and your changes lost.

On systems without the crontab command you are probably reduced to editing the crontab directly - In this case you may need to 'inform' cron about your changes - usually by sending it a HUP signal as root;  
    kill -HUP <cronpid>
Consult your system documentation for the locations of crontabs, man cron is a good place to start.

If when you run crontab you see the error;
    You are not allowed to use this program
then you may need to have a look at Setting up users.

• minute (0-59),
• hour (0-23),
• day of the month (1-31),
• month of the year (1-12),
• day of the week (0-6 with 0=Sunday).
• command

An asterix ('*') in any of the first 5 fields means All occurrences. Illegal or out-of-range values will cause the whole line to be ignored by cron.

An example of a typical (and not actually very good) cron entry;
27 16 * * 1-5 (cd /home/hems/prod/cron; ./alertsCronjobs.sh prod 3m. 1>>./alertsCronjobs.out 2>>./alertsCronjobs.err)

Means, at 16:27 on every weekday (Monday-Friday), change directory to /home/hems/prod/cron and run the script ./alertsCronjobs.sh with the 2 arguments prod 3m.

Stdout ("1>>"), the normal output from script, is redirected to the file alertsCronjobs.out Stderr ("2>>"), any error messages, is sent to the file alertsCronjobs.out.

Both of these files are appended to ( ">>") so will likely grow infinitely over time. Contrast that with the use of ">" which would cause the named files to be overwritten each time the script is run.

Without redirection cron will usually send all output by (Unix) email to the owner of the crontab.

The illustrated crontab entry is not terribly good because

Reliance on user's home directory

This is a root crontab. One should never rely on any user's home directory for such a task. What happens if 'hems' leaves, is demoted, fired, etc... ?

Starting a subshell

Whilst in itself not a huge problem the login shell of the user may be changed at some point with detrimental affects on the job.

Redirection

as a point of principle redirection should be done in the script itself.

Using cron successfully / Troubleshooting

Many newcomers to cron have difficulty getting their first cron job to run successfully. The most common symptom is that the job never seems to get scheduled. In reality the problem is generally that cron did run the job but that it failed for some reason. There are several key points to bear in mind when setting up a cron job

1. Cron jobs do not inherit the login environment of the user. Specifically environment variables set in .login or .profile, etc. will simply not be available. The absolute single most common reason cron jobs fail is that the PATH setting is not what the script writer assumed.
   
   
2. Cron jobs do not have a stdin associated with them. Whilst cron will not execute the .login/.profile of a user, it will start the shell normally - thus causing the shell's startup file (.cshrc,.bashrc,etc.) to be executed. Many people unwisely put commands in these files that rely on there being a terminal associated with the shell - common example;
       stty erase ^h
   Also some command used in the cronjob may be relying on a terminal being present. For example, if you are trying to start 'vi' in batch mode, Don't. Instead replace it with ed or sed.
   
   
3. Cron job output will be sent by Unix mail to the crontab owner. This includes all normal output (stdout) and error messages (stderr). I stress Unix mail because that may not be the email you are used to reading. See Mail from cron below.
   
   
4. Permissions on the crontab itself. Many cron implementations will not even look at a crontab if the permissions on the file are set incorrectly. Generally permissions should be extremely tight - with only read/write for the owner, and NO permissions for group or others.
   This is usually not a problem is you use the 'crontab -e' command rather than editing the crontab directly. Unfortunately not all systems provide the crontab command.
   
   

Mail from cron

Any output your cron job produces, both stdout and stderr, will be sent to you by Unix email. Errors from your script which make complete sense when run from the command line seem meaningless and curiously out of context when seen in a mail message, consider the mail message
   set: No match.
Most likely your C-shell script didn't match any files for some set command

Not many sites these days tie their corporate mail systems in with their Unix system. So you may have to learn a few commands in order to read your mail...

To check your mail whilst logged on to Unix use the 'mail' command. Marginally better is the BSD mail reader- on Solaris /usr/ucb/Mail on Linux 'mailx' (tip: hitting Return a lot generally works). If you find you need to do this on a regular basis you'll probably want to get a copy of Elm - a most respectable non-GUI mail reader.

Cron's subject line is usually 'Output from Cron'.

You may not be aware that most shells have built in mechanisms for checking your mail on a regular basis. ksh/bash users should checkout their documentation for the variables MAIL,MAILCHECK, & MAILPATH. C-shell users have a look at the set mail ...  command.

Cron will only send mail if there is some output from your script. Silence in this regard means nothing more.. If you want cron to not mail, then you should arrange within your script to redirect all output to both stdout and stderr somewhere else - probably to some log file. Administrators may wish to consider syslog entries - see Making your own Syslog entries in the companion Syslog FAQ.

Note: If your are reading this section because your mail box keeps filling up with the message:
   Not a Terminal
then you really need to be reading No Stdin on cron jobs as there is likely something in your shell's startup file causing this.

Writing scripts for Cron

Cron looks like a great facility to people who are new it to, but those same people quickly learn that writing a script that runs in cron is an order of magnitude more difficult than any other. On the plus side your scripting skills will improve in leaps and bounds once you master the difficulties.

Scripts run in root crontabs run with the full 'power' of the 'root' user. I would encourage you to avoid using root cronjobs wherever feasible. Murphy Law of computing applies;
    To Err is Human,
    But to really foul-up requires a Computer.

Be aware that root cron jobs represent a massive security hole if they are not managed correctly.

Set Environment

All cron scripts need to be bulletproof against changes in environment variables. This means all environment variables that your script uses (especially PATH) should be set explicitly in the script itself.

Error Checking

A cron script needs a much higher level of error checking than one run from the command line. If a command in a cron script can fail then the odds are at some point it will.
You need to consider EVERY line in a cron script for what will happen if it fails. This is why your scripting will improve greatly.

Explicitly redirect

Explicitly redirect all output streams to a fixed, known location and screw down the permissions on those files.

Always Specify Interpreter

Scripts should always explicitly name the interpreter they are to be run with via the '#!<shell>' keyword in the first line of the script. E.g. A Perl script might have as its first line;
     #!/usr/bin/perl

Test, Test, and Test again..

Long before your script goes anywhere near cron it should have been thoroughly tested. Your script is going to be run unattended and repeatedly by a very stupid computer. What's more if it goes awry you are going to be blamed.
Obviously the script should be run from the command line - check that it requires no intervention. Most of the time your script should not produce any output - if it does you'll want to annotate that output so that it makes sense in an email.

The best test is to background the script ( '&' ) using nohup and then Log Off. You should be able to log back in and see your script running.

Secure Script Permissions

Obviously scripts run in root's cron should be read/write for root only. You may choose others to be able to read the script but I advise against it - root cron jobs are a favourite target amongst Unix hackers.
Watch the permissions your editor leaves on the file after changing it - your umask should probably be 077.

Keep root cron scripts local

Scripts run in root's crontab should be kept on the local filesystem (i.e. not NFS, etc) wherever possible, and certainly when you don't control the fileserver. This improves both the security and resilience of the job.

Be Considerate

When writing scripts that will run frequently, have a thought for the system and any other users. That job which runs every 5 minutes scanning your home directory - is it really needed ? Is there a better way? What happens if it takes more than 5 minutes to complete? Such antisocial practices may well cause your username to be quietly added to cron.deny..

Ask someone who knows

Generally the Unix Sys Admins on a site know more about cron than anyone else. Most Unix systems come with a few cron jobs already installed and if your SA has been in the job for more than a few weeks he or she has almost certainly had to deal with it.

Finally, if you have to write a root cron script and lack a little confidence get someone else to look at it before putting it live. I've seen a whole network of Unix boxes wiped out by a single misplaced space character in a root crontab....

Debugging Cron

If the tips above haven't helped, you probably now want to turn logging on.

I'll mention now that cronlog generally isn't that much help, 99% of the time cron is running your job and the real problem is with your script. Personally I generally skip cronlog and just start debugging the script.

Enabling Cronlog

Cron logging is not enabled by default because of the huge log files it tends to generate. The means of enabling cron logging varies between platforms, and requires root access.

Solaris admins need to modify the file /etc/default/cron and change CRONLOG=NO to CRONLOG=YES. You will need to stop and restart cron. The log file itself is normally /var/cron/log.

Linux's cron uses syslog. To enable full cron logging you should create a new entry in /etc/syslog.conf that reads something like;

          cron.debug	      /var/log/cron

Then 'touch /var/log/cron' and finally kill -HUP syslog to tell it to re-read its config file.

Reading Cronlog

If you haven't seen a cronlog yet you're likely wondering why a section on it ? Below is an excerpt from a Solaris cronlog showing two cronjobs;

>  CMD: [ -x /usr/sbin/rtc ] && /usr/sbin/rtc -c > /dev/null 2>&1
>  root 383 c Mon Jun 16 02:01:00 2003
<  root 383 c Mon Jun 16 02:01:00 2003 rc=1
>  CMD: /export/home/andrew/quote/insert_html_quote.pl
>  andrew 435 c Mon Jun 16 02:10:00 2003
<  andrew 435 c Mon Jun 16 02:10:01 2003
 

The 'CMD' line shows the command as it appears in the crontab. ">" at the start of the line indicates the startup of the job, "<" it's completion. The owner and pid of the child process is shown along with the start/end times. "rc" stands for Return Code, if the job exits with a non-zero status (failure) this is shown as "rc=XXX".

As you can see cronlog is not terribly useful.

In the highly unlikely event that you cron job does not get run skip to Your Job is not appearing in Cronlog. Most likely though you'll just need to read on...

Your Job is appearing in the Cronlog

Now that you have proved that cron is running your script you have to face the possibility that there is a problem with your code. First have a another look at Troubleshooting and Writing Scripts for Cron.. Before reading on, check your Unix mail - you are looking for the subject line 'Output from Cron'.

Options for debugging scripts run from cron are limited. The principle I employ revolves around the crude, albeit effective technique of making entries in a log file. This simple mechanism is also extremely portable - works in any language, no debugging tools required. The following examples use bourne shell syntax.

Close to the top of the script put three lines like;
    logfile="/tmp/myscript.log"
    cp /dev/null $logfile
    echo "This is a message from myscript `date`" >> $logfile

Then at strategic locations throughout the script put lines like these;
    ...
    echo "HERE 1" >> $logfile
    ....
    echo "HERE 2" >> $logfile
    ...
    echo "HERE 3" >> $logfile

Reschedule your script using 'crontab -e' and check /tmp/myscript.log to see how far your script gets. Insert more "echo HERE.." lines and repeat until you find the offending statement.

When you do find the problem try not to kick the nearest filing cabinet and scream at your computer as this many offend your colleagues

Your Job is NOT appearing in the Cronlog

There are few reasons for this unlikely state of affairs;

• cron is not running. How to check
• crontab entry is incorrect. Quite often the hour / minute fields are inadvertently swapped. Fields in crontab
• Your script doesn't have execute permission; chmod 700 <script>
• The script interpreter is not where it should be - check first line of script
• Your username is either not in cron.allow, or is in cron.deny. See Setting up Users.
  

Your Job is sometimes in Cronlog

How can this be ? Well cron makes a number of checks before running each job, one such being a call to getpwent(3c). getpwent retrieves the user's passwd entry from whatever source is specified in /etc/nsswitch.conf. Medium and larger sites often use a "naming service" such as NIS or LDAP to centralise user account administration.

If your job appears to be run sometimes and not others, it could be that the name service in use is experiencing intermittent problems. Check your system logs (generally Syslog) around the scheduled time of your job.
[ Thanks to Pete B. with Perot systems for pointing this one out ]

Still got problems ?

Then you are probably reduced to debugging cron at a process level. The most basic tool being the sturdy old 'ps' command. Check that cron is actually running thus;

      sun (ksh) % ps -ef | grep cron
      root 159 1 0 May 24 ? 0:16 /usr/sbin/cron 

Setting up Users for Cron

    cron.allow
    cron.deny

As with all things Unix related the location of these files varies with the platform. Under Solaris they live in /etc/cron.d, under Linux in /etc.

The format of these files is simple; one username per line. Their interaction together is logical but takes some thinking about:

If the allow file exists then a user must be listed in it to be able to use cron. If the allow file does not exist but the deny does, then the user must not be listed in deny in order to be able to use cron.

With this is mind it should be obvious that maintaining both cron.allow and cron.deny doesn't make a lot of sense.

The logic of this arrangement is simply that it allows administrators to either only allow certain users to use cron, or allow most people whilst excluding a number of troublemakers.

How you use these files depends very much on your site and what function individual machines perform. Most Unix variants default to allowing most users, with a cron.deny file in place for pseudo-user accounts. This is probably fine for general use systems.

For systems that perform critical functions, whether they be system maintenance (e.g.Backups), or mission critical you probably want to reverse the logic by removing the cron.deny and listing a minimal number of users in cron.allow.