Prevent Running Of Duplicate Cron Jobs

Overview

When you set up a cron job, sometimes you need to make sure that you will always have just one running instance at a time. This is useful also when you want to be sure that a script that can take longer than expected does not get executed again if the previous call hasn’t finished.

This can be done with another caller shell script that detects a running instance before executing it again (e.g.: using pidof), or you can use programs written specifically to handle this situation (e.g.: flock or run-once).

In this tutorial we will be using different approaches to have just one instance of an example script called main.sh that simply prints “Hello” in an infinite while loop.

#!/bin/sh
while true
 do
    echo "Hello"
    sleep 2
 done

And make it executable.

$ chmod +x main.sh
$ ./main.sh
Hello
Hello
...

Script approach

We make a script caller.sh that launches the program we want to run only if the process’ id of the script isn’t already running.

To find the process ID of our running script (omitting the calling script) we use pidof.

In caller.sh:

if pidof -o %PPID -x "main.sh">/dev/null; then
    echo "Process already running"
	exit 1
fi

-x     Scripts too - this causes the program to also return process id's of shells running the named scripts.
-o omitpid
             Tells pidof to omit processes with that process id. The special pid %PPID can be used to name the parent process  of
             the pidof program, in other words the calling shell or shell script.
EXIT STATUS
	0      At least one program was found with the requested name.
	1      No program was found with the requested name.

Then the first time we run caller.sh it will launch main.sh, successive calls would exit the script.

$ ./caller.sh
Hello
Hello
...

If we try to launch it again from another shell:

$ ./caller.sh
./main.sh already running

Lock file approach

A lock file is an ordinary file that it is created before executing the script, and removed after the script finishes.

This way if any other command tries to execute the same script using the same lock file it will exit or wait until it can run it.

Using flock

Most Linux distros already comes with the flock command.

flock manages locks from within shell scripts or from the command line.

We use flock to execute the script, specifying explicitly the lock file to use, and to exit if the script is already running with the -n parameter.

$ flock -n /tmp/myfind.lock myscript.sh

For example, editing the crontab to execute the command every 5 minutes should look like:

*/5 * * * * /usr/bin/flock -n /tmp/ms.lockfile /usr/local/bin/my_script --some-parameter

Every time the script takes longer than 5 minutes to execute, the cronjob will fail and leave the original script to finish.

Using run-one

In some distros like Ubuntu, there is also the run-one utility that handles the lock automatically.

Example crontab with run-one for the previous script:

*/5 * * * *   run-one /usr/local/bin/my_script --some-parameter

Summary

It is a good practice to avoid duplicate running instances of cron jobs as they may lead to several problems that should not be overlooked and handled accordingly when defining them.

Reference

• Crontab http://man7.org/linux/man-pages/man5/crontab.5.html
• Prevent duplicate cron jobs running
• File locking
• shell script execution check if it is already running or not
• pidof(8) - Linux man page
• flock(1) - Linux man page
• run-one Ubuntu manual