Use flock to Stop Overlapping Backup Jobs

Your backup takes longer than usual. Cron starts the next run anyway. Now two jobs compete for the same disks, and the task that was already slow gets even slower. Linux flock gives that problem a small, boring fix: only one cooperating job gets the lock.

I prefer a visible skipped run to a growing queue of backups fighting over the same storage. But the word cooperating matters. A lock is not a security boundary, and it does not magically coordinate every program that touches your files.

The short version: put a nonblocking exclusive lock inside the backup script, use one stable local lock path, and report contention separately from backup failure. Then deliberately start two harmless instances before trusting the setup.

A schedule is not a concurrency limit

Cron launches commands when their schedule matches. It does not know whether yesterday’s upload is still crawling along, and manually starting the same script can create another concurrent run.

Application-level locks sometimes help. A backup tool may already protect its repository, but that does not necessarily serialize your database dump, export cleanup, and upload sequence. Check the tool’s documented behavior before adding another layer.

Our backup guide covers what to preserve. This tutorial addresses a narrower question: how to stop two copies of the same local workflow from running together.

Put the lock inside the script

These examples target Bash and the util-linux version of flock on Linux. Confirm the executable locations first:

command -v bash
command -v flock
flock --version

Use the paths returned by your machine if they differ from the examples. Create a private lock directory as the account that will run the backup:

install -d -m 700 "$HOME/.local/state/backup-locks"

Do not run that command through sudo unless root is actually your backup account. Cron and manual invocations must use the same account and resolve the same lock path for this example to coordinate them.

Save this as backup-locked.sh:

#!/bin/bash
set -euo pipefail
umask 077

lock_dir="$HOME/.local/state/backup-locks"
if [[ ! -d "$lock_dir" ]]; then
  printf 'Missing lock directory: %s\n' "$lock_dir" >&2
  exit 1
fi

exec 9>>"$lock_dir/nightly-backup.lock"
if /usr/bin/flock --exclusive --nonblock --conflict-exit-code 75 9; then
  :
else
  rc=$?
  if [[ "$rc" -eq 75 ]]; then
    printf 'Backup skipped: another run holds the lock.\n' >&2
  else
    printf 'Could not acquire backup lock (exit %s).\n' "$rc" >&2
  fi
  exit "$rc"
fi

# Harmless demonstration. Replace with your foreground backup commands.
printf 'Lock acquired; starting demonstration.\n'
sleep 10
printf 'Demonstration finished.\n'

The shell opens file descriptor 9, then asks flock to acquire an exclusive lock on it. --nonblock refuses to wait when another instance holds that lock. Exit code 75 is our explicit convention for contention, not a universal backup status code.

The script keeps the descriptor open while its foreground commands run. On ordinary completion, closing the descriptor releases the lock, provided no other process still holds a reference to it. An empty lock file remaining afterward is normal.

I put the lock inside the script because a cron-only wrapper is easy to bypass during troubleshooting. A manual invocation should follow the same rule as the scheduled one.

Prove that a second run cannot enter

Make the demonstration executable and check its Bash syntax:

chmod 700 ./backup-locked.sh
bash -n ./backup-locked.sh

In one terminal, run:

./backup-locked.sh

While it is sleeping, run this in a second terminal under the same account:

./backup-locked.sh
printf 'Exit status: %s\n' "$?"

The second invocation should report that it skipped the backup and exit with 75. After the first finishes, another invocation should acquire the lock normally. These are expected results for your test, not evidence that a production backup has completed.

During preparation for this article, we also exercised the locking primitive locally: a competing process was rejected with 75, the lock could be acquired after release, and a wrapped command’s nonzero exit status was preserved. That validates those mechanics, not your storage, credentials, or recovery procedure.

Replace the demonstration, not the safety checks

Remove the demonstration’s three commands and insert your actual backup steps after the lock block. Keep them in the foreground. For a simple sequence, set -e stops after a failed standalone command; pipefail prevents an earlier failure in a pipeline from being hidden by a successful final command.

Bash error handling still has exceptions around conditionals and other constructs. If your workflow is complicated, handle important failures explicitly rather than treating set -e as a transaction manager.

Do not append & to your backup command and immediately exit the script. Children can inherit the descriptor, making the lifetime of the lock less obvious. Detached workloads need a deliberately designed supervisor and locking model.

The lock also does not make a database dump consistent or freeze an application’s uploads. Follow the application’s backup procedure, and test a restore independently.

Schedule the locked entry point

Install the script in a stable location owned by the backup account, such as /home/backup/bin/backup-locked.sh. For that account’s crontab, a daily example is:

17 2 * * * /home/backup/bin/backup-locked.sh

Replace the path with your real account and script location. Ensure that cron supplies the expected HOME, and use absolute paths for backup programs rather than depending on your interactive shell’s configuration.

Keep stderr visible to your logging or notification mechanism. Cron mail only helps when mail delivery is configured; an error nobody receives is not an alert. A skipped run should not send the same success heartbeat as a completed backup.

For scheduling with systemd, see our timer guide. A timer does not restart an already active instance of its target service, but a separately invoked script or another unit can still bypass that protection. Shared entry-point locking can cover those cooperating paths.

Do not delete the lock file to unlock a job

The lock belongs to the opened file, not a magical property of its filename. If you delete the pathname while a process holds the old file open, a later process can create a different file at that path and acquire a different lock. You have now defeated your own serialization.

Leave the file in place. If a lock appears stuck, investigate the processes and open descriptors before terminating anything. A surviving child might legitimately still be holding it.

Keep the directory private and on a local filesystem with working locking semantics. A writable shared directory invites pathname tampering, and NFS or CIFS behavior depends on the filesystem and mount configuration. The util-linux flock manual documents those limitations.

Know where this stops working

Two servers with separate local lock files do not coordinate. Two containers with separate filesystems do not coordinate either. Use a scheduler with appropriate concurrency controls or a distributed locking mechanism when the workflow genuinely spans machines.

Likewise, running the script as another Unix account changes $HOME and therefore changes this example’s lock path. For multi-account coordination, provision a shared lock location and permissions deliberately. Do not solve it by making a sensitive directory world-writable.

A commercial VPN has no role in acquiring this lock or making backups consistent. If you separately need outbound browsing privacy on an administrative device, NordVPN is an optional affiliate product, not a requirement for this tutorial:

Make the next skipped run useful

I would start with one job that occasionally overruns, not retrofit every scheduled command tonight. Test contention with the harmless demonstration, install the same guard in the real entry point, and verify how your monitoring reports exit 75.

Then investigate repeated skips. A lock prevents overlap; it cannot fix a backup that never finishes. Your goal is one completed, restorable backup on schedule, not merely one process holding a file open.