Three guard rails worth building every time

1. Safe to repeat

Your automation will run twice. A retry fires, two schedules overlap, someone triggers it manually while it is already going.

The property you want is called idempotence: running it twice leaves the same result as running it once. Ways to get it:

• Check-then-act. “Copy the file if it is not already there” rather than “copy the file”.
• Mark what is done. Keep a record of processed items and skip them next time. Write the marker after the work genuinely succeeded — never before, and never on an error.
• Use a lock. Refuse to start if another copy is already running.

A word on locks: make sure the lock is one that clears itself. A lock file left behind by a process that was killed will block every future run forever, and it will do it silently. Prefer a mechanism the operating system releases automatically when the process ends.

2. Safe to interrupt

Power fails mid-run. The network drops halfway through an upload. Ask: if I kill this right now, what state is it in?

The dangerous shape is a job that deletes before it writes, or writes a file in place over hours. Prefer: write to a temporary name, then rename when complete. A rename is instantaneous, so at any moment the destination is either the old complete thing or the new complete thing — never a half-written one.

3. Safe when the input is wrong

One day the file will be empty, or truncated, or yesterday's, or in a different format because a vendor changed something without telling you.

Add a sanity check on the input before acting: is it non-empty, is it roughly the expected size, is it newer than the last one, does it have the columns it should? Then stop if the check fails. Do not attempt to cope cleverly — a clever recovery from an input you did not anticipate is how a small problem becomes a data-loss incident.

The rule underneath all three

Refusing to run is always cheaper than running wrong. Build every guard rail to fail in the direction of doing nothing.