tar "Cannot open: No such file or directory" in Docker when the folder exists on your machine

tar failed in the container because the folder it writes into, data/deploy/, is git-ignored: it existed on my PC and nowhere in the container, and creating it before tar runs fixed it. On 29 October, Rubric’s edition was published at 05:50 UTC, the morning email went out with a link to it, and at 06:10 my live check reported the edition live. It wasn’t. The page answered 404 until I deployed it by hand at 09:31 UTC, and the first of the three things that went wrong was this line in the job’s log:

tar -czf ../../app/data/deploy/site-….tar.gz: Cannot open: No such file or directory

A folder that only existed on my PC

The deploy builds the site, packs it with tar -czf into data/deploy/ and uploads the archive to the server. On my PC that folder existed: it’s git-ignored, and an earlier run had created it. The day before, the jobs had moved into containers on a VM, and there the folder existed nowhere. .dockerignore keeps data out of the image, and the data volume is mounted at /data, not /app/data. tar doesn’t create the folder for the file it writes, and when that folder is missing you get exactly this error.

The fix is one line before tar, plus a test that archives into a folder that doesn’t exist yet. Simplified:

// before
await run("tar", ["-czf", relative(dir, file), "."], { cwd: dir });

// after
mkdirSync(dirname(file), { recursive: true });
await run("tar", ["-czf", relative(dir, file), "."], { cwd: dir });

A step that reported done

That error should have stopped the morning. It didn’t, because the upload sat inside the publish step, wrapped in a try/catch:

} catch (err) {
  run.log.error(`site deploy failed: ${(err as Error).message}`);
  built += " · site deploy FAILED";
}

The step logged the failure, wrote site deploy FAILED into its summary line and finished. The scheduler saw a finished step and had nothing to retry. The email went out in the same step, linking to a page that wasn’t there.

The deploy is now a stage of its own after the publish. It fails like any other stage, alerts on its first failure only, and the scheduler runs it again every quarter hour for six hours after the publish. The email waits until the upload has landed.

A check that trusted the database

This is the part worth copying. The live check asked my own database:

if (row?.status === "published") return `edition ${date} is live`;

published meant the edition was committed and pushed. It never meant a reader could open the page, and the check treated the two as one. Now it asks the site:

const res = await fetch(url, { method: "HEAD", redirect: "manual", signal: AbortSignal.timeout(20_000) });
return { live: res.status === 200, status: res.status, url };

A missing edition page answers 404 to a HEAD request just as it does to a GET, so a 200 means a reader gets the page. redirect: "manual" counts a redirect as not live, so the URL needs its trailing slash. Without it, the server answers 308.

A row that says “published” is the pipeline’s own account of what it did. Readers get the page, so the page is what gets checked, and nothing that links to the edition trusts the database alone any more. The social posts at 08:00 and 13:00 ask the page first, and while it isn’t 200 they post nothing and send me an alert.

One more race

The 06:10 check and the 08:00 posts also hit a smaller bug that morning. Two job containers pulled the same git checkout in the same second and broke each other’s fetch (“Cannot fast-forward to multiple branches”). Each pull now waits up to two minutes for a lock on the shared volume:

flock -w 120 /site/.pull.lock git -C /site pull --ff-only