# Missed Scheduled Jobs: Decide Whether to Skip, Catch Up, or Backfill

<a id="main-content"></a>

Operations &amp; Systems

October 28, 2026  ·  8 min read  ·  Victor Valentine Romo

On this page

1. [Separate the Appointment, the Period, and the Attempt](<#separate-the-appointment-the-period-and-the-attempt>)
2. [Inventory a Fictional Three-Day Interruption](<#inventory-a-fictional-three-day-interruption>)
3. [Compare Skip, Selective Reprocessing, and Full Backfill](<#compare-skip-selective-reprocessing-and-full-backfill>)
4. [Inventory Effects Before Replaying Anything](<#inventory-effects-before-replaying-anything>)
5. [Give Each Interval One Owner and One Evidence Record](<#give-each-interval-one-owner-and-one-evidence-record>)
6. [Check Airflow's Version and Recovery Controls](<#check-airflow-s-version-and-recovery-controls>)
7. [Reconcile Each Output Against Its Interval](<#reconcile-each-output-against-its-interval>)
8. [Set Stop Conditions Before Starting the Range](<#set-stop-conditions-before-starting-the-range>)
9. [Sources](<#sources>)

The scheduler returns. Three reporting periods are missing.

Running the job once might prepare today's report. Running it repeatedly might prepare today's report three times. Neither action establishes what happened to the missing periods.

Start recovery with an interval inventory. Identify the outputs the business still needs, the inputs available for those periods, and the effects another attempt could create.

The worksheet below proposes a bounded recovery method. Its reporting job, interruption, records, counts, and decisions are fictional. The example reads synthetic records and prepares local draft reports. It sends no messages and changes no business source records.

<a id="separate-the-appointment-the-period-and-the-attempt"></a>

## Separate the Appointment, the Period, and the Attempt

Use **scheduled time** for the intended appointment. Use **execution time** for when processing starts. Use **data interval** for the period represented by the output.

For Airflow 3.3.2's daily interval example, the logical date marks the interval's start, not the actual execution time. Manual runs differ: their interval depends on the timetable and trigger path. Inspect the resolved boundaries instead of assuming they follow a supplied logical date. The [Dag Runs documentation](<https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dag-run.html>) explains both cases.

Use **[retry](</articles/api-rate-limits-retry-budget/>)** in the recovery worksheet for another attempt at the same task and period after a failed attempt. Use **catchup** for the scheduler creating missing historical interval runs under its configured policy. Use **backfill** for a deliberately bounded historical processing request.

Airflow documents scheduler catchup separately from [backfill creation](<https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/backfill.html>). Other systems may use these labels differently. Record the engine and its definitions before translating an operating decision into a command.

For the business record, preserve all three [timestamps](</articles/event-time-processing-time-business-reports/>) or boundaries. A job that executes on October 25 can still represent October 21. Labeling that output only with its creation date would hide its purpose.

The [reporting automation workflow](</articles/reporting-automation-b2b-services/>) identifies where the report is prepared. Add the interval definition beside that workflow. Recovery needs the same definition as ordinary processing.

<a id="inventory-a-fictional-three-day-interruption"></a>

## Inventory a Fictional Three-Day Interruption

Consider a daily synthetic request report. Each report counts records by status for [one UTC calendar day](</articles/automation-time-zones-daylight-saving/>). Include the starting midnight and exclude the following midnight.

The report reads an immutable synthetic input file. It prepares a draft in an isolated review directory. A reviewer decides whether the draft represents the intended interval. There is no automatic distribution or publication step.

The fictional scheduler is unavailable across three intended appointments. After it returns, the coordinator finds this inventory:

| Reporting interval, UTC | Expected appointment | Evidence found | Required decision |
| --- | --- | --- | --- |
| October 21 to October 22, 2026 | October 22 at 00:10 | No accepted draft | Is this period still required? |
| October 22 to October 23, 2026 | October 23 at 00:10 | Partial temporary output | Discard as evidence of completion; preserve for inspection |
| October 23 to October 24, 2026 | October 24 at 00:10 | No accepted draft | Is this period still required? |

These are three missing accepted outputs. The temporary output is not a fourth period. It also does not make the second period complete.

Keep [the interruption record](</articles/incident-postmortem-small-service-team/>) separate from the interval inventory. The interruption describes unavailable execution. The inventory describes missing business evidence. One outage can affect several outputs differently.

Before choosing a recovery path, confirm that the synthetic input file covers all three periods. Record the file identifier and [checksum](</articles/checksum-verification-business-file-handoffs/>). A historical query against today's changed source would need a different evidence statement.

<a id="compare-skip-selective-reprocessing-and-full-backfill"></a>

## Compare Skip, Selective Reprocessing, and Full Backfill

Ask the report owner which historical outputs remain useful. Use that answer to choose a recovery scope.

**Skip** closes a missing period without preparing its report. Choose it only when the owner accepts the gap. Record the interval, reason, [approver](</articles/consulting-deliverable-acceptance-criteria/>), and downstream consequence. Leave the absence visible.

**Selective reprocessing** prepares only named intervals. Use it when a specific review needs one historical period or when inputs exist for only part of the gap. Record the excluded intervals and their disposition separately.

**Full backfill** requests every required interval in the bounded gap. Use it when the owner needs the complete sequence and the inputs support that sequence. Full refers to the approved range, not every historical date the scheduler can discover.

| Choice in the fictional case | Work requested | Evidence after review |
| --- | --- | --- |
| Skip all three | No historical drafts | Three explicitly accepted gaps |
| Reprocess October 22 only | One isolated draft | One reviewed report; two unresolved or accepted gaps |
| Backfill the three-day range | Three isolated drafts | Three individually reconciled reports |

None of these choices is automatically correct. If the reports support [a daily comparison](</articles/missing-intervals-business-time-series/>), skipping changes what the comparison can show. If only one period informs a pending decision, preparing all three may add unnecessary work.

Write the selected scope before processing. Do not let the number of runs the tool offers determine which reports the business needs.

<a id="inventory-effects-before-replaying-anything"></a>

## Inventory Effects Before Replaying Anything

List every step the job can execute. Include branches that are normally quiet, [downstream triggers](</articles/business-system-dependency-map/>), and cleanup work.

For the fictional report, the permitted effects are reading synthetic files and writing draft files in the recovery directory. The source remains unchanged. The reviewer controls any later use of the drafts.

If a real job also sends notifications, updates records, or starts another workflow, [pause the replay plan](</articles/incremental-api-sync-checkpoint-safety/>). Identify the authorization and controls for each effect. A request to rebuild a report does not resolve those separate decisions.

Keep [webhook delivery recovery](</articles/webhook-architecture-b2b/>) separate from the scheduler's interval inventory. A delivery attempt and a missing daily report are different units of work. Record their relationship if one triggers the other.

Inspect the selected workflow version. A historical interval does not necessarily imply historical processing logic. State whether recovery uses the current definition or a preserved earlier definition.

If a dependency cannot be inspected, mark its effects unknown. Reduce the scope to a verified read-only path or hold recovery. Do not treat absence of an observed effect as proof that the branch cannot execute.

<a id="give-each-interval-one-owner-and-one-evidence-record"></a>

## Give Each Interval One Owner and One Evidence Record

Assign a recovery coordinator and an output reviewer. The coordinator controls the approved interval list. The reviewer accepts or rejects the resulting reports.

Use one evidence row per interval. Record the input identity, processing definition, attempt identity, output location, and acceptance state.

| Evidence field | Fictional record |
| --- | --- |
| Job | synthetic-request-summary |
| Interval | 2026-10-22T00:00:00Z through 2026-10-23T00:00:00Z, end excluded |
| Definition | request-summary-v1 |
| Input identity | Named fixture file and recorded SHA-256 |
| Recovery attempt | recovery-02 |
| Output | Separate draft path for this interval and attempt |
| Duplicate check | No accepted output for the same interval, input, and definition |
| Acceptance | Pending reviewer reconciliation |

The checksum identifies the input bytes used. It does not establish that those bytes contain every required record. Keep [completeness](</articles/data-quality-dimensions-business-reports/>) checks separate.

Define a proposed deduplication key from the job, interval, input version, and processing definition. Check for an accepted output with that key before preparing another candidate. Keep attempt identifiers separate so failures remain traceable.

If two operators claim the same interval, stop one attempt and resolve ownership before continuing. A shared list without an enforced reservation is a coordination aid. Do not describe it as a concurrency guarantee.

<a id="check-airflow-s-version-and-recovery-controls"></a>

## Check Airflow's Version and Recovery Controls

The documentation reopened for this article identifies Airflow 3.3.2. Its default `scheduler.catchup_by_default=False` creates the latest interval at activation; `catchup=True` requests missing interval runs. Check deployed overrides before relying on that [documented default](<https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dag-run.html>).

Do not enable catchup merely to see what happens. First determine whether the workflow reads the assigned interval or uses the current date. The fictional recovery contract requires explicit start and end boundaries in every attempt.

Airflow 3.3.2's [backfill controls](<https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/backfill.html>) distinguish reprocessing modes. `none` avoids existing logical-date runs; `failed` permits failed ones; `completed` permits completed or failed ones. Queued or running latest runs prevent another creation.

Its backfill concurrency limit operates independently of the DAG limit. Its dry run lists candidate dates; creation still depends on run states at execution. Treat the list as planning evidence.

This worksheet supplies no executable backfill command. Have the engine owner translate the accepted interval list into the installed version's controls. Review the resulting candidate dates before authorizing execution.

For the fictional three-day report, propose one active recovery interval at a time. Require the coordinator to release each next interval after the reviewer accepts its predecessor. A concurrency limit alone does not impose that review step. These are proposed operating rules for the example.

<a id="reconcile-each-output-against-its-interval"></a>

## Reconcile Each Output Against Its Interval

Use a tiny synthetic fixture before processing the approved range. Include records on both boundaries and at least one duplicated identifier that the definition must handle explicitly.

For the fictional October 22 interval, prepare nine synthetic input rows. Place six inside the interval, one before it, and two at or after its excluded end. The six included rows contain five distinct request identifiers. The sixth row repeats one request with identical fields.

For this fixture, count each request identifier once after applying the interval filter. [Identical duplicates](</articles/crm-data-quality-automation/>) contribute no extra count. Stop for review if duplicate identifiers have conflicting fields. The expected count is five distinct requests.

Assign three distinct requests to open status and two to closed status. After processing, require a total of five with those two subtotals. These values are fixture expectations, not business statistics.

Check the output's interval label, input checksum, definition identifier, and attempt identifier. Then inspect the selected records. A matching total can conceal the wrong membership if one excluded record replaces one required record.

Repeat the acceptance check for each historical interval. Do not approve three reports from one successful sample. Keep any period with unavailable inputs in a separate held state.

Airflow 3.3.2 determines run status from terminal leaf tasks and warns about success despite earlier failures under certain trigger rules. Its [status documentation](<https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dag-run.html>) reinforces why a green run is insufficient output evidence.

<a id="set-stop-conditions-before-starting-the-range"></a>

## Set Stop Conditions Before Starting the Range

Write stop conditions where the coordinator can use them during execution.

For the fictional recovery, stop on an unexpected interval, changed input checksum, conflicting accepted output, missing source period, unexplained subtotal, or effect outside the read-only reporting boundary.

Also stop if recovery overlaps ordinary processing without an agreed ownership rule. Assign the next current interval explicitly. Do not assume that finishing historical work transfers responsibility for today's output.

When a stop condition occurs, preserve the attempted interval and its evidence. Mark remaining intervals unprocessed. Record [the owner of the next decision](</articles/decision-journal-business-operators/>) and the condition required to resume.

After recovery, reconcile the full approved list. Every interval needs a reviewed report, an accepted gap, or a named unresolved condition. A job counter alone cannot express those distinctions.

Put the final record in the [operations handover](</articles/operations-handover-small-team/>). Include the next ordinary appointment and the person who owns it. Keep temporary drafts separate from accepted outputs until their reviewer decides.

Close the gap with evidence for every period.

<a id="sources"></a>

## Sources

- [Apache Airflow 3.3.2: Dag Runs](<https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dag-run.html>). Logical dates, catchup configuration, and run-status limitations. Checked 2026.10.28.
- [Apache Airflow 3.3.2: Backfill](<https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/backfill.html>). Bounded historical runs, reprocessing choices, concurrency, and candidate-date dry runs. Checked 2026.10.28.
