How adaptive polling works: 3 minutes at the gate, 12 hours a week out

Airlines do not push. Under every flight webhook anyone sells there is a poll, and the whole product is the answer to one question: when do we ask next? Ask too often and a flight three weeks out costs as much as one at the gate. Ask too rarely and a gate change reaches the passenger after they have walked to the wrong pier.

This is the function that answers it, as it runs in FlightByte today, with the numbers and the reasons those numbers exist. If you are building your own poller, take it. If you are deciding whether to, the previous guide is the argument.

The shape of the problem

A flight's information changes at a rate that depends on where it is in its life:

• A week out, the schedule changes rarely and matters little. Nobody is at the airport.
• The day of, gates, terminals and delays appear and move. The cost of being an hour late is small; the cost of being ten minutes late is not yet real.
• The last hour before departure, everything moves and somebody is standing in a queue reading a board.
• In the air, little changes until the approach, when the arrival estimate firms up.
• After landing, there is a gate to learn, then a baggage belt, then nothing ever again.

A fixed interval is wrong at both ends: wasteful early, late when it counts. The fix is not a better interval. It is a function of the flight's state.

The decision function

Trimmed of imports and logging, this is decideNextPoll from the poller. It takes the latest snapshot and the clock, and returns the delay before the next poll or null to stop watching.

export const decideNextPoll = (snapshot, now, postArrivalGraceMinutes) => {
  const phase = snapshot.status.phase;

  if (phase === "cancelled") {
    // Cancellations are occasionally reversed, so keep a light watch briefly.
    return { delayMs: 30 * MINUTE, reason: "cancelled: light watch for reinstatement" };
  }

  if (phase === "arrived") {
    const arrivedAt = snapshot.arrival.times.actualGate?.utc;
    const sinceArrival = arrivedAt ? (now - Date.parse(arrivedAt)) / MINUTE : Infinity;
    if (sinceArrival >= postArrivalGraceMinutes) {
      return { delayMs: null, reason: "arrived: grace window elapsed" };
    }
    // The baggage belt is often published after the gate arrival.
    return { delayMs: 5 * MINUTE, reason: "arrived: watching for baggage claim" };
  }

  if (phase === "landed") return { delayMs: 2 * MINUTE, reason: "landed: awaiting gate arrival" };

  if (phase === "diverted" || phase === "redirected") {
    return { delayMs: 5 * MINUTE, reason: "diverted: tracking closely" };
  }

  if (phase === "en_route") {
    const eta = snapshot.arrival.bestTimeUtc;
    const minutesToArrival = eta ? (Date.parse(eta) - now) / MINUTE : Infinity;
    if (minutesToArrival <= 45) return { delayMs: 3 * MINUTE, reason: "en route: approaching arrival" };
    return { delayMs: 10 * MINUTE, reason: "en route" };
  }

  if (phase === "departed" || phase === "boarding") {
    return { delayMs: 3 * MINUTE, reason: "off gate: awaiting takeoff" };
  }

  // Still scheduled: before departure, only the clock can decide.
  const departure = snapshot.departure.bestTimeUtc;
  if (!departure) return { delayMs: 30 * MINUTE, reason: "no departure time known" };

  const msToDeparture = Date.parse(departure) - now;
  if (msToDeparture < -6 * HOUR) {
    // Departure long past but the status never advanced: the feed is stale.
    return { delayMs: null, reason: "stale: departure over 6h ago with no status change" };
  }
  if (msToDeparture <= 45 * MINUTE) return { delayMs: 3 * MINUTE, reason: "departure imminent" };
  if (msToDeparture <= 2 * HOUR) return { delayMs: 10 * MINUTE, reason: "within 2h of departure" };
  if (msToDeparture <= 6 * HOUR) return { delayMs: 20 * MINUTE, reason: "within 6h of departure" };
  if (msToDeparture <= 24 * HOUR) return { delayMs: HOUR, reason: "within 24h of departure" };
  if (msToDeparture <= 48 * HOUR) return { delayMs: 6 * HOUR, reason: "within 48h of departure" };
  return { delayMs: 12 * HOUR, reason: "far future" };
};

Two rules of thumb are hiding in the order of those branches.

Before departure, the clock decides. After it, the phase does. Until the aircraft moves, the only thing we know is the departure time, so the interval is a function of distance to it. Once the status advances, time-to-departure is meaningless and the phase takes over: three minutes off the gate, ten in cruise, three again on approach, two once it has landed. The approach case reads the arrival estimate; a flight is more interesting in its last 45 minutes than its first.

Terminal states are not quite terminal. A cancelled flight is watched every 30 minutes for a while, because cancellations get reversed and flight.reinstated is a real event. An arrived flight is watched every five minutes until the baggage grace window, 45 minutes by default, has passed, because the belt is usually published after the gate. Only then does the function return null, the subscription becomes completed, and nothing about that flight is ever fetched again.

There is one more null: a flight whose departure time is more than six hours in the past and whose status never moved. That is not a flight, it is a stale record, and polling it forever would be paying for nothing.

What it decides, in a table

The reason strings are not decoration. They are stored with every poll and shown in the admin panel, so "why was this flight last checked forty minutes ago" has an answer that is not a log file.

Whatever the function returns is clamped: never faster than once a minute, never slower than once every twelve hours. The clamp is there so a bug in the policy cannot become a bill.

Flights that do not exist yet

Subscriptions can be created a week ahead, and airlines publish the operational record a few days out. Until then the upstream has no row for the flight, and there is no snapshot to read a departure time from. A second, smaller function handles that, keyed on the subscription's date:

While this runs the subscription is pending. When the flight appears it becomes active, you receive subscription.activated, and the main function takes over. If the date goes by without the flight ever appearing, subscription.completed says so.

What one flight costs

Adding the intervals up over a two-hour flight gives the poll count by how far ahead the subscription was created:

A five-minute loop over the same six hours makes 72 requests and is still five minutes late at the gate. The adaptive schedule spends about the same on a same-day subscription, is three times as fresh where it matters, and costs almost nothing for the week before.

The number that matters more: subscribers on the same leg share one poll. Fifty bookings on tonight's TK1848 are one job in the queue, not fifty. Adding a subscriber never adds upstream load, which is why plans are priced by watched flights rather than by subscriptions.

How it runs

The policy is a pure function; the machinery around it is where production lives.

• One delayed job per flight key. The job id is the flight key, so a second subscription on the same leg finds the job already there. The next poll is scheduled from the job's completed event, replacing any queued job, so the newest decision always wins.
• A reconciler every five minutes. Delayed jobs survive restarts, but a job that exhausts its retries would otherwise silently stop a flight from ever being polled again. The reconciler asserts that every watched flight has a job in the queue and re-creates the ones that do not. Its first run after a deploy is staggered by a few random seconds so a rolling restart does not stampede the upstream.
• Retries with backoff, and a breaker. A failed poll retries with exponential backoff from ten seconds. All upstream calls go through one client with a shared token bucket and a per-provider circuit breaker that opens after eight consecutive failures, so an upstream outage degrades into a pause rather than a flood.
• Then the diff. The new snapshot is compared with the previous one and turned into typed changes. Each subscription gets only the change types it asked for, with delay changes filtered by its own delayThresholdMinutes, and one delivery per subscription goes to the webhook queue with its own retries and log.

Lookups are not polls

GET /v1/flights/… reads from a cache whose lifetime also follows the flight: thirty minutes a day out, five minutes on the day, sixty seconds within three hours of departure or in the air. A page that shows the current state on load is cheap and included in the plan. Polls, by contrast, always go upstream; that is their purpose, and it is why they are the thing you pay for.

If you need an answer that is guaranteed fresh, refresh=true on a lookup skips the cache. If you need to be told, subscribe.

What you cannot tune, and why

There is no per-subscription interval. The cadence is the product: it is what makes a watched flight cost what it costs and arrive when it arrives, and a knob that let one subscriber poll every minute for a week would be a knob that let one subscriber set everyone's bill. The knobs you do have are the ones that shape what you hear: events for which change types, and delayThresholdMinutes for how much a delay has to move before you are told.

If the policy above is wrong for a case you have, tell us with the flight. Every row in that table came from one.