Status and scope

This is the current checked-out contract for packages/collector. Numeric values below describe code and tests, not a promise that the upstream feed will provide enough data to satisfy them. The specification covers normalized vehicle observations and the derived records used by the tracker, ETA endpoints, service state, and optional road analytics.

Normalized input

The collector polls vehicle locations on a 20-second cycle. A normalized observation MUST contain:

The upstream parser also records a raw_source_events row for each fetch attempt when it can journal it. Source outcomes are ok, network_error, http_error, invalid_json, contract_error, or invalid_data. When that journal write succeeds, a failed fetch has source evidence but cannot produce a normalized observation.

Raw storage and dedupe

Every valid normalized observation is inserted into vehicle_observations. The table has a unique constraint on (vehicle_id, reported_at). A duplicate MUST be ignored and counted; it MUST NOT be treated as a collection failure.

The collector preserves reportedAt and collectedAt separately. Movement order, age, and derived intervals use the reported event time. Collection latency uses the difference between the two. Raw persistence completes before transit intelligence runs. If the raw batch fails, that batch has no derived processing attempt.

Route position quality

The projector selects canonical route geometry using the observation's route ID and direction. It stores route progress in meters and as a ratio, the current stop segment, the nearest stop, and both route and stop distances.

There is no heading-delta rejection rule in the current projector. Older 120-meter and 60-degree values are not part of this contract.

Transit event lineage

The derived records form a lineage, not a single confidence value. A route position can support transit events, a stop candidate, or a separate road-matching run, with each branch retaining its own failure state.

This diagram answers which derived records a usable position can create. It does not mean every position creates every record. Quality, progress direction, trip continuity, stop evidence, and optional road matching gate each edge.

Trip runs

A good position MUST start an open trip_run when no open run exists for the vehicle. The processor continues an open run only when the vehicle, route, and direction match, the reported-time gap is positive and no more than 10 minutes, and the route has not wrapped from late progress to early progress.

When continuation fails, the old run is closed and a new run starts. It is completed only when its starting progress ratio is at or before 0.25 and its last progress ratio reaches at least 0.75. Otherwise it is abandoned. An abandoned run records incomplete evidence; it does not assert that a vehicle physically abandoned service.

Stop passages and segment runs

For a continuing run, forward progress creates a passage for every ordered stop whose passage progress is greater than the previous position and no greater than the current position. The processor interpolates the passage time between the two report times. A transit_segment_run is written only when the immediately preceding stop passage exists for the same trip and the calculated travel time is positive. Its interval is the previous passage's observed time to the current passage's observed time.

Stop visits

A good position starts a stop_visit_candidate when its nearest stop is within 75 meters and its speed is at most 5 km/h. When upstream speed is null, the processor uses route progress to derive speed only if a previous position has the same route and direction, a positive gap no greater than 60 seconds, and derived speed at most 5 km/h.

The candidate is continued while it refers to the same trip and stop and still meets the dwelling condition. When it ends, it becomes a stop_visit only if it has at least two samples. The visit stores arrived_at from the candidate's first report, departed_at from its last report, observed dwell as the interval between them, sample count, and 40 seconds of uncertainty. The uncertainty represents two current 20-second collection intervals. A one-sample candidate is cleared without a visit.

ETA and confidence

Stop-arrival queries select each vehicle's latest good route position when its reportedAt is no more than 120 seconds old and its segment is before the requested stop. The vehicle-specific ETA query uses the latest good position; the resulting estimate has low confidence when that position is more than 90 seconds old. If no good position exists, the response is unavailable with reason: no_position. If the requested stop is not on the vehicle's current route, the reason is stop_not_on_route; if the stop is behind the position, the response is passed.

For each remaining segment, the baseline selector requires 30 historical segment runs. It prefers samples from the same Davao weekday and hour, then the same hour, then all samples for that segment. If none reaches 30, it uses a fixed 18 km/h speed and a p90 equal to 1.5 times the p50, and exposes the segment as a fallback.

The estimator sums the p50 and p90 of the remaining segment durations, scaling the current segment by the remaining route share. The current implementation does not add a separate stop-dwell median to this sum. That distinction is important when reading stop_visits: dwell is stored for intelligence and profiles, but it is not silently added to this ETA formula.

Recent segment durations are grouped by vehicle over the last 30 minutes. At least three independent vehicles and a current p50 at least 1.3 times the historical p50 are required for a live adjustment. A base high-confidence estimate remains high when five or more vehicles support the adjustment; an adjusted estimate supported by three or four is capped at moderate. Two vehicles do not adjust the estimate. Recent samples are excluded from historical baseline selection.

The ETA response MUST expose p50Seconds, p90Seconds, confidence, and evidence counts for segments, historical segments, fallback segments, minimum historical samples, and position age. This gives the caller a way to distinguish a measured estimate from a fallback or an old position.

Route service state uses the same segment and passage evidence but has a separate contract. It reports no_recent_service when there is no last departure or the last departure is older than two hours. It reports limited_data when the usual headway has fewer than 30 samples. With a baseline, a last departure or latest headway at least 1.25 times the usual p90 is long_gap; a latest headway at most 0.6 times the usual p50 is bunched; otherwise the state is normal.

Optional road intelligence

Road processing is enabled only when VALHALLA_URL is configured. It reads good route positions joined to their source observations. The processor takes at most three vehicles per run, requests at most 12 points per vehicle, and waits for at least four points.

An accepted match creates a road_match_run and eligible road_segment_samples. The sample's uncertainty is the source interval duration. Road analytics groups current samples by road segment and takes each vehicle's median pace before aggregation. A slowdown needs three distinct recent vehicles in the last 30 minutes, at least 30 historical samples, and a current p50 pace at least 1.3 times the historical p50. Five recent vehicles and at least 60 historical samples produce high road confidence; other qualifying results are moderate.

The road cursor is part of the processing contract. A successful match persists the run, samples, and cursor together. A transient matcher failure leaves the cursor unchanged and requests a retry after 60 seconds. An unacceptable match advances past the rejected prefix without writing samples. A route or direction discontinuity is therefore not retried as if it were a temporary network error.

Failure and null behavior

The collector keeps successful location results when another vehicle request fails. If the schedule provider fails, it preserves the last good public snapshot instead of replacing it with a new snapshot. Transit intelligence catches failures per observation and continues through the batch. The road processor's failure or absence does not stop raw collection, route projection, stop visits, or transit ETA history.

The public snapshot becomes stale after 60 seconds, while data health classifies individual reports as fresh through 40 seconds, stale through 90 seconds, and expired after 90 seconds. These are operational signals, not replacements for the ETA thresholds above.

Acceptance examples

The current tests provide concrete contract cases:

• Repeating the same vehicle/report-time packet inserts it once and counts the second attempt as a duplicate. Reprocessing the same observations creates no additional route positions or transit events.
• Three ordered R603 PM observations create three route positions, one started trip, two stop passages, and one segment run.
• A progress jump above 100 km/h is stored as gps_jump with no passage or segment run.
• Two stationary reports at 00:00:00 and 00:00:20 create one stop visit with 20 seconds of observed dwell, 40 seconds of uncertainty, and sample count 2.
• Four road points with nearest-stop distances [200, 50, 200, 200] produce one stored road sample and two skipped intervals after a successful match.

These examples come from the collector tests and define behavior for the current implementation. They do not establish live-feed coverage or the accuracy of the upstream coordinates.

Source anchors: schemas.ts, network.ts, intelligence.ts, eta.ts, road-intelligence.ts, and the collector tests.