Sync Engine v2 — Build Specification

Master spec for the multi-agent build. Every agent MUST read this fully before writing code, then explore its exemplar files for conventions. When this spec and repo conventions conflict on style, follow the repo; on architecture, follow this spec.

Mission

A production-grade Strava→leaderboard sync engine living as a bounded module inside admin-backend (Laravel 10, /var/www/togoactive-development/admin-backend) and admin-frontend (React+Vite+Tailwind, /var/www/togoactive-development/admin-frontend). It replaces the legacy old-prod "Stage 1/2/3" crons for future events only (event id > 50). Today only event 52 qualifies. Events ≤ 50 remain on legacy forever; the legacy codebase at /var/www/togoactive is read-only reference — never modify it.

Hard safety rules (all agents)

1. NEVER run php artisan as root in admin-backend (root-owned cache shards break php-fpm days later). For smoke checks use sudo -u www-data php artisan <cmd>, and only read-only commands (route:list, custom --dry-run). Never run cache/config/optimize commands.
2. Never write to the TGP database (mysql_tgp connection). Reads are allowed. The TGP projector class exists but must be hard-disabled (shadow_mode + config flag both default safe).
3. Only the new sync_* tables on the default mysql connection are writable. They already exist on the live DB (created ahead of the build) — migrations must be Schema::hasTable-guarded so php artisan migrate is a no-op when tables exist.
4. Do not modify /var/www/togoactive (old prod). Do not run npm dev servers. Do not commit — the orchestrator commits at the end.
5. admin-backend is served live by php-fpm — code you write is live on save. New files/routes are inert until wired, which is why routes and Kernel entries land late in the build order.
6. Timezone everywhere: Asia/Singapore (both apps + old prod agree).

Pipeline (cron-driven batch, NO Laravel queue daemon)

QUEUE_CONNECTION=sync stays untouched. The pipeline is scheduler-driven batch commands; the "queue" is rows in sync_activities by status. State machine:

received → fetched → accepted → counted
                   → rejected(reason)        [terminal unless re-evaluated]
                   → flagged  → (admin approve → accepted → counted)
                              → (admin reject → rejected)

Commands (signature → cadence):

• sync:tick — every minute, withoutOverlapping. Orchestrates serially: Intake → Fetch → Evaluate → Tally → Rank. Each step records a sync_runs row. Exits immediately (cheap query) when no event has engine='v2'. Supports --dry-run (log what would happen, write nothing) and --event= (restrict).
• sync:gallery — every 15 min. Photos for counted activities with photos_count > 0 only, per-event toggle enable_gallery.
• sync:alerts — every minute. Evaluates the alert catalog, upserts sync_alerts by fingerprint, auto-resolves cleared conditions.
• sync:reconcile — daily 03:05 + on-demand; per-event toggle enable_reconcile. Compares our counted set vs Strava athlete-activity listing (via proxy) and vs challenge_leaderboard (parity).
• sync:prune — daily 03:40. Retention: sync_runs 14d, sync_activity_log 90d, sync_quota_samples 7d, resolved sync_alerts 30d, replayed dead letters 30d.
• sync:replay {deadLetterId} — manual/API-triggered replay.

Step semantics

Intake — tail challenge_activities_log (TGP DB, read-only; verify PK/columns with DESCRIBE before coding). Own cursor in sync_engine_state (key intake_cursor), never touch tga_sync (legacy owns that column). Map: log.strava_id → TGP users.userid → TGA users.tgp_userid → event_users rows → events with engine='v2' and inside sync window (events_dates registration_start → leaderboard_end; cron_force_sync-equivalent flag in rules bypasses the floor). Insert one sync_activities row per (event, activity) via insertOrIgnore (unique key dedups).

Fetch — for received rows (oldest first, batch ≤ budget): POST to the legacy proxy https://strava.togoparts.com/strava-get-activity.php with the same payload legacy Stage2 uses (access_token, strava_id, activity_id, refresh_token, token_expiry, uid:0, userid) using tokens from TGP users row. Response handling mirrors legacy: body string containing 'Rate Limit Exceeded' → stop batch, mark quota throttled; '401 Unauthorized' → dead-letter + set status rejected reason token_dead (do NOT modify TGP users.strava_error — legacy owns it); other non-success → attempts+1, ≥3 → dead letter + rejected reason fetch_failed. On success: store full raw JSON in raw + promote columns (see schema). Use Strava's own start_date_local and timezone fields — never recompute local time from UTC. Every call increments the quota counter.

Quota budgeter — self-accounting (the proxy doesn't forward Strava's rate-limit headers). Rolling 15-min window counter in sync_engine_state + samples in sync_quota_samples. Config sync_engine.quota_per_window default 120 (deliberately far below the shared 600/15min app limit while legacy also runs — NEVER raise the ceiling without human sign-off). Fetch/gallery/reconcile all draw from the same budget; live fetch has priority.

Evaluate — pure local, no API. Order: window check → type allow-list → dedup → suspicious scoring. Writes rule_version used. Reasons are stable snake_case codes (outside_window, type_not_allowed, duplicate_overlap, duplicate_external_id, too_long, manual_not_allowed, …). Dedup checks (same event): overlapping elapsed-time window for the same user; identical external_id for the same user; cross-user identical (start_date_utc ±60s AND distance ±1%) marks the LATER row flagged possible_shared_ride. Suspicious: per-sport thresholds from rules JSON; each hit +1 (strava_flagged +2, HR-missing-above-speed +1, manual-without-device +1, moving/elapsed ratio, min elapsed). score ≥ flag_threshold → flagged (review queue), unless rules.suspicious.auto_exclude → rejected reason suspicious_auto. Passing rows → accepted. Every transition appends sync_activity_log.

Tally — accepted → add to sync_totals (and sync_team_totals via event_teams/ event_team_users membership — explore TGA schema; if team mapping is unclear, tally individuals and leave a TODO service seam) then mark counted + dirty=1 on the totals row. Distances in KM (TGP convention — Strava gives metres; divide by 1000 exactly once, here).

Rank — for events with dirty rows: recompute rank in sync_totals (window function ROW_NUMBER() OVER (ORDER BY total_distance DESC) — MySQL 8), clear dirty. Debounce: skip if ranked < 60s ago.

Reconcile/parity — compare per-user totals vs TGP challenge_leaderboard (cid from settings) → store diff summary in sync_engine_state key parity:{eventId} (JSON: generatedAt, matched, missing_in_v2, missing_in_tgp, mismatched list capped at 50). Exposed via API.

Schema (already created on live DB — write guarded migrations to match EXACTLY)

Tables (all mysql connection): sync_engine_state (k PK, v, updated_at), sync_event_settings, sync_activities, sync_activity_log, sync_totals, sync_team_totals, sync_runs, sync_quota_samples, sync_alerts, sync_dead_letters, sync_settings_audit. Run SHOW CREATE TABLE (read-only, default connection) to mirror them precisely in migrations and models. Key columns of sync_event_settings: event_id PK, engine('off'|'v2') default 'off', shadow_mode default 1, enable_indoor/enable_team/enable_gallery/enable_reconcile, rules JSON, rules_version, updated_by, timestamps.

Rules JSON default (seeded when a settings row is first created)

{
  "allowed_types": ["Ride","Run","Walk","Hike","TrailRun","VirtualRide","VirtualRun"],
  "indoor_types": ["VirtualRide","VirtualRun"],
  "force_sync_before_start": false,
  "max_distance_km": 500,
  "manual_policy": "indoor_only",
  "suspicious": {
    "ride": {"max_avg_speed_kmh": 60, "max_max_speed_kmh": 95, "max_distance_km": 300},
    "run":  {"max_avg_speed_kmh": 20, "max_max_speed_kmh": 30, "max_distance_km": 100},
    "min_elapsed_s": 60, "moving_ratio_min": 0.5,
    "hr_required_above_kmh": 30, "flag_threshold": 2, "auto_exclude": false
  },
  "dedup": {"overlap": true, "external_id": true, "cross_user": true}
}

Alert catalog (rule_key → severity → condition → auto-resolve)

• tick_missing crit — no successful sync:tick run in 5 min (only when any event engine=v2)
• pipeline_stalled crit — received-status rows > 0 older than 10 min
• reconcile_drift crit — parity mismatch beyond 1% of users for a non-shadow event
• fetch_failure_rate warn — >20% fetch failures in last hour (min 5 attempts)
• quota_throttled warn — budgeter throttled within last window
• dead_letter_new warn — dead letters created in last 10 min
• token_death_spike warn — >5 distinct token_dead rejections in 24h
• intake_silence warn — engine event live > 24h with zero intake in 12h
• config_change info — settings changed (from audit) Fingerprint = rule_key:{event_id|global}. Active alert with same fingerprint → update last_seen. Condition cleared → state resolved + resolved_at.

API contract (all under existing Route::middleware(['auth:admin']) group, prefix v1)

Global — SyncOpsController:

• GET /sync-ops/overview → { success, data: { events: [{eventId,name,engine,shadowMode, queue:{received,flagged,deadLetters}, countedToday, lastTickAt, health}], pipeline: {steps: [{step,lastRunAt,lastOutcome,avgMs24h,items24h,failures24h}]}, quota: {callsMade,budget, throttled,windowStart}, alerts: {critical,warning} } }
• GET /sync-ops/alerts?state=&severity= · POST /sync-ops/alerts/{id}/ack {note} · POST /sync-ops/alerts/{id}/resolve
• GET /sync-ops/runs?step=&hours=24 · GET /sync-ops/quota?hours=24
• GET /sync-ops/dead-letters · POST /sync-ops/dead-letters/{id}/replay
• GET /sync-ops/audit · GET /sync-ops/tokens (dead-token users among v2-event participants; join TGA users/event_users with TGP users.strava_error=111 — cid-scoped, cached 10 min)

Per-event — EventSyncEngineController (404-guard eventId <= 50 with message "Sync Engine v2 is only available for events after id 50"):

• GET /events/{eventId}/sync-engine → settings (creates default row with engine='off' on first read)
• PUT /events/{eventId}/sync-engine → save; diff → sync_settings_audit; rules changed → rules_version+1
• GET /events/{eventId}/sync-engine/status → per-event pipeline stats + parity summary
• GET /events/{eventId}/sync-engine/activities?status=&search=&page=&perPage= (paginated)
• GET /events/{eventId}/sync-engine/activities/{id} → full detail incl. lifecycle log + raw highlights
• POST /events/{eventId}/sync-engine/activities/{id}/review {action: approve|reject, note}
• GET /events/{eventId}/sync-engine/parity
• POST /events/{eventId}/sync-engine/backfill {dryRun: true|false, sinceDays} (re-scan intake from log history for this event's participants; respects budget; dryRun returns counts only)

Response envelope: { success: bool, data: ..., message? } — mirror ActivityController. Errors: per-section try/catch fault tolerance in overview (an unavailable section returns null + errors map — same philosophy as the Analytics dashboard).

Frontend (admin-frontend)

Conventions: Tailwind utility classes matching existing pages (bg-white rounded-xl border border-gray-200, text-xs/sm, primary-500 accents), lucide-react icons, api util from src/utils/api.js (FormData PUT = POST + _method=PUT), swal util for confirms (never native alert/confirm), no backticks inside dangerouslySetInnerHTML CSS. Exemplars: src/pages/ActivitiesPage.jsx, src/components/activities/* (board/table/chart patterns), src/components/event-manage/leaderboard/LeaderboardSync.jsx (per-event settings page pattern: SectionCard/ToggleCard/SaveBar, useState+getInitialForm, isDirty JSON compare, useOutletContext).

Pages:

1. Global /sync-ops (lazy route, sidebar item "Sync Ops"): tab layout — Overview (status wall cards: green/amber/red per event + pipeline step tiles + quota gauge), Alerts (inbox: filter chips by state/severity, ack/resolve with note, severity-colored rows), Pipeline (runs table + duration sparkline per step), Dead Letters (table + replay button), Tokens (dead-token list), Audit (settings change log). Poll overview every 60s.
2. Header alert strip: global red banner across admin when any critical alert is active (poll 60s, dismiss = navigate to /sync-ops alerts tab). Bell count for warnings.
3. Per-event leaderboard/sync-engine (nav item visible only when event id > 50): sections — Engine (on/off + shadow badge explaining shadow mode), Jobs (indoor/team/gallery/ reconcile toggles), Rules editor (typed inputs incl. per-sport suspicious thresholds; save bumps version; show current version), Review Queue (flagged activities table: evidence columns speed/ HR/device/score/reasons, approve/reject buttons, batch select), Live Status (queue counts, last runs), Parity card (matched/missing/mismatched + refresh), Backfill card (dry-run preview then execute, confirm via swal).

File ownership (agents MUST stay inside their set)

• backend-core: admin-backend/app/SyncEngine/** (Models/, Services/, Support/), admin-backend/config/sync_engine.php, admin-backend/database/migrations/*_sync_engine_*.php
• backend-pipeline: admin-backend/app/Console/Commands/Sync*.php, admin-backend/app/Console/Kernel.php, admin-backend/app/SyncEngine/Pipeline/**
• backend-api: admin-backend/app/Http/Controllers/Api/V1/SyncOpsController.php, .../EventSyncEngineController.php, admin-backend/app/Http/Requests/SyncEngine*.php, admin-backend/routes/api.php (append inside auth:admin group only)
• frontend-ops: admin-frontend/src/pages/SyncOpsPage.jsx, admin-frontend/src/components/sync-ops/**, admin-frontend/src/utils/syncOpsFormat.js
• frontend-event: admin-frontend/src/components/event-manage/leaderboard/SyncEngine*.jsx (new files only — do NOT touch router/nav)
• integration (single agent, runs last): admin-frontend/src/router.jsx, Sidebar.jsx, Header.jsx (alert strip), event-manage nav registration (find where leaderboard/sync nav is declared and add sync-engine gated to id > 50; also SettingsSearchPalette if events are indexed there), npm run build, www-data crontab install, smoke tests.

Integration & ops wiring (integration agent)

1. Wire routes/nav; npm run build in admin-frontend must pass (fix imports if broken).
2. php -l every new PHP file; sudo -u www-data php artisan route:list | grep sync must show routes.
3. sudo -u www-data php artisan sync:tick --dry-run must exit 0 quickly (no v2 events enabled).
4. Install www-data crontab (preserve existing entries): * * * * * cd /var/www/togoactive-development/admin-backend && php artisan schedule:run >> storage/logs/scheduler.log 2>&1
5. Kernel: schedule the five sync commands (tick/alerts every minute withoutOverlapping, gallery */15, reconcile 03:05, prune 03:40) — everything no-ops while all engines are 'off'.
6. Ensure storage/logs writability stays www-data; do NOT chown anything to root.