PDP tracker: add scalable event+hydration ingestion path (cursor-based) #615
Description
Problem
Current PDP tracker behavior performs periodic scan-based reads from RPC. This is acceptable for PoC, but does not scale with proof set growth and RPC latency.
Goal
Define and implement an ingestion API that allows replacing full scans with an indexer-style path while preserving tracker external behavior.
Proposed direction
- Event-driven ingestion from contract logs (cursor-based, resumable)
- Multicall hydration for state fields needed by deal updates
- Durable cursor/checkpoint in DB for restart-safe progress
- Keep current tracker deal upsert/state transitions as consumer behavior
Scope (phase 1)
- Define internal interfaces for:
- event source (by range/cursor)
- hydration batcher
- cursor store
- Add cursor schema/table and migration
- Implement event+hydrate pipeline behind feature flag/config toggle
- Add metrics/logging for lag, processed events, hydration batch sizes, and retry/backoff
Non-goals
- Building a full external indexer service in this phase
- Perfect historical backfill optimizations
Acceptance criteria
- Tracker can run in event-driven mode without full periodic scans
- Restart resumes from persisted cursor
- Bounded RPC calls per cycle and observable lag metrics
- Existing deal state update behavior remains unchanged
Context
Follow-up from PR #613 discussion (PDP config/ops layer), with agreement that scan loop is PoC and not scalable long-term.