Skip to content

Bug: gmail watch serve delivers empty messages due to Gmail API history race condition #379

@okhan1980

Description

@okhan1980

Environment

  • gogcli v0.11.0
  • macOS 26 (arm64)
  • Gmail API via OAuth
  • Google Pub/Sub push delivery via Tailscale Funnel

Summary

When using gog gmail watch serve to receive Gmail Pub/Sub push notifications, the handler consistently returns empty messages: [] arrays. The root cause is a race condition between Google's Pub/Sub notification timing and Gmail's INBOX indexing — messages are not yet queryable via the History API when the push arrives.

Steps to Reproduce

  1. Set up Gmail watch via gog gmail watch start --topic <topic> --label INBOX
  2. Configure Pub/Sub push subscription pointing to gog gmail watch serve endpoint
  3. Send a test email to the watched account
  4. Observe push arrives at the handler within 1-3 seconds
  5. gog gmail watch serve immediately calls Gmail History API with the pushed historyId
  6. History API returns empty results because Gmail hasn't finished indexing the message

Expected Behavior

The serve handler should include a configurable delay (e.g. --fetch-delay 5) before querying the History API, allowing Gmail to finish indexing. Alternatively, implement a retry with exponential backoff when History returns empty results for a new historyId that is greater than the stored one.

Actual Behavior

gog gmail watch serve calls the History API immediately upon receiving the Pub/Sub push and forwards messages: [] to the configured hook endpoint. No retry is attempted.

Workaround

Built a custom Python HTTP handler that:

  1. Receives Pub/Sub pushes directly (bypassing gog gmail watch serve)
  2. Waits 5 seconds for Gmail indexing
  3. Calls gog gmail history --since <historyId> --json to get message IDs
  4. Calls gog gmail get <id> --json for each message
  5. Forwards notifications downstream

This works reliably. The 5-second delay resolves the race condition in all observed cases.

Suggested Fix

Add a --fetch-delay <seconds> flag to gog gmail watch serve (default: 3-5s). This is a well-known Gmail API behavior — the Pub/Sub notification arrives before the message is fully indexed and queryable via History API.

Alternatively, implement retry logic: if History returns empty for a historyId greater than the stored one, retry 2-3 times with 2-3 second delays before giving up.

Additional Context

This affects anyone using gog gmail watch serve for real-time email notifications. The setup works perfectly — GCP topic, subscription, watch registration — but the runtime fails silently at the last mile. Users will see pushes arriving but zero messages forwarded.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions