feat: telemetry and security hardening #49
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…on guide This commit introduces a detailed PostHog instrumentation blueprint for the OpenChat monorepo, covering web, server, and extension components. It outlines baseline setup, event taxonomy with triggers and properties, common properties, dashboards and insights strategies, implementation notes by file, and next steps for full analytics coverage. Key additions include event naming conventions, super-property registrations, enhanced event properties for auth, chat, settings, sync, and extension features, as well as recommended dashboard KPIs and monitoring plans to track acquisition, onboarding, chat health, personalization, and extension engagement. This documentation will guide engineering alignment and implementation to enable actionable analytics through PostHog. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
…trumentation - Reduced and optimized core event sets to focus on high-leverage insights, minimizing low-signal noise. - Standardized event naming conventions to lowercase snake_case with scope.action structure. - Updated super-properties registration for better session identification including guest handling. - Enhanced event properties for chat creation, streaming, attachments, and OpenRouter API key lifecycle. - Added new telemetry events for sync connection state, fallback storage usage, and chat rate limiting. - Revised dashboard and funnel definitions to reflect streamlined event model and metrics. - Improved implementation notes for consistency and expanded coverage across web, server, and extension components. This update enables effective monitoring of acquisition, chat usage, reliability, and onboarding activation without excess event volume. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
- Add base super properties and sanitize event properties for client and server PostHog usage - Track fallback storage usage in server router for chats and messages - Add detailed client event tracking in web chat handler and components - Introduce PosthogBootstrap component for consistent client identification and property registration - Enhance event tracking with user workspace IDs, auth states, theme info, and OpenRouter API key status - Track marketing CTA clicks and landing visits with detailed metadata - Improve rate limit and error event captures for chat and OpenRouter APIs - Add client property registration on model selection and OpenRouter key management - Add sync connection state events to PostHog This update improves analytics granularity and consistency across client and server, enabling better user behavior insights and troubleshooting. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
…og-features-qkjs37
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Greptile Overview
Greptile Summary
This PR implements comprehensive PostHog telemetry (v2) across the OpenChat web application and server. The changes add structured event tracking for user behavior, system reliability, and product analytics across14 core event types including marketing visits, chat operations, API key management, and sync connection states. A new documentation file (posthog.md) provides the strategic framework, while instrumentation is added throughout key components like the chat composer, model selector, sidebar, and API handlers. The implementation includes proper data sanitization, environment-aware configuration, user identification for both authenticated and guest users, workspace grouping, and privacy-compliant IP hashing. The telemetry captures both positive and negative user journeys (successful operations and failures/rate limits) with consistent metadata structure.
Important Files Changed
Changed Files
| Filename | Score | Overview |
|---|---|---|
posthog.md |
5/5 | New comprehensive telemetry documentation defining 14 core events and implementation strategy |
apps/web/src/components/model-selector.tsx |
4/5 | Added model selection tracking with PostHog client properties registration |
apps/web/src/components/account-settings-modal.tsx |
4/5 | Instrumented OpenRouter key save/remove operations with masked key tracking |
apps/web/src/components/providers.tsx |
4/5 | Enhanced pageview tracking with referrer analysis and PosthogBootstrap integration |
apps/web/src/components/openrouter-link-modal.tsx |
4/5 | Added key prompt tracking with reason detection and duplicate event prevention |
apps/web/src/lib/posthog.ts |
4/5 | Enhanced client telemetry with property sanitization and workspace grouping |
apps/server/src/lib/posthog.ts |
5/5 | Implemented server-side PostHog client with environment-aware configuration |
apps/web/src/components/app-sidebar.tsx |
4/5 | Added comprehensive user identification and dashboard entry tracking |
apps/web/src/lib/sync.ts |
4/5 | Instrumented WebSocket connection states for reliability monitoring |
apps/web/src/components/posthog-bootstrap.tsx |
4/5 | New component for centralized PostHog initialization and user property management |
apps/web/src/lib/posthog-server.ts |
4/5 | Server-side PostHog configuration with multi-platform environment support |
apps/web/src/components/chat-room.tsx |
4/5 | Added extensive chat operation tracking including error handling and rate limits |
apps/web/src/app/api/chat/chat-handler.ts |
4/5 | Comprehensive API telemetry with privacy-compliant IP hashing and rate limit tracking |
apps/server/src/routers/index.ts |
4/5 | Server router instrumentation with fallback storage monitoring and chat events |
apps/web/src/components/chat-composer.tsx |
4/5 | File attachment event tracking for both accepted and rejected attachments |
apps/web/src/components/hero-section.tsx |
4/5 | Marketing analytics with CTA tracking and landing page visit instrumentation |
Confidence score: 4/5
- This PR is safe to merge with careful review of the client-side rendering change in hero-section.tsx
- Score reflects comprehensive telemetry implementation with good privacy practices, but deducted one point due to the hero section conversion from SSR to CSR which may impact SEO performance on the marketing landing page
- Pay close attention to
apps/web/src/components/hero-section.tsxfor potential SEO implications from the server-to-client rendering change
Sequence Diagram
sequenceDiagram
participant User
participant Web as Web App
participant PostHog as PostHog Client
participant Server as Server API
participant ServerPH as Server PostHog
participant OpenRouter as OpenRouter API
User->>Web: Visit landing page
Web->>PostHog: captureClientEvent("marketing.visit_landing")
User->>Web: Click "Try OpenChat" CTA
Web->>PostHog: captureClientEvent("marketing.cta_clicked")
User->>Web: Access dashboard
Web->>PostHog: captureClientEvent("dashboard.entered")
User->>Web: Click "New Chat"
Web->>Server: client.chats.create()
Server->>ServerPH: capturePosthogEvent("chat.created")
Server-->>Web: Return chat ID
Web->>PostHog: captureClientEvent("chat.created")
User->>Web: Open OpenRouter key modal
Web->>PostHog: captureClientEvent("openrouter.key_prompt_shown")
User->>Web: Save API key
Web->>PostHog: captureClientEvent("openrouter.key_saved")
User->>Web: Attach file
Web->>PostHog: captureClientEvent("chat.attachment_event")
User->>Web: Send message
Web->>PostHog: captureClientEvent("chat_message_submitted")
Web->>Server: POST /api/chat
Server->>ServerPH: captureServerEvent("chat_message_stream")
Server->>OpenRouter: Stream text request
OpenRouter-->>Server: Streaming response
Server->>Server: persistMessageImpl()
Server->>ServerPH: captureServerEvent("chat_message_stream", completion)
Server-->>Web: Stream response
Note over Web, ServerPH: If rate limited
Server->>ServerPH: captureServerEvent("chat.rate_limited")
Note over Web, ServerPH: If DB fallback used
Server->>ServerPH: capturePosthogEvent("workspace.fallback_storage_used")
Note over Web, PostHog: If WebSocket connection issues
Web->>PostHog: captureClientEvent("sync.connection_state")
Context used:
- Context from
dashboard- AGENTS.md (source)
16 files reviewed, 6 comments
|
|
||
| useEffect(() => { | ||
| if (!resolvedWorkspaceId) return; | ||
| if (identifyRef.current === resolvedWorkspaceId && session?.user) return; |
Contributor
There was a problem hiding this comment.
logic: The condition identifyRef.current === resolvedWorkspaceId && session?.user may not prevent re-identification when transitioning from guest to authenticated user with the same workspace ID
| process.env.POSTHOG_DEPLOYMENT_REGION ?? process.env.VERCEL_REGION ?? "local"; | ||
|
|
||
| const BASE_SUPER_PROPERTIES = Object.freeze({ | ||
| app: "openchat-server", |
Contributor
There was a problem hiding this comment.
style: App name 'openchat-server' seems inconsistent - this is the web app server-side code, not the actual server app
Comment on lines
+38
to
+41
| identifyClient(identifier, { | ||
| workspaceId: workspaceId ?? identifier, | ||
| properties: { auth_state: session?.user ? "member" : "guest" }, | ||
| }); |
Contributor
There was a problem hiding this comment.
style: Ensure PostHog identify calls don't create performance bottlenecks with frequent re-identification of the same user
| registerClientProperties({ has_openrouter_key: true }); | ||
| captureClientEvent("openrouter.key_saved", { | ||
| source: "modal", | ||
| masked_tail: key.slice(-4), |
Contributor
There was a problem hiding this comment.
style: Masking only the last 4 characters may not be sufficient for API key security - consider masking more or using a hash
Comment on lines
+198
to
205
| captureClientEvent("chat.attachment_event", { | ||
| chat_id: chatId, | ||
| result: "rejected", | ||
| file_mime: file.type || "application/octet-stream", | ||
| file_size_bytes: file.size, | ||
| limit_bytes: MAX_ATTACHMENT_SIZE_BYTES, | ||
| }); | ||
| continue; |
Contributor
There was a problem hiding this comment.
style: Consider moving the telemetry call outside the file processing loop to avoid redundant calls and improve performance when multiple files are rejected
Comment on lines
+226
to
+234
| for (const file of added) { | ||
| captureClientEvent("chat.attachment_event", { | ||
| chat_id: chatId, | ||
| result: "accepted", | ||
| file_mime: file.type || "application/octet-stream", | ||
| file_size_bytes: file.size, | ||
| limit_bytes: MAX_ATTACHMENT_SIZE_BYTES, | ||
| }); | ||
| } |
Contributor
There was a problem hiding this comment.
style: The telemetry loop could be optimized by batching events or capturing a single event with file count instead of individual events per file
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Testing