Open
Conversation
* feat: implement native Go x-client-transaction-id generation This PR implements native Go generation of the x-client-transaction-id header required by Twitter/X's updated API security. ## Changes ### New Files - x_transaction.go: Core transaction ID generation logic ported from Python - x_cubic.go: Cubic Bézier interpolation for animation calculations - x_utils.go: Helper functions (floatToHex, interpolate, rotation matrix) ### Modified Files - api.go: Integrate transaction ID generation into authenticated requests - auth.go: Update bearer tokens for new API endpoints - auth_test.go: Support loading cookies from cookies.json file - scraper.go: Add transaction context caching struct - search.go: Update GraphQL endpoint and feature flags for x.com - account.go: Update API endpoints to api.x.com - util.go: Update base URL to x.com - search_test.go: Reduce test tweet counts (150 -> 10) to avoid rate limits - tweets_test.go: Reduce test tweet counts (150 -> 10) to avoid rate limits - .gitignore: Add cookies.json to prevent accidental commits ## Technical Details The x-client-transaction-id header is generated by: 1. Fetching x.com homepage to extract verification key and SVG animation data 2. Parsing ondemand.s.*.js to get key byte indices 3. Computing animation key via cubic Bézier interpolation 4. Hashing with SHA256 using method, path, timestamp, and animation key 5. XOR encoding with random byte and base64 encoding The transaction context (verification key + animation key) is cached for 10 minutes to avoid fetching the homepage on every request. The actual transaction ID is unique per request due to timestamp and random components. ## Testing - All search tests pass with native Go implementation - No Python dependency required * refactor: remove unused features and simplify tests ## Removed Features (per issue requirements) - Bookmarks (bookmarks.go) - Followers/Following (follows.go) - Spaces (spaces.go) - Scheduled tweets (schedule.go) - Media timeline (medias.go) - Media upload (upload.go) - Home/ForYou timeline (from tweets.go) - Tweet creation/deletion/like/retweet (from tweet.go) - Login/LoginOpenAccount methods (from auth.go) ## Authentication Changes - IsLoggedIn() now only checks for auth_token cookie presence - No longer hits Twitter endpoints to verify login status - Removed username/password login support - Cookie-based authentication only ## Endpoint Updates All endpoints updated to x.com/api.x.com: - profile.go: UserByScreenName, UserByRestId - trends.go: guide.json - replies.go: TweetDetail - tweets.go: All timeline endpoints - api.go: guest/activate ## Test Simplification - Simplified all tests to verify functionality only - Removed tests that relied on specific tweet content - Removed tests for deleted features - All tests now pass reliably * chore: final audit and cleanup ## Code Cleanup - Removed unused Scraper fields: isOpenAccount, oAuthToken, oAuthSecret, includeReplies - Removed OAuth signing code (no longer needed for cookie-only auth) - Removed FetchTweetsByUserIDLegacy function (only used with OpenAccount) - Removed commented-out code - Simplified GetTweet to remove isOpenAccount branch ## Updates - Updated User-Agent to Chrome 131 (December 2024) - Added missing feature flags (rweb_lists_timeline_redesign_enabled, tweetypie_unmention_optimization_enabled) - Standardized feature flags across all endpoints - Simplified auth.go (removed unused logout API call, cleaner cookie handling) ## Best Practices - All endpoints use api.x.com - All feature flags are current as of December 2024 - Bearer token is consistent across all requests - Transaction ID caching implemented (10 minute TTL) All 19 tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.