Skip to content

Add multi-proxy support & granular proxy selection#160

Open
CorentinB wants to merge 2 commits intomasterfrom
feat/more-proxy-stuff
Open

Add multi-proxy support & granular proxy selection#160
CorentinB wants to merge 2 commits intomasterfrom
feat/more-proxy-stuff

Conversation

@CorentinB
Copy link
Collaborator

Implements a granular multi-proxy system with fine-grained control over proxy selection.

Features

  • Multi-proxy support: Configure multiple proxies with automatic round-robin selection
  • Network filtering: Filter proxies by IPv4/IPv6 support via ProxyNetwork enum
  • Type classification: Categorize proxies as Mobile/Residential/Datacenter via ProxyType enum
  • Domain routing: Route specific domains to specific proxies using glob patterns
  • Per-proxy statistics: Track RequestCount, ErrorCount, and LastUsed per proxy
  • Context-based selection: External callers can request specific proxy types via context

Example: Context-based proxy type selection

// Configure proxies with different types
httpClient, _ := warc.NewWARCWritingHTTPClient(warc.HTTPClientSettings{
    Proxies: []warc.ProxyConfig{
        {
            URL:     "socks5://mobile-proxy:1080",
            Network: warc.ProxyNetworkAny,
            Type:    warc.ProxyTypeMobile,
        },
        {
            URL:     "socks5://datacenter-proxy:1080",
            Network: warc.ProxyNetworkAny,
            Type:    warc.ProxyTypeAny,
        },
    },
})

// Request specific proxy type via context
ctx := warc.WithProxyType(req.Context(), warc.ProxyTypeMobile)
req = req.WithContext(ctx)
resp, _ := httpClient.Do(req)

Breaking change

Replaces Proxy string field with Proxies []ProxyConfig in HTTPClientSettings.

- Support multiple proxies with round-robin selection
- Add ProxyNetwork enum (IPv4/IPv6 filtering)
- Add ProxyType enum (Mobile/Residential/Datacenter)
- Add per-domain routing with glob patterns
- Add per-proxy statistics (RequestCount, ErrorCount, LastUsed)
- Add context-based proxy type selection
- Breaking change: replace Proxy string with Proxies []ProxyConfig
@CorentinB CorentinB added the enhancement New feature or request label Nov 20, 2025
@CorentinB CorentinB self-assigned this Nov 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants