-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Labels
Description
Issue Description
New HTTP clients are created for each LLM request instead of reusing connections, causing significant overhead.
Location
crates/terraphim_service/src/llm.rs (lines 285, 540, 569)
Current Code
```rust
// Creating new client for each request
let client = reqwest::Client::new();
// ... use client once ...
```
Impact
- CRITICAL priority - Performance
- Creates new connection pool for every client initialization
- 30-50% connection overhead for repeated requests
- Increased memory usage
Recommended Fix
```rust
use once_cell::sync::Lazy;
use reqwest::Client;
static HTTP_CLIENT: Lazy = Lazy::new(|| {
Client::builder()
.pool_max_idle_per_host(10)
.pool_idle_timeout(Duration::from_secs(90))
.timeout(Duration::from_secs(120))
.build()
.unwrap()
});
// Use: &*HTTP_CLIENT instead of creating new clients
```
Expected Improvement
- 30-50% reduction in connection overhead
- Reduced memory allocations
- Faster repeated requests
References
- Identified in performance analysis for PR fix(zipsign): update zipsign API to v0.2 #429
- Related to LLM client architecture