fix(downloader): keep hedge alive through body download

[shikhar]

Summary

• race full download attempts so hedges stay alive through body streaming
• keep latency/metrics behavior unchanged

Testing

• cargo +nightly fmt
• cargo nextest run

Issue

Closes #52

[greptile-apps]

Greptile Overview

Greptile Summary

This PR changes the hedged download flow so the “primary attempt” future stays alive until the full response body is streamed (rather than completing early once headers are received), by introducing attempt_full() which wraps attempt_inner() + handle_result() and racing those full attempts.

The overall behavior is still a primary request with an optional hedge after a per-bucket threshold; metrics/latency accounting is performed in handle_result() after the body is collected.

One correctness concern: the hedge request is still issued against the same bucket as the primary attempt. If the intent of hedging is to reduce tail latency by racing a different bucket/replica, this change (and the existing code) won’t provide that benefit.

Confidence Score: 3/5

• Moderately safe to merge, but verify hedging semantics around bucket selection.
• The change is a localized refactor to ensure the hedged race covers full body streaming, which aligns with the PR intent. The main risk is behavioral: hedged attempts are not routed to a different bucket/endpoint, so if the system expects cross-bucket hedging, this won’t improve tail latency and could add redundant load to the same bucket.
• src/object_store/downloader.rs

Important Files Changed

Filename	Overview
src/object_store/downloader.rs	Refactors hedged download to keep primary attempt alive through body streaming by racing full attempts; hedge still targets the same bucket as primary, which may defeat cross-bucket hedging if that was intended.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant Downloader
    participant Stats as BucketedStats
    participant S3 as aws_sdk_s3::Client

    Caller->>Downloader: download(buckets, object, range, req_config)
    Downloader->>Stats: attempt_order(buckets)
    Downloader->>Downloader: attempt(primary_bucket, object, range, req_config)

    alt Primary finishes before hedge trigger
        Downloader->>S3: get_object(range)
        S3-->>Downloader: GetObjectOutput
        Downloader->>S3: stream body (collect)
        Downloader->>Stats: observe(bucket, Ok(latency))
        Downloader-->>Caller: ObjectPiece
    else Hedge trigger fires
        Downloader->>Stats: hedging_threshold(bucket, start_time)
        Downloader->>Downloader: attempt_full(primary) (kept alive)
        Downloader->>Downloader: attempt_full(hedge)
        par race full attempts (incl. body streaming)
            Downloader->>S3: get_object(range) [primary]
            S3-->>Downloader: GetObjectOutput
            Downloader->>S3: stream body (collect)
        and
            Downloader->>S3: get_object(range) [hedge]
            S3-->>Downloader: GetObjectOutput
            Downloader->>S3: stream body (collect)
        end
        Downloader->>Stats: observe(bucket, Ok(latency) or Err)
        Downloader-->>Caller: ObjectPiece or DownloadError
    end

Loading

[greptile-apps]

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}