Skip to content

Conversation

@flavorjones
Copy link
Contributor

Return a more-complete 304 from TransformingTransport.RoundTrip to address a segfault we're seeing in production. The necessary conditions for the issue are:

  1. A remote service returning a 303 response which must contain a location URI that changes every time, so that the real location of the image is never cached. As an example, if the remove service redirects to S3, the presence of the X-Amz-Security-Token accomplishes this.

  2. The image responses must match by Etag, so that imageProxy.should304() returns true causing TranformingTransport.RoundTrip() to return a bare 304 response.

If those conditions are met, then the bare 304 Response returned will be used and read by one of the callers. Specifically, the lack of a Body causes a segfault.

So let's make it more like a real Response and use http.NoBody so when it's used it doesn't cause things to explode.


This issue involves a complex set of interactions between objects, which I tried to illustrate in a flow diagram here.

image

This diagram may not actually be helpful to readers, and for that I'm sorry -- but constructing it was very helpful to me to understand the preconditions and the fix (which is relatively simple).

to address the segfault we're seeing in production. The necessary
conditions for the issue are:

1. The 303 must return a location URI that changes every time, so that
   the real location of the image is never cached. As an example, if the
   remove service redirects to S3, the presence of the
   `X-Amz-Security-Token` accomplishes this.

2. The image responses must match by Etag, so that
   imageProxy.should304() returns true causing
   TranformingTransport.RoundTrip() to return a bare 304 response.

If those conditions are met, then the bare 304 Response returned will
be used and read by one of the callers. Specifically, the lack of a
Body causes a segfault.

So let's make it more like a real Response and use http.NoBody so when
it's used it doesn't cause things to explode.
@willnorris
Copy link
Owner

I knew the recursive transforming transport was a little bonkers when I first wrote it... I keep meaning to revisit that, or at the very least document how it works. This diagram is simultaneously amazing and terrifying... thanks for sharing this! Walking through the test cases now, but this fix looks pretty straightforward.

@codecov
Copy link

codecov bot commented Jun 6, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.75%. Comparing base (ef50c1f) to head (7428b67).
Report is 23 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (ef50c1f) and HEAD (7428b67). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (ef50c1f) HEAD (7428b67)
2 1
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #472       +/-   ##
===========================================
- Coverage   85.21%   59.75%   -25.47%     
===========================================
  Files           6       12        +6     
  Lines         717     1200      +483     
===========================================
+ Hits          611      717      +106     
- Misses         76      451      +375     
- Partials       30       32        +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@willnorris willnorris merged commit df0b6d3 into willnorris:main Jun 6, 2025
9 of 10 checks passed
@flavorjones flavorjones deleted the flavorjones/upstream-304-patch branch June 6, 2025 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants