Enforce 6-Hour Hard Limit on Multipart Uploads (lakeFS LinkAddressTime Constraint)

[carloea2]

Background

In our current upload flow we:

1. Call lakeFS to initiate a multipart upload, which returns a physical address + uploadId.
2. Upload each part directly to MinIO using that address.
3. Finalize by calling lakeFS to complete (with the MinIO ETags) or abort (with the same address + uploadId).

Problem

lakeFS treats the “link address” returned during init as time-bounded.

• There’s a constant LinkAddressTime set to 6 hours:
  https://github.com/treeverse/lakeFS/blob/master/pkg/catalog/catalog.go#L84
• In the completion path, lakeFS rejects finalize/abort operations if the time since address creation exceeds that window:
  https://github.com/treeverse/lakeFS/blob/e77dd296d6e3d84e6083928bcc82fa186eed917c/pkg/catalog/catalog.go#L3197

So any multipart upload that takes longer than 6 hours will fail on complete/finish/abort.

Options considered

Option A: Build/patch lakeFS ourselves

Option B: Use MinIO multipart APIs directly, then import into lakeFS

• Pros: Avoids the lakeFS 6-hour “link address” limitation for the multipart upload itself.

• Cons: Import commits the file automatically, which breaks assumptions in our frontend:

  • We currently rely on diff to show uploaded files as uncommitted changes and then create a version explicitly.
  • If files are committed immediately via import, we would need broader changes across endpoints and UI flows to preserve expected behavior.

Option C: enforce a hard 6-hour limit for uploads

[chenlica]

I support Option C. Hopefully the LakeFS community can make this parameter configurable in the future so that our problem will be solved automatically.

[xuang7]

Quick update on the 6-hour limitation:

• Reached out to the LakeFS team regarding link address expiry time configurability. Currently it is not configurable and is tied to the GC grace period.
• Opened an upstream feature request.