Emit explicit directory entries in tar build context

[andrew-anyscale]

Without explicit directory entries, BuildKit synthesizes intermediate directories with permissions that vary across Docker versions and platforms. On some Docker Engine versions on bare Linux, these directories end up non-traversable by non-root users, causing permission denied errors during COPY.

Topic: fix-tar-dir-perms
Labels: draft
Signed-off-by: andrew andrew@anyscale.com

[andrew-anyscale]

Reviews in this chain:
└#459 Emit explicit directory entries in tar build context

[andrew-anyscale]

#	head	base	diff	date	summary
0	56084b78	ce65f90a	diff	Mar 16 19:15 PM	2 files changed, 117 insertions(+)
1	d78102c1	ce65f90a	diff	Mar 16 19:47 PM	2 files changed, 8 insertions(+)

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical issue where Docker Engine on certain platforms would encounter 'permission denied' errors when copying files due to BuildKit's inconsistent handling of implicit directory permissions within tar build contexts. By explicitly adding tar entries for all parent directories with a standard '0755' mode, the change ensures consistent and traversable directory structures, preventing build failures related to file access.

Highlights

• Explicit Directory Entries: Introduced a mechanism to explicitly emit tar headers for all implicit parent directories within the build context.
• Permission Fix: Addressed an issue where BuildKit's synthesized intermediate directories had inconsistent permissions, leading to 'permission denied' errors during 'COPY' operations for non-root users.
• New implicitDirs Function: Implemented a new helper function implicitDirs to identify and collect all necessary parent directory paths.
• Enhanced Testing: Added dedicated unit tests (TestTarStreamImplicitDirs, TestTarStreamImplicitDirsNone) to ensure the correct generation and handling of implicit directory entries in the tar stream.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• wanda/tar_stream.go
  • Imported the path package.
  • Added a new function implicitDirs to identify all parent directories for files being added to the tar stream.
  • Modified the writeTo method to iterate through and write explicit tar headers for these implicit directories before writing the actual file entries.
• wanda/tar_stream_test.go
  • Added TestTarStreamImplicitDirs to verify that implicit directories are correctly identified and written to the tar stream with the expected permissions.
  • Added TestTarStreamImplicitDirsNone to ensure that no implicit directories are generated when files are at the root level.

Activity

• The pull request was created by andrew-anyscale to address a known issue with tar build contexts and directory permissions.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a mechanism to explicitly add directory entries to the tar build context, which is a good approach to solve permission issues with synthesized directories in BuildKit. The implementation correctly identifies implicit directories from file paths and adds them to the tar stream. The new tests cover the main functionality well. I've found one potential issue in an edge case where a file entry might have a name that conflicts with an implicit directory. I've left a comment with a suggestion to make the logic more robust.

[aslonnie]

likely not needed, and it adds complexity.

[aslonnie]

probably what we need is supporting setting uid/gid's for user, so that user does not need to worry about syncing between user's filesystem and in-container-image filesystem