fix: Need to include zip64 extra field in central directory (#353)

[Pr0methean]

Fix #353

[amazon-q-developer]

⏳ I'm reviewing this pull request for security vulnerabilities and code quality issues. I'll provide an update when I'm done

[amazon-q-developer]

✅ I finished the code review, and left comments with the issues I found. I will now generate code fix suggestions.

[Pr0methean]

/gemini

[gemini-code-assist]

Hi @Pr0methean! It looks like you tried to invoke me with /gemini. To use my features, please specify a command after /gemini.

For example:

• To request a code review: /gemini review
• To get a pull request summary: /gemini summary

You can also type /gemini help to see a full list of available commands.

[Pr0methean]

/Gemini help

[gemini-code-assist]

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

[Pr0methean]

@gemini-code-assist How can I fix the failing tests?

[gemini-code-assist]

Hi @Pr0methean, I understand you're looking for help with failing tests. While I don't have direct access to the specific error messages or logs from your CI/CD pipeline, which are usually the best source of information for debugging, I can offer some general guidance based on the changes in this pull request.

To effectively troubleshoot the failing tests, I recommend the following steps:

1. Review CI Logs Thoroughly: The most crucial step is to examine the detailed output from your CI runs. Look for:

   • Error messages and stack traces: These will pinpoint the exact location and nature of the failure.
   • Test names: Identify which specific tests are failing.
   • Environment details: Note the Rust version (MSRV, stable, nightly), feature flags (--no-default-features, --all-features, default features), and operating system (Windows, MacOS, Ubuntu) under which the tests are failing.

2. Reproduce Locally: Try to reproduce the failing tests on your local machine using the same environment (Rust version, features, OS) as the CI. This will allow for faster iteration and debugging.

3. Focus on Changed Areas: Based on the changes in this pull request, the most likely areas to investigate for new test failures are:

   • tests/append_near_4gb.rs: This is a newly added test file, and new tests often expose edge cases or regressions. Pay close attention to the assertions and the data being written/read in these tests, especially test_append_near_4gb and test_append_near_4gb_with_1gb_files, as they deal with large file sizes and ZIP64 boundaries.
   • src/read.rs - parse_extra_field function: The logic for parsing extra fields has been significantly refactored. Any issues in correctly interpreting or preserving these fields (especially ZIP64 extra fields) during reading could lead to validation failures in existing or new tests.
   • src/write.rs - ZIP64 handling: The new logic to force ZIP64 for files starting near the 4GB boundary (if header_start > spec::ZIP64_BYTES_THR) and the explicit writing of zip64_extra_field_block in the central directory header are critical. Ensure these changes correctly generate valid ZIP64 structures that can be read back by the zip crate itself or other ZIP tools.

Remember that the PR description highlights the importance of passing all checks across the 3-dimensional matrix (MSRV/stable/nightly, feature flags, and OS). A failure in one specific combination might indicate an environment-dependent issue or a subtle bug related to feature interactions.

Once you have the specific error messages, you can often narrow down the problem much more quickly. Let me know if you get more details from the CI logs!

[alex-sc-kao]

@Pr0methean Hi I'm working on making a large zip (500 files, ~8GB), with FileOptions.large_file(true).
But I still got an error Zip archive inconsistent: entry 233: invalid Zip64 extra field while opening the zip file, is it related with this PR?

[Pr0methean]

@alex-sc-kao Possibly. Are any of this branch's tests failing on your machine?

[alex-sc-kao]

@alex-sc-kao Possibly. Are any of this branch's tests failing on your machine?

yeah, that invalid_header test failed on my machine too

test invalid_header ... FAILED

failures:

---- invalid_header stdout ----

thread 'invalid_header' (1619591) panicked at tests/issue_234.rs:29:18:
Unexpected value: Err(Io(Error { kind: UnexpectedEof, message: "failed to fill whole buffer" }))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace