Request for Inclusion of Driver and README Files in HLKX Generation

[zjmletang]

Currently, it seems that the HLKX files generated during testing do not include the driver and README files. The README file is meant to describe the errata. As a result, if we want to submit it for signing by Microsoft, we will need to manually repackage the driver and README files. Is it possible to have the generated HLKX file include both the driver and README files? This would reduce one manual step

[kostyanf14]

Initially, this is a task for https://github.com/HCK-CI/rtoolsHCK.

Is it possible to have the generated HLKX file include both the driver and README files?

I don't know. Need to investigate HLK API .

[YanVugenfirer]

As a separate task I would add also the signing of hlkx package as part of the full cycle

[kostyanf14]

@zjmletang

Is it possible to have the generated HLKX file include both the driver and README files?

Theoretically yes, https://learn.microsoft.com/en-us/windows-hardware/test/hlk/api/packagewriteradddriver-method and https://learn.microsoft.com/en-us/windows-hardware/test/hlk/api/packagewriteraddsupplementalfiles-method. I can't prioritize this task for now based on our plans, but as usual, feel free to send PRs for review. Initial patches are needed for https://github.com/HCK-CI/rtoolsHCK

[zjmletang]

@kostyanf14 Sure, thank you very much. I'll submit a PR later, but it might take some time as I've been quite busy recently.

[kostyanf14]

Hi @zjmletang

Do you have any POC of this feature?

[zjmletang]

@kostyanf14 Not yet. I've been tied up with other matters for the past six months. If anyone has time, feel free to take it on.

[zjmletang]

Hi @kostyanf14
I have developed a PoC (Proof of Concept) for this feature. I was wondering if there is any existing implementation in progress or if this is already on your roadmap? If not, I’d be happy to clean up my patches and submit them for your review to see if they would be a good fit for the project."

Proposed Solution
Add two new command-line options to include driver and supplemental files directly in the generated HLKX package:
--driver-path - Path to the driver directory (reusing existing option)
--supplemental-path - Path to supplemental files directory (new option)

demo:
nohup bin/auto_hck test --drivers NetKVM --platform Win2022x64 --driver-path /root/letang/HCK-CI/driver/58120/netkvm/ --package-with-playlist --select-test-names /root/letang/HCK-CI/driver/58120/netkvm/test.json --supplemental-path /root/letang/HCK-CI/driver/58120/netkvm/supp/ &>succ&

[kostyanf14]

@menli820 Please sync with @zjmletang about this task.

@zjmletang We planned to do it, but due to the holiday, I have no info.

Add two new command-line options to include driver and supplemental files directly in the generated HLKX package:
--driver-path - Path to the driver directory (reusing existing option)
--supplemental-path - Path to supplemental files directory (new option)

I preferred to add --package-with-driver and --supplemental-path (or --add-supplemental-path) options. First of all, for backward compatibility. The second one is to prevent a crash before we make sure that the API works for all (or at least the latest) HLK releases.

[zjmletang]

By the way, should we consider adding a flag to control the merging of external .hlkx files during packaging? We’ve seen cases where specific tests (eg. NDISTest 6.5 - [2 Machine] - OffloadLSO) require standalone execution to pass. Having this parameter would provide the flexibility to merge those individually run test results.

[kostyanf14]

should we consider adding a flag to control the merging of external .hlkx files during packaging?

Short answer - Yes
Long answer: Let's do "driver" and "supplemental" as a step 1, and merging as a step 2. I want to discuss how you and our QE use merging operation to make this operation more generic.

[zjmletang]

@kostyanf14 The two-step plan sounds perfect to me.

To answer your question, my use case is quite straightforward:

Standalone Phase: I first run the specific tests that require standalone execution (to ensure they pass or to handle special environment requirements).
Full Suite Phase: Then, I trigger the full test suite. My goal is to be able to specify those previously generated standalone .hlkx files during this stage.
The Result: The final .hlkx package generated by the full run should already include those merged results, making it ready for Microsoft submission immediately.

One further suggestion:
It would be even better to have a "post-action" feature, where users can trigger their own scripts once the test is complete. This would allow me to automatically move or collect the generated .hlkx files into a designated directory for easier management before the final merging step.

[menli820]

@zjmletang @kostyanf14 Thanks for the sharing and comments

1. QE agreed with the --package-with-driver and --supplemental-path (or --add-supplemental-path) options .
2. And as far as I know, QE don's suffer the specific tests that require standalone execution to pass until now.
3. Merging operation ： We indeed tend to merge win10_x86. hlkx (with all case pass except the errata) and win10_x64.hlkx (with all case pass except the errata) packages into one hlkx package to generage a win10.hlkx then submit it to the MSFT.

[zjmletang]

@kostyanf14 @menli820
Regarding this task, I have submitted two PRs. Please review them when you have a moment.
#830
HCK-CI/rtoolsHCK#67

@menli820 I’ve verified that they work correctly in my local environment, but it would be great if you could also validate them in your test environment if possible.

[kostyanf14]

@zjmletang Great! Many thanks