Need to get full header

[jimmywarting]

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the feature has not already been requested

🚀 Feature Proposal

I need to get the full raw headers from each field, as is, without any modification.

Motivation

You do strip out quite a bit, and leave many things behind.

headers that are not known to you gets discarded, headers meta ;key=value pair gets discarded too.

And some things even get modified as you apply to lowercase here and there.

I need to parse the headers on my own, so it would be good if you could send the headers as a 2D iterable array
(as one header could appair twice)

// x-forwared-for: 192.168.0.1
// x-forwared-for: 192.168.0.2

headers = new Headers([
  ['x-forwared-for', '192.168.0.1'],
  ['x-forwared-for', '192.168.0.2']
])
headers.get('x-forwared-for') // 192.168.0.1, 192.168.0.2

Example

the arguments length is getting out of control cuz they are so many now. and i don't need all of them... An object would be better...

-   busboy.on('field', (fieldname, val, fieldnameTruncated, valTruncated, encoding, mimetype) => {
+   busboy.on('field', (field) => {
      console.log(`Field [${field.fieldname}]: value: ${field.val}`);
    });

so that i could do:

busboy.on('field', (field) => {
  console.assert(Array.isArray(field.headers))
  Object.fromEntries(field.headers)
  new Headers(field.headers)
})

[Uzlopak]

@gurgunday
@KhafraDev

Your opinions? Afaik you wanted to replace busboy in undici with this version. Do you see any drawbacks?

[jimmywarting]

fyi, related to: nodejs/undici#2283

[Uzlopak]

@gurgunday

Can you please take over this issue? I am currently bound by refactoring probot as at work upgrading our github bots is stalled. So I am kind of mentally blocked to handle more than two super complex packages.

I see following tasks:

• talk with @KhafraDev and other undici majntainers regarding this issue, and determine if this is what they want
• investigate how to return full headers, and why we had limited headers in the first place (maybe there is something we oversee)
• investigate the proposal to use object-parameter, also in regard of performance of instantiating objects
• probably unit tests have to be heavily touched

[jimmywarting]

undici is probably not going to expose full headers from any field, it's just for mine and other sake of being able to know what all the headers on a given field are...

some custom libraries batch multiple request into one single multipart/form-data
https://docs.arangodb.com/3.11/develop/http/batch-requests/

So knowing all headers could be useful for others who are interested in this busboy. (like a custom express middleware)

[jimmywarting]

2D array is important b/c a header could be added twice.
like set-cookie

[gurgunday]

Your opinions? Afaik you wanted to replace busboy in undici with this version. Do you see any drawbacks?

Should be doable, I'll try to take a look at this

[Uzlopak]

@jimmywarting
I am not against this proposal. I just want to have a clear scope. If undici doesnt need it per se but you need it, we should maybe make it configurable with a option, which maybe could be passed down from undici to busboy.

[KhafraDev]

Fields (Section 5) that are sent or received before the content are referred to as "header fields" (or just "headers", colloquially).

field-value = field-content
field-content = field-vchar
[ 1( SP / HTAB / field-vchar ) field-vchar ]
field-vchar = VCHAR / obs-text
obs-text = %x80-FF

I would expect the header parsing to allow semicolons.

[jimmywarting]

If undici doesnt need it per se but you need it

Undici is going to need it to get access to the raw header in order to get the full / as is content-type
it needs it do create a File with the exact same type, as provided by content-type... no cut of meta key/value pairs. no lowercased transformation.
(maybe also being able to parse content-disposition attachment header in a own way if needed)

undici is just not going to expose the raw headers to the client using fetch.