Make fixed length metadata available

[dmichalski]

Hi,

I have a use case where I read a file with fixed length columns and write some of the rows back into another file (with same format). I can't use getRawdata() as the memory footprint would be too high.

My problem now is that whenever I access a column which is filled with blanks in the input file it is treated as empty and therefore the output row becomes shorter than the original. E.g.::

Mapping:

<PZMAP> <COLUMN name="ColA" length="4" /> <COLUMN name="ColB" length="4" /> <COLUMN name="ColC" length="4" /> </PZMAP>
Input:
AAAA CCCC

Output (getString("ColA") + getString("ColB") + getString("ColC")):
AAAACCCC

If I would know the metadata of the col (as defined e.g. in the metaData attribute in the RwoRecord) a possible workaround would be to use String getString(String var1, Supplier<String> var2);. Of course an extension to getString to do the filling automatically would be even greater imho.

So questions are:

1. Am I missing something out? Is there functionality to do what I want?

2. Is there a reason for the metasata not accessible?

Greetings,

Dennis

[benoitx]

Hi Dennis I'd be happy to check this but it would really help if you could provide with a Unit Test that I could include in the build? Many thanks Benoit

…

On Tue, 2 Jun 2020 at 12:52, dmichalski ***@***.***> wrote: Hi, I have a use case where I read a file with fixed length columns and write some of the rows back into another file (with same format). I can't use getRawdata() as the memory footprint would be too high. My problem now is that whenever I access a column which is filled with blanks in the input file it is treated as empty and therefore the output row becomes shorter than the original. E.g.:: Mapping: <PZMAP> <COLUMN name="ColA" length="4" /> <COLUMN name="ColB" length="4" /> <COLUMN name="ColC" length="4" /> </PZMAP> Input: AAAA CCCC Output (getString("ColA") + getString("ColB") + getString("ColC")): AAAACCCC If I would know the metadata of the col (as defined e.g. in the metaData attribute in the RwoRecord) a possible workaround would be to use String getString(String var1, Supplier<String> var2);. Of course an extension to getString to do the filling automatically would be even greater imho. So questions are: 1. Am I missing something out? Is there functionality to do what I want? 2. Is there a reason for the metasata not accessible? Greetings, Dennis — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#55>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB542I52JYITKV6RRHZNLTRUTRYLANCNFSM4NQU5G3A> .

[dmichalski]

Hi, I am willing to help as far I can but don't understand exactly what you need. A test which reflects the desired behaviour? Greetings Dennis Benoit Xhenseval <notifications@github.com> schrieb am Di., 2. Juni 2020, 17:24:

…

Hi Dennis I'd be happy to check this but it would really help if you could provide with a Unit Test that I could include in the build? Many thanks Benoit On Tue, 2 Jun 2020 at 12:52, dmichalski ***@***.***> wrote: > Hi, > > I have a use case where I read a file with fixed length columns and write > some of the rows back into another file (with same format). I can't use > getRawdata() as the memory footprint would be too high. > > My problem now is that whenever I access a column which is filled with > blanks in the input file it is treated as empty and therefore the output > row becomes shorter than the original. E.g.:: > > Mapping: > > <PZMAP> <COLUMN name="ColA" length="4" /> <COLUMN name="ColB" length="4" > /> <COLUMN name="ColC" length="4" /> </PZMAP> > Input: > AAAA CCCC > > Output (getString("ColA") + getString("ColB") + getString("ColC")): > AAAACCCC > > If I would know the metadata of the col (as defined e.g. in the metaData > attribute in the RwoRecord) a possible workaround would be to use String > getString(String var1, Supplier<String> var2);. Of course an extension to > getString to do the filling automatically would be even greater imho. > > So questions are: > > 1. > > Am I missing something out? Is there functionality to do what I want? > 2. > > Is there a reason for the metasata not accessible? > > Greetings, > > Dennis > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#55>, or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAB542I52JYITKV6RRHZNLTRUTRYLANCNFSM4NQU5G3A > > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#55 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2PHEOZ27VLUMVCGCLUXTLRUUKRRANCNFSM4NQU5G3A> .

[benoitx]

Yes, so it will fail until I fix the problem. Thanks Benoit

…

On Tue, 2 Jun 2020 at 19:45, dmichalski ***@***.***> wrote: Hi, I am willing to help as far I can but don't understand exactly what you need. A test which reflects the desired behaviour? Greetings Dennis Benoit Xhenseval ***@***.***> schrieb am Di., 2. Juni 2020, 17:24: > Hi Dennis > > I'd be happy to check this but it would really help if you could provide > with a Unit Test that I could include in the build? > > Many thanks > > Benoit > > On Tue, 2 Jun 2020 at 12:52, dmichalski ***@***.***> wrote: > > > Hi, > > > > I have a use case where I read a file with fixed length columns and write > > some of the rows back into another file (with same format). I can't use > > getRawdata() as the memory footprint would be too high. > > > > My problem now is that whenever I access a column which is filled with > > blanks in the input file it is treated as empty and therefore the output > > row becomes shorter than the original. E.g.:: > > > > Mapping: > > > > <PZMAP> <COLUMN name="ColA" length="4" /> <COLUMN name="ColB" length="4" > > /> <COLUMN name="ColC" length="4" /> </PZMAP> > > Input: > > AAAA CCCC > > > > Output (getString("ColA") + getString("ColB") + getString("ColC")): > > AAAACCCC > > > > If I would know the metadata of the col (as defined e.g. in the metaData > > attribute in the RwoRecord) a possible workaround would be to use String > > getString(String var1, Supplier<String> var2);. Of course an extension to > > getString to do the filling automatically would be even greater imho. > > > > So questions are: > > > > 1. > > > > Am I missing something out? Is there functionality to do what I want? > > 2. > > > > Is there a reason for the metasata not accessible? > > > > Greetings, > > > > Dennis > > > > — > > You are receiving this because you are subscribed to this thread. > > Reply to this email directly, view it on GitHub > > <#55>, or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AAB542I52JYITKV6RRHZNLTRUTRYLANCNFSM4NQU5G3A > > > > . > > > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#55 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AA2PHEOZ27VLUMVCGCLUXTLRUUKRRANCNFSM4NQU5G3A > > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#55 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB542LLHVLUYIKXLZRCEGDRUVCERANCNFSM4NQU5G3A> .

[dmichalski]

Ok. As I said it is not really a problem, more a lack of feature and the implementation of the feature could be discussed. But anyway I wrote tests for two ideas how to do it.

Hope that helps...

FixedLengthStringRetrievalTest.java.txt