Add an optional cache for ParquetFragmentStreamer

[caciolai]

What does this PR do? Please describe:

• When instantiating N data pipelines across different partitions of the same Parquet Dataset, currently ParquetFragmentStreamer will initialize N separate Dataset instances, with much duplicated work in filesystem scanning metadata parsing etc

Does your PR introduce any breaking changes? If yes, please list them:

• The cache mechanism is controlled by a flag set to False by default
• Previously existing code path(s) are not affected

Check list:

• Was the content of this PR discussed and approved via a GitHub issue? (no need for typos or documentation improvements)
• Did you read the contributor guideline?
• Did you make sure that your PR does only one thing instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests?
• Did you verify new and existing tests pass locally with your changes?
• Did you update the CHANGELOG? (no need for typos, documentation, or minor internal changes)

[artemru]

i would pass max_cache_size as a param here instead of use_cache

[artemru]

i would pass max_cache_size as a param here instead of use_cache

[caciolai]

yes but how can you set dynamically (tied to an object instance) a singleton component that you want to exist across instances?

[artemru]

maybe as some global context...

[caciolai]

what if different ParquetFragmentStreamer instances ask for different cache sizes? or were you thinking of something like AssetStore i.e. a singleton handled by fairseq2 and controlled via the recipe? that would be a deeper change though I think, as I would need to touch recipe composition

[caciolai]

thinking that we probably want to also cache

• stream_parquet_fragments
• list_parquet_fragments
  as they also do some heavy ops, even if re-using the same dataset object...