Don't index sitemap in opensearch

[JaspreetSinghChahal]

In stormcrawler, I want to fetch and parse sitemaps to discover new urls but don't want to index it in opensearch.

I can't exclude it in regex as new urls won't be discovered then.

[rzo1]

The sitemap parser bolt will set a metadata key isSitemap and use filter keys in the indexer's configuration to remove them from indexing: indexer.md.filter

[JaspreetSinghChahal]

It seems to be taking only single key. How to do it for more keys, example "isSitemap=false", "isFeed=false"?

[rzo1]

@JaspreetSinghChahal There’s a mismatch between the current implementation and the documentation. From my perspective, it makes more sense to align the implementation with what the documentation specifies. Therefore, it would be useful to allow this parameter to accept a YAML list instead of just a single key-value.

Any implementation should remain backward compatible, still supporting the previous configuration format. Would you be interested in submitting a PR with the fix and a test?