Skip to content

Add variant and metrics to events_plot mapping#223

Open
frederichaase wants to merge 1 commit intoclearml:masterfrom
frederichaase:master
Open

Add variant and metrics to events_plot mapping#223
frederichaase wants to merge 1 commit intoclearml:masterfrom
frederichaase:master

Conversation

@frederichaase
Copy link

We had some issues with duplicated plots in our ClearML deployment.

The reason was that Elastiscearch infered text for these fields and we do an aggregation query.

This PR sets them explicity to type keyword, which should make it more stable given that field inference works slightly different in different Elasticsearch versions.

@Ponsaille
Copy link

Happy new year!
Were are we on this PR, would be really nice to have this additional stability! @jkhenning

@jkhenning
Copy link
Member

Hi @Ponsaille, sorry for the delay 🙏
How would this work is you only add these to the schema? The server won't store them there, I think

@evg-allegro
Copy link
Contributor

Hi @Ponsaille , @fhaase2 which version of ES do you use? We already have this mapping in events.json file and it is applied to all the events indices. While more specific mappings just add missing fields for specific event types. Thus the resulting indices have the mapping that is combination of events.json and then a more specific one like events_plot.json or events_log.json etc. This works for us in ES versions from 5.x to 7.x including.
When you restart the apiserver container the mappings are applied as part of the apply_mappings script. Do you see any errors from it in the apiserver log?

@frederichaase
Copy link
Author

We use ES 8.5.1. And per default, the indices created automatically by ClearML have the wrong mappings in this version. So this PR makes sure that the created index has correct mapping, even in ES > 7.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants