Current implementation tests on

[henrikmarklund]

Hey, thanks for sharing!

When training on partially corrupted labels, shouldn't we still evaluate on accurate labels? Current implementation evaluates on a dataset that has corrupted labels.

Or perhaps I'm missing something?

Thanks!

[pluskid]

@henrikmarklund Depending on what you want to do. If you want to evaluate the generalization performance under the situation where the underlying data distribution has some probability of generating random labels, then you need to set the test data to also contain random labels. If on the other hand, you want to evaluate what is the effect of artificially corrupted training data with random labels, then the test data could be the original clean data. Hope this clarifies your question.

[henrikmarklund]

It does! Thank you! :)