-
Notifications
You must be signed in to change notification settings - Fork 4
Description
With the new ExperimentRepeater, we discourage within-trial restarts. For larger experimental budgets, we may now have many instance repetitions.
To "negotiate" between the old and the new experimental data, the cocopp module has recently implemented a (tentative) evals_appended property to resemble the old within-trial restarted evals from the new data. evals_appended are possibly used in the median_evals method, however only when evals_are_appended is True. However, without "faking" restarts, the median computation is not commensurable with the ert.
A better way to use the new data seems to be to extend evals_with_simulated_restarts with the option to have only within-instance restarts, which is in particular applicable when instances_are_uniform is False. Computing within-instance restarts could be based on DataSet.detEvals_by_instance.