How to name and fetch experiments when new parameters are added?

[pulkitag]

The ability to conveniently run parameter sweeps greatly eases experimentation. Right now this is very well supported if the parameters which are being swept are defined apriori. However, almost everytime a new parameter is introduced after looking at results of already conducted experiments. When this new parameter is added, the naming of experiment changes because a different hash string is produced in presence of this new variable. Therefore, it becomes impossible to retrieve old experiments without putting if conditions that conditionally ignore certain variables while generating hash strings. This is inconvenient.

Possible solutions include:

• Creating a database that saves the parameter of each experiment and the associated hash string. So now if a new parameter is added it will not match the list of already stored variables. The con of this approach is that if the database file is lost, one loses track of all the experiments. Thus this database file must be carefully backed up.

Any other alternatives?