Make sure that bn_type is set to NonAffineBN for SplitCIFAR experiments! This was an error in our initial release of this repo.