Has anyone managed to achieve plausible results in the first stage of training the LiDB?
If so, would you share your dataset and config?
I do not understand why this is this first step necessary at all, as it should be difficult achieving plausible results using a 1 rank LoRA on any dataset.