Sparse Sequential Merging Algorithm

[EMZEDI]

[EMZEDI]

Sequential merging with LoRA as a baseline.
Train on T1 -> save a copy -> without freezing, train on T2 -> merge the LoRA modules. (Iterative)

---

Since model merging is orthogonal to what algorithm is used for single task training, many ablations are needed to verify the accountability of the work. Possibly targeting a single type of non stationarity first, and generalizing later.