Merge Dask-GLM into Dask-ML #557
Conversation
Work taken from combined effort with @mcg1969
1. Keep `y` as a dask array rather than a numpy one (this may introduce a performance regression for threaded use) 2. Use dispatched `dot` rather than `ndarray.dot` in a few cases 3. Use numpy for the lstsq computation
- uses dask for x-update step in ADMM
- Flake8 fixes
…nsultan-master
Merge branches/forks, with the caveat that further testing needs to be added for the ADMM code.
…to merge-dask-glm
|
I'm still in the process of familiarizing myself with dask-ml as a user (I'm evaluating whether we can/should use it in astroML), but if I can, I would love to help out e.g. with this PR, so no more need to familiarize both with dask-ml and dask-glm. Is it only the conflicts that's holding up this PR, or something substantial? I can certainly deal with the former (and fix a few CI related open issues), and if there is a clear way I can deal with the latter, too. |
|
This fell on the back-burner since it's mostly just a development workflow thing. It shouldn't have any user-facing changes. The high-level split will still be
Which pieces were you interested in? |
|
FYI @mmccarty fixed the merge conflicts. Will see if CI passes. |
|
Am curious what the status is here. Is the plan still to get this in? |
15 participants
This cleans up code duplication (estimators, sparse handling) and merges in dask/dask-glm#79. I'm hoping that by focusing things here I'll be able to stay on top of things better.
This PR has the original commit history from dask-glm. I think we should avoid squashing so that those commits are preserved.
Closes #243