Pull requirements/host from pyproject.toml

[jakirkham]

In addition to pulling the Conda recipe's requirements/run from pyproject.toml, pull requirements/host from pyproject.toml. This should help solidify around pyproject.toml being the single source of truth.

[jakirkham]

Toggling for CI

[pentschev]

Failures are unrelated, tracking that in #1134 .

[pentschev]

LGTM, thanks @jakirkham !

[ajschmidt8]

@jakirkham, is this a pattern we plan on using for other RAPIDS libs?

I think dask-cuda's requirements are a bit simpler than other libs (e.g. no pin_compatible or pin_subpackage).

So I'm wondering if this would work for more complex cases

[jakirkham]

Yeah this is something I'm looking into. Please see PR ( rapidsai/rmm#1220 ) for example

[ajschmidt8]

Temporarily requesting changes to block this PR from being merged.

I think we need to talk more about dependency management based on @vyasr's comment below.

• rapidsai/rmm#1220 (comment)

Seems like we have conflicting efforts in motion so we should probably sync on the best way to handle this so that we're consistent across all repositories.

[jakirkham]

Generally agree we should discuss how we handle requirements in RAPIDS more generally

However with Dask-CUDA would push back on this slightly since there is a longer history here of wanting requirements in one place that is pulled into other sources ( #569 ) ( #893 ). Additionally Dask-CUDA being a pure Python package has generally been more aligned with requirements in Conda & PyPI packages. So in the Dask-CUDA case this is more a continuation of how things have been as opposed to a new approach

[vyasr]

The main question from the rmm PR that IMO needs to be answered before we move forward with changes like this even in dask-cuda is how we want to handle the interaction between rapids-dependency-file-generator and meta.yaml reading from pyproject.toml. Those are two different approaches for single-sourcing dependencies, and I would prefer not adding two levels of indirection (dependencies.yaml->pyproject.toml->meta.yaml) where only one might suffice if we could directly write meta.yaml from dependencies.yaml. The open questions in the rmm PR would help us understand whether there is a strong argument in favor of the two pass approach (maybe because we simply decide that supporting meta.yaml is too much trouble for dfg).