Skip to content

Add pre-commit hook to check dask version consistency#148

Merged
rapids-bot[bot] merged 2 commits intorapidsai:mainfrom
bdice:add-dask-version-check-hook
Jan 29, 2026
Merged

Add pre-commit hook to check dask version consistency#148
rapids-bot[bot] merged 2 commits intorapidsai:mainfrom
bdice:add-dask-version-check-hook

Conversation

@bdice
Copy link
Contributor

@bdice bdice commented Jan 29, 2026

Summary

  • Adds a pre-commit hook that validates dask, dask-core, and distributed versions are consistent across dependencies.yaml, pyproject.toml, and conda/recipes/rapids-dask-dependency/recipe.yaml

This addresses the issue seen in #142 and #147 where the conda recipe was not updated when the dask version was changed in dependencies.yaml.

The hook:

  • Uses only Python standard library (no external dependencies)
  • Only runs when relevant files are modified
  • Provides clear error messages showing which files have mismatched versions

@bdice bdice requested review from a team as code owners January 29, 2026 00:51
@bdice bdice requested a review from KyleFromNVIDIA January 29, 2026 00:51
@bdice bdice added feature request New feature or request non-breaking Introduces a non-breaking change labels Jan 29, 2026
@jakirkham jakirkham added improvement Improves an existing functionality and removed feature request New feature or request labels Jan 29, 2026
@bdice bdice self-assigned this Jan 29, 2026
@bdice
Copy link
Contributor Author

bdice commented Jan 29, 2026

This will fail checks until #147 is forward-merged.

@jakirkham
Copy link
Member

Thanks Bradley! 🙏

So this should fail already as they are inconsistent currently

python_run:
common:
- output_types: [pyproject, requirements]
packages:
- dask==2026.1.1
- distributed==2026.1.1

dependencies = [
"dask==2026.1.1",
"distributed==2026.1.1",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit dependencies.yaml and run `rapids-dependency-file-generator`.

run:
- dask ==2025.12.0
- dask-core ==2025.12.0
- distributed ==2025.12.0

@bdice
Copy link
Contributor Author

bdice commented Jan 29, 2026

Output:

check dask version consistency...........................................Failed
- hook id: check-dask-versions
- exit code: 1

ERROR: Dask version inconsistency detected!

Dask version mismatch:
  dependencies.yaml: 2026.1.1
  pyproject.toml: 2026.1.1
  recipe.yaml: 2025.12.0
  recipe.yaml (dask-core): 2025.12.0
Distributed version mismatch:
  dependencies.yaml: 2026.1.1
  pyproject.toml: 2026.1.1
  recipe.yaml: 2025.12.0

All dask, dask-core, and distributed versions must be identical across:
  - dependencies.yaml
  - pyproject.toml
  - conda/recipes/rapids-dask-dependency/recipe.yaml

https://github.com/rapidsai/rapids-dask-dependency/actions/runs/21461284750/job/61814178623?pr=148

@pentschev
Copy link
Member

This is a great idea, thanks @bdice ! I now have a new, more challenging ask based on this idea, can we proliferate this idea and do the same with our dependencies.yaml and our conda recipe.yaml files? It is more challenging because there's not always an equivalency, sometimes we need to dumb down pip packages due to missing packages in PyPI (for example, OpenMPI and NUMA), and package names are not always consistent, so I'm uncertain whether that can be done (programmatically at least).

@pentschev
Copy link
Member

/merge

@rapids-bot rapids-bot bot merged commit 4ce610c into rapidsai:main Jan 29, 2026
11 checks passed
@bdice
Copy link
Contributor Author

bdice commented Jan 30, 2026

can we proliferate this idea and do the same with our dependencies.yaml and our conda recipe.yaml files?

Great question. tl;dr We haven't implemented this yet. It is tracked in rapidsai/dependency-file-generator#7. This was blocked for a long time due to the meta.yaml format not being especially conducive to generation, but it may be possible now that we've migrated to rattler-build and recipe.yaml.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improves an existing functionality non-breaking Introduces a non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants