Activate base conda env in entrypoint.sh

[jayavenkatesh19]

Towards #857

Activates the base environment before Jupyterlab is launched by adding the conda activation script to entrypoint.sh.

Me and @ncclementi built the container and tested out cupy=14.0.1 on this container and everything runs smoothly

CC: @taureandyernv

[jameslamb]

Thanks, I think this and #859 are complementary and it could be worth pursuing both. But please see my suggestions about testing.

[jameslamb]

Suggested change

	- 'test -d "$CONDA_PREFIX" && conda config --show-sources'
	- '[[ "${CONDA_PREFIX}" == "/opt/conda" ]];'

conda config --show-sources will always be successful as long as conda is installed and on PATH, won't it? I'm confused about why that's included here. If it's just left over from debugging, please remove it.

Here we should also test for the exact value of CONDA_PREFIX, not only that it's set... these tests are expected to describe the things that should always be true of these images, to help us avoid breaking changes. e.g. if we change CONDA_PREFIX to /opt/rapids/conda or something in the future, we should be alerted to that potentially being a user-facing breaking change by a test failing.

And we should test that env base is activated here, too. This image still runs the entrypoint and therefore activates the environment, doesn't it?

docker/Dockerfile

Line 185 in ae4d636

ENTRYPOINT ["/home/rapids/entrypoint.sh"]

[jayavenkatesh19]

Oops that did slip through from debugging.

Following the code in run-validation-checks.sh, I put both the tests in base.yml as neither of them are notebook-specific and will need to be run on both the rapidsai images.

[jayavenkatesh19]

Looking at the CI failures, I realized that testing for CONDA_DEFAULT_ENV would need us to actually run the container, whereas using exec through container-canary bypasses the entrypoint.sh and does not activate the base environment. container-canary also does not support adding -i flag for an interactive shell to test out environment activation in .bashrc

As a result, I am leaving the test out.

It is possible to use a login shell with the exec probe from container-canary and that should cause the environment to be active and it looks like #859 is testing that, so happy to leave this for that PR.

[jameslamb]

Ah ok, then yeah this is fine. Thanks for working through that.

[jakirkham]

Thanks Jaya! 🙏

Agree with James this would be good to include. However have left a couple comments that we should follow up on first

[jakirkham]

Sourcing the profile.d script is handled by a login shell along with any other configurations included there. So think we should do this step by configuring a login shell. Otherwise we risk missing other relevant scripts and recreating the logic already included in top-level profile configuration

Running conda activate is also good to include in the entrypoint. Would drop base as that is implied by conda activate. Also over time the main environment has been renamed from root to base. So not specifying it would save us future potential churn

Lastly would do this before the install steps above so that the active environment can be used effectively

[jayavenkatesh19]

Love all of these suggestions, and they do make this more streamlined!

Thanks @jakirkham, I made these changes.

[jameslamb]

I'll merge this once CI passes. I can admin-merge past the failing check-nightly-ci job, and happy to do that because this will fix the nightlies.

Thanks for working through this!

[jameslamb]

I think this is in a good state and it looks to me like all reviewers' comments have been addressed, merging this to get CI going again.

Once it builds, @jayavenkatesh19 or @ncclementi could you trigger a new nightly run with tests at https://github.com/rapidsai/docker/actions/workflows/publish.yml?

[ncclementi]

There is something off on the versioning, but I can't find where the issue is.

After @jayavenkatesh19 re-triggered the publish, the images where publish but they show as 26.02a instead of 26.04a, see https://hub.docker.com/r/rapidsai/notebooks/tags . But I can't find any instance on the repo making reference to 26.02 the Docker file has ARG RAPIDS_VER=26.04 and all other instances seems to be referencing to 26.04.

cc: @jameslamb in case you've seen this before.

[jameslamb]

😫 I suspect something's wrong with the repo's tags.

The version is computed here:

docker/.github/workflows/build-test-publish-images.yml

Lines 113 to 126 in d9ca067

	- name: Compute RAPIDS_VER
	id: compute-rapids-ver
	run: |
	GIT_DESCRIBE_TAG="$(git describe --tags --abbrev=0)"
	GIT_DESCRIBE_TAG="${GIT_DESCRIBE_TAG:1}" # remove leading 'v'
	ALPHA_TAG=""
	if [[ $GIT_DESCRIBE_TAG =~ [a-z] ]]; then
	echo "Most recent tag is an alpha tag"
	ALPHA_TAG="a"
	fi
	RAPIDS_VER="$(echo $GIT_DESCRIBE_TAG | awk 'BEGIN{FS=OFS="."} NF--')" # Convert full tag to YY.MM
	echo "RAPIDS_VER=${RAPIDS_VER}" | tee -a ${GITHUB_OUTPUT}
	echo "ALPHA_TAG=${ALPHA_TAG}" | tee -a ${GITHUB_OUTPUT}

$ git checkout main
$ git pull upstream main
$ git fetch upstream --tags
$ git describe --tags --abbrev=0
v26.02.00a

That's a problem, we want to see v26.04.00a there.

Probably some mistake with #851.

Since we haven't published ANY v26.04.00a nightlies yet, thankfully the fix should be easy... delete the existing v26.04.00a tag, recreate it at a point after v26.02.00a. I'll do that.

[jameslamb]

I think I've fixed this, see https://github.com/rapidsai/github-infrastructure/issues/52#issuecomment-4034306294

@jayavenkatesh19 @ncclementi can you please try another build and let me know how it goes?

[ncclementi]

@jameslamb Great work, that fixed it. The job is still in progress, but some of the images are published already with the right tag.

[jameslamb]

Ok great! Thank you for helping to get builds going here again!