Skip to content

Fix unified pip devcontainer UCX base tags #766

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
msarahan merged 4 commits into from
Sep 9, 2026
Merged

Fix unified pip devcontainer UCX base tags #766

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2286e75
Fix unified pip UCX base tags
msarahan Sep 8, 2026
5b72ce4
Keep pip OpenMPI from shadowing system UCX
msarahan Sep 9, 2026
93801e3
Pair conda tileiras with its CUDA toolchain
msarahan Sep 9, 2026
cfd9391
Remove devcontainer CUDA_HOME override
msarahan Sep 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/cuda12.9-pip/devcontainer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"args": {
"CUDA": "12.9",
"PYTHON_PACKAGE_MANAGER": "pip",
"BASE": "rapidsai/devcontainers:26.10-cpp-cuda12.9-ucx1.19.0-openmpi5.0.10"
"BASE": "rapidsai/devcontainers:26.10-cpp-cuda12.9-ucx1.21.0-openmpi5.0.10"
},
"cacheFrom": [
"ghcr.io/rapidsai/devcontainers/devcontainer:26.10-cuda12.9-pip"
Expand Down
2 changes: 1 addition & 1 deletion .devcontainer/cuda13.3-pip/devcontainer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"args": {
"CUDA": "13.3",
"PYTHON_PACKAGE_MANAGER": "pip",
"BASE": "rapidsai/devcontainers:26.10-cpp-cuda13.3-ucx1.19.0-openmpi5.0.10"
"BASE": "rapidsai/devcontainers:26.10-cpp-cuda13.3-ucx1.21.0-openmpi5.0.10"
},
"cacheFrom": [
"ghcr.io/rapidsai/devcontainers/devcontainer:26.10-cuda13.3-pip"
Expand Down
22 changes: 19 additions & 3 deletions features/src/rapids-build-utils/opt/rapids-build-utils/bin/make-pip-env.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,24 @@ make_pip_env() {
local -r new_env_path="$(realpath -m "/tmp/${env_file_name}")";
local -r old_env_path="$(realpath -m "${HOME}/.local/share/venvs/${env_file_name}")";

local -a excluded_packages=(ninja);

# The PyPI OpenMPI wheel bundles its own UCX libraries. Do not install it when the
# devcontainer is configured to use system UCX, because loading those bundled
# libraries alongside system UCX can produce an ABI-incompatible mix. The base
# image already provides an OpenMPI build linked against its system UCX.
if [ "${RAPIDS_LIBUCX_PREFER_SYSTEM_LIBRARY:-false}" = true ]; then
excluded_packages+=(openmpi);
fi

# Create the python env without ninja.
# ninja -$(ulimit -n) fails with `ninja: FATAL: pipe: Too many open files`.
# This appears to have been fixed 13 years ago (https://github.com/ninja-build/ninja/issues/233),
# so that fix needs to be integrated into the kitware pip ninja builds.
rapids-make-pip-dependencies --exclude <(echo ninja) "${OPTS[@]}" > "${new_env_path}";
rapids-make-pip-dependencies \
--exclude <(printf '%s\n' "${excluded_packages[@]}") \
"${OPTS[@]}" \
> "${new_env_path}";

if test -f "${new_env_path}"; then

Expand All @@ -63,7 +76,6 @@ make_pip_env() {
. "${HOME}/.local/share/venvs/${env_name}/bin/activate";
python -m pip install -U pip;
python -m pip install "${pre[@]}" -U -r "${new_env_path}";
python -m pip uninstall -y ninja >/dev/null 2>&1;
# If the venv does exist but it's different from the generated one,
# print the diff between the envs and update it
elif ! diff -BNqw "${old_env_path}" "${new_env_path}" >/dev/null 2>&1; then
Expand All @@ -82,9 +94,13 @@ make_pip_env() {
. "${HOME}/.local/share/venvs/${env_name}/bin/activate";
python -m pip install -U pip;
python -m pip install "${pre[@]}" -U -r "${new_env_path}";
python -m pip uninstall -y ninja >/dev/null 2>&1;
fi

# Keep excluded packages out of reused venvs as well as newly-created ones.
# shellcheck disable=SC1090
. "${HOME}/.local/share/venvs/${env_name}/bin/activate";
python -m pip uninstall -y "${excluded_packages[@]}" >/dev/null 2>&1;

cp -a "${new_env_path}" "${old_env_path}";
fi
}
Expand Down