Skip to main content

CUDA architecture detection

· 4 min read
Daniel Ching
Daniel Ching
Member of conda-forge/core

Some conda-forge CUDA packages, especially in machine learning, support only some CUDA architectures, but until now nothing stopped conda from installing them on an incompatible machine. The first sign of trouble was a run-time error like no kernel image is available for execution on the device, or even a crash. The new __cuda_arch virtual package and cuda-arch metapackage let conda prevent the mismatch at install time instead.

What are __cuda_arch and cuda-arch?​

conda already knows about your CUDA driver: the __cuda virtual package reports the latest CUDA Toolkit version that the available CUDA driver fully supports, and the cuda-version metapackage constrains it, so you never install CUDA libraries beyond what your driver can run. __cuda_arch and cuda-arch do the same for your CUDA devices.

  • __cuda_arch is a virtual package created by the nvidia-virtual-packages conda plugin. Its version, as detected by the plugin, is the minimum compute capability of all the CUDA devices on the machine.
  • cuda-arch is a metapackage on conda-forge, one version per compute capability. A package built with cuda-arch=6.0 is installable only where __cuda_arch>=6.0, which mirrors the forward compatibility of PTX.

Packages depend on the cuda-arch metapackage rather than on __cuda_arch directly, for the same reason they depend on cuda-version rather than __cuda: a metapackage can carry run_exports and can be pinned by users without setting an environment variable.

Using it​

If you use conda, install the plugin into your base environment:

conda install -n base -c conda-forge nvidia-virtual-packages
note

Other package managers, such as pixi and mamba, do not use this plugin and must implement __cuda_arch support separately. Check your package manager's release notes.

Once __cuda_arch is detected, conda will only select packages that your GPUs can run, and there is nothing else to configure. Unlike cuda-version, where a new driver lets you choose between, say, a CUDA 12 or CUDA 13 environment, cuda-arch offers no choice to make: your hardware decides. If you have several GPUs, the least capable one decides, so the result runs on all of them. If a package offers several variants, conda picks the most suitable one for your hardware.

If something goes wrong, such as conda reporting that __cuda_arch is missing, see the known issues.

Using it in a feedstock​

Maintainers who want to express the minimum supported CUDA architecture of their package can opt in by adding a cuda_arch_version variant variable to conda_build_config.yaml and cuda-arch {{ cuda_arch_version }} to their host and test requirements. conda-forge's CI sets CONDA_OVERRIDE_CUDA_ARCH from cuda_arch_version automatically.

cuda-arch may be used to split a package into variants by CUDA architecture. This means smaller targeted binaries for groups of compute capabilities instead of one large binary with code for all supported devices.

A few things to keep in mind:

  • Always include PTX for the highest architecture each variant targets. Only the minimum architecture on the system is reported, so a machine may also have newer devices that can run your binary only by JIT compiling PTX.
  • Leave no gaps. A variant installs on every device from its minimum architecture up to the next variant's minimum, so it must run on all of them. For example, a 5.0 variant without PTX plus a 7.0 variant leaves 6.x devices with a binary that cannot run.
  • Sub-architectures such as 100f cannot be expressed as cuda-arch versions, but you can still target them at build time.
  • More variants mean more CI jobs, so use this when the benefit justifies the cost.

The how-to guide has an example conda_build_config.yaml and recipe, in both meta.yaml and recipe.yaml form. The cudnn-feedstock and kokkos-feedstock are real examples.

Feedback​

Please report problems or ideas in the nvidia-virtual-packages repository.