macOS ARM builds on conda-forge¶
A new platform osx-arm64
has been added to the build matrix of conda-forge.
osx-arm64
packages are built to run on upcoming macOS arm64 processors marketed
as Apple Silicon
. An installer for this platform can be found
here.
This will install a conda environment with python and conda in it. Installed conda
will be able to install packages like numpy, scipy
. Currently there are about
100 packages out of 10000 packages pre-built for this platform.
All these packages are built on conda-forge’s current macOS x86_64
infrastructure.
In order to do so, we have made lots of changes to the infrastructure including,
conda, conda-build, conda-smithy, constructor, conda-forge-ci-setup
to facilitate
cross-compiling which is the process of compiling a package that will run on a
host
platform (osx-arm64
in our case), with the compilation done on a
build
platform (osx-64
or linux-64
in our case).
osx-arm64
is the first conda platform that is completely
bootstrapped using conda-build’s cross-compiling facility. Previously, when adding
a new platform, conda-build was built with an existing python and pip environment
on the new platform. With cross-compiling, when the compilers and a sysroot is set up
on a different platform, an existing conda-build installation (on osx-64
and linux-64
in this case) will be able to start building packages right away.
Cross-compiling builds for osx-arm64
¶
In order to cross compile packages for osx-arm64
we need compilers. So, we first
built clang=11.0.0.rc1
which has support for targetting osx-arm64
. We also
built compiler-rt=11.0.0.rc1
as a universal build support both osx-64
and
osx-arm64
.
Linker, archiver, otool
, install_name_tool
was built using the
cctools-port project by Thomas Pöchtrager.
One issue we ran into was that the macOS 11.0 Big Sur Beta 7 required that all
executables and shared libraries be ad-hoc signed which is signing without a
verified signature. On suggestion of cctools-port
developer we added support
to cctools-port
to sign these executables using ldid
which can be used
on Linux as well as macOS to sign.
Using these, the first cross compiled package we built was libcxx
to facilitate
C++ builds. For the osx-arm64
sysroot we used the MacOSX11.0.sdk
already
installed on Azure pipelines and Travis-CI. Due to licensing issues, we cannot
distribute this, but it can be downloaded from the Apple developer website
even on Linux.
With clang we have a C/C++ compiler, but lack a Fortran compiler.
We used the GCC fork for darwin-arm64.
First, a cross compiler (build == host != target
) was built. Using that compiler,
we built a cross-native
compiler (build != host == target
) which gave use the
shared libraries like libgfortran.dylib
.
We also added support for cross compiling rust programs to the rust packages
in conda and installing rust_osx-64
on Linux will give you a compiler that will
build packages for osx-64
.
As we haven’t done cross-compilation before, many packages needed to be updated.
Most were trivial changes that we automated later on. These included getting a newer
config.sub
to identify the new autotools platform arm64-apple-darwin20.0.0
,
adding options to CMake with the environment variable CMAKE_ARGS
to correctly
set up the toolchain and recipes were update to use cmake ${CMAKE_ARGS} ..
.
Running tests when building were also disabled by guarding commands like
make check
, make test
, ctest
with the env variable
CONDA_BUILD_CROSS_COMPILATION
.
Cross-compiling python extensions is quite tricky as distutils
is not really
setup to do this. Thanks to the project crossenv
this is unofficially supported with a few quirks. With crossenv
, we can run a
python on the build machine (osx-64
or linux-64
in this case) that acts like
it is on osx-arm64
. crossenv
monkey-patches a few functions like
os.uname
and sets up values like _PYTHON_SYSCONFIG_DATA
to make python
running on osx-64
or linux-64
behave like osx-arm64
. One issue is that,
monkey-patching sys.platform
doesn’t work and therefore if a python package
in it’s setup.py
uses sys.platform
to differentiate OSes this will lead
to unintended consequences if you are cross-compiling from linux-64
.
Therefore, we have to use osx-64
as our build
system when cross-compiling
for osx-arm64
. Note that packages using sysconfig.get_platform()
will get
the correct platform.
For creating an installer for conda, we needed a standalone conda executable
to bootstrap the conda environment. For other platforms we relied on
conda-standalone
which is a standalone conda executable created using
pyinstaller
. Since pyinstaller
does not support cross-compile, we
decided to use micromamba
as the bootstrapper and added features to
micromamba
so that it can function as the bootstrapper.
How to add a osx-arm64
build to a feedstock¶
All the below changes will be done by a bot and the packages the bot will send PRs to is determined by the list of packages at conda-forge-pinning and their dependences. If you would like to add support, please send a PR adding the feedstock name to the above list. After that PR is merged, you can monitor the status at conda-forge status-page and if a particular PR is stalled you can send a PR to the feedstock to fix it.
Following instructions are for when you want to add support manually.
Add the following to conda-forge.yml
(on Linux or OSX),
build_platform:
osx_arm64: osx_64
test: native_and_emulated
You can rerender using,
conda smithy rerender
For python packages, add one or more of the following to recipe/meta.yaml
as needed, noting that you must only add numpy, cython, and/or pybind11 if they are used in host: as well,
requirements:
build:
- python # [build_platform != target_platform]
- cross-python_{{ target_platform }} # [build_platform != target_platform]
- cython # [build_platform != target_platform]
- numpy # [build_platform != target_platform]
- pybind11 # [build_platform != target_platform]
For autotools package, add the following to recipe/meta.yaml
,
requirements:
build:
- gnuconfig # [unix]
and to recipe/build.sh
,
# Get an updated config.sub and config.guess
cp $BUILD_PREFIX/share/gnuconfig/config.* .
For cmake packages, add the following to recipe/build.sh
,
cmake ${CMAKE_ARGS} ..
For meson
packages, add the following to recipe/build.sh
,
meson ${MESON_ARGS} builddir/
Note
Conda automatically creates a cross build definition file
when cross-compiling, and adds
the necessary argument to ${MESON_ARGS}
to point meson
to that file.
${MESON_ARGS}
is only defined when cross-compiling, not for normal builds.
For rust packages, add the following to recipe/meta.yaml
,
requirements:
build:
- {{ compiler('rust') }}
If there’s a line like make check
in recipe/build.sh
that cannot be
run when cross-compiling, do the following,
if [[ "$CONDA_BUILD_CROSS_COMPILATION" != "1" ]]; then
make check
fi
After these changes, another rerendering might be required.
- Some useful jinja variables,
build_platform
- conda subdir forBUILD_PREFIX
. eg:linux-64
target_platform
- conda subdir forPREFIX
. eg:osx-arm64
- Some useful environment variables,
build_platform
target_platform
CONDA_BUILD_CROSS_COMPILATION
- 1 if cross compilingCMAKE_ARGS
- arguments to pass to cmakeCC_FOR_BUILD
- C compiler for build platformCXX_FOR_BUILD
- C++ compiler for build platformHOST
- a triplet for host passed to autoconf. eg:arm64-apple-darwin20.0.0
BUILD
- a triplet for build passed to autoconf. eg:x86_64-conda-linux-gnu
- Some useful configure options in
conda-forge.yml
build_platform
- a dictionary mappingbuild
subdir tohost
subdir. eg:build_platform: osx_arm64: osx_64 linux_ppc64le: linux_64 linux_aarch64: linux_64
test_on_native_only
- a boolean to turn off testing on cross compiling. If the tests don’t require emulation (for eg: check that a file exists), thentest_on_native_only: false
will run the tests even when cross compiling.
Building locally¶
For building locally add the following in $HOME/conda_build_config.yaml
.
SDKROOT:
- /path/to/MacOSX11.0.sdk
After that, look for the config you want to run in .ci_support
folder
in the root of the feedstock
For eg: .ci_support/osx_arm64_.yaml
. Then run,
conda build recipe -m .ci_support/osx_arm64_.yaml -c conda-forge -c conda-forge/label/rust_dev
This should start a new build for osx-arm64
.
Testing packages¶
In order to test packages intended to run on future Apple Silicon hardware, Apple provides a machine called Developer Transition Kit (DTK). Jonathan Helmus and Eli Rykoff has helped with testing these packages on DTKs. Thanks to Eli Rykoff, we are now running tests for these packages as a daily cron job which has led to finding several bugs in our cross compiling infrastructure and also bugs in our recipes.
To test cross compiled recipes, transfer the built conda package to the host
and run,
conda build --test /path/to/package -c conda-forge
This work would not have been possible without the help of many people
including the upstream maintainers of compiler infrastructure
(which includes conda, conda-build, cctools, tapi, cctools-port,
ldid, llvm, clang, compiler-rt, openmp, libcxx, crossenv, rust, gcc-darwin-arm64),
conda-forge/help-osx-arm64
team including Matt Becker, Eli Rykoff and Uwe Korn who
sent PRs to fix recipes, conda-forge/bot
team and also all the conda-forge maintainers
of the 100 feedstocks who reviewed and fixed PRs.
Isuru Fernando