Recently we've been able to add GPU-enabled TensorFlow builds to conda-forge! This was quite a journey, with multiple contributors trying different ways to convince the Bazel-based build system of TensorFlow to build CUDA-enabled packages. But we managed, and the pull request got merged.
As 2020 winds down, the Core team thought it'd be fun to review some of the big accomplishments our community has made this year.
The conda-forge community has grown immensely this year. Here are some
numbers to help give you an idea of the scale of our growth.
conda packages this
year, along with a corresponding number of new feedstocks.conda-forge channel on
anaconda.org exceeded 100 million downloads per month.conda-forge channel passed 2 billion total,
all-time downloads.conda-forge Core team and at least two members to the
staged-recipes team.conda-forge
GitHub organization.We've also shipped a ton of big updates to our core infrastructure this year. These updates include
PyPy support: We added support for PyPy 3.6 and now supply
one of the biggest stacks of PyPy-enabled packages in the PyPy
ecosystem.automerge label or through an opt-in setting
in the conda-forge.yml.R 4.0 migration: This migration was the first one to use our
automerge infrastructure at scale. With it, we completed a
complete rebuild/upgrade of the R ecosystem in about a week.Python updates: We deprecated Python 2.7, completed the
Python 3.8 migration, and got about 75% of the way through the
Python 3.9 migration.GCC 9 and clang 11.sysroot in preparation for the
CentOS 6 EOL. We hope to complete the move to CentOS 7 early next
year.miniconda-like
installers. These support a broad range of platforms, including
osx-arm64 and linux-aarch64.defaults channel by rebuilding the msys2 recipes.osx-arm64 platform. This platform is our first one to use a
fully cross-compiled infrastructure.We know that this year has been extremely difficult for so many of our
community members and that the fantastic success of conda-forge would
not have been possible without the active participation and support of
our community. Thank you everyone so much for the work you put into
conda-forge this year, making it the wonderful, community-led
resource that it is.
We wish everyone a happy, healthy, and peaceful new year!
Various members of the community have raised questions publicly and
privately about the implications of Anaconda's new Terms of Service
(TOS) on anaconda.com. First of all, we understand your concerns. We
would like to explain a bit how conda-forge works, how the TOS change
affects us and conda-forge users, and what our plans as a community
are for the future.
A new platform osx-arm64 has been added to the build matrix of
conda-forge. osx-arm64 packages are built to run on upcoming macOS
arm64 processors marketed as Apple Silicon. An installer for this
platform can be found
here.
tl;dr Depending on specific version numbers of underlying libraries may be too inaccurate and cause headaches as upstream libraries evolve and change. A more detailed approach is needed. In this post I outline current and potential work on a path towards a more complete inspection of requirements based on APIs and dynamic pinning of libraries.
Recently I've been thinking about operational risk (op. risk). Operational risks arise from failures of processes, for instance a missing email, or an automated software system not running properly. Many commercial institutions are interested in minimizing op. risk, since it is risk that produces no value, as opposed to risks associated with investing. This is also something I think about in my job at Lab49, where I'm a software engineering consultant focusing on financial institutions. I think there is also a good analogy for Conda-Forge, even though we are not a commercial outfit. In this case the risk we incur isn't the potential for lost earnings but frustration for our users and maintainers in the form of bugs and lackluster user experience. In this post I explore three main sources of operational risk for Conda-Forge: Automation, Top-Down Control, and Self-Service Structure.