Recently we've been able to add GPU-enabled TensorFlow builds to conda-forge! This was quite a journey, with multiple contributors trying different ways to convince the Bazel-based build system of TensorFlow to build CUDA-enabled packages. But we managed, and the pull request got merged.
As 2020 winds down, the Core team thought it'd be fun to review some of the big accomplishments our community has made this year.
conda-forge community has grown immensely this year. Here are some
numbers to help give you an idea of the scale of our growth.
- The community has added 3,751 new, unique
condapackages this year, along with a corresponding number of new feedstocks.
- For the majority of 2020, the
anaconda.orgexceeded 100 million downloads per month.
- In July of 2020, the
conda-forgechannel passed 2 billion total, all-time downloads.
- We've grown our core developer community, adding seven new members
conda-forgeCore team and at least two members to the
- We now have over 2,500 recipe maintainers in the
Big New Features
We've also shipped a ton of big updates to our core infrastructure this year. These updates include
PyPysupport: We added support for
PyPy3.6 and now supply one of the biggest stacks of
PyPy-enabled packages in the
- automerge: We now support the automatic merging of PRs on
feedstocks using the
automergelabel or through an opt-in setting in the
R4.0 migration: This migration was the first one to use our
automergeinfrastructure at scale. With it, we completed a complete rebuild/upgrade of the
Recosystem in about a week.
Pythonupdates: We deprecated
Python2.7, completed the
Python3.8 migration, and got about 75% of the way through the
- compiler upgrades: We upgraded our compiler infrastructure to
- CentOS 7 and CentOS 6 EOL: We shipped an option to enable our
compilers to use the CentOS 7
sysrootin preparation for the CentOS 6 EOL. We hope to complete the move to CentOS 7 early next year.
- miniforge: We built our own standalone,
miniconda-like installers. These support a broad range of platforms, including
- standalone Windows stack: We fully decoupled our Windows recipes
defaultschannel by rebuilding the
- Apple silicon support: We added support for Apple silicon with
osx-arm64platform. This platform is our first one to use a fully cross-compiled infrastructure.
- CUDA support: We added support for building CUDA packages on windows and added CUDA 11.0 support.
We know that this year has been extremely difficult for so many of our
community members and that the fantastic success of
not have been possible without the active participation and support of
our community. Thank you everyone so much for the work you put into
conda-forge this year, making it the wonderful, community-led
resource that it is.
We wish everyone a happy, healthy, and peaceful new year!
Various members of the community have raised questions publicly and
privately about the implications of Anaconda's new Terms of Service
anaconda.com. First of all, we understand your concerns. We
would like to explain a bit how
conda-forge works, how the TOS change
affects us and
conda-forge users, and what our plans as a community
are for the future.
A new platform
osx-arm64 has been added to the build matrix of
osx-arm64 packages are built to run on upcoming macOS
arm64 processors marketed as
Apple Silicon. An installer for this
platform can be found
tl;dr Depending on specific version numbers of underlying libraries may be too inaccurate and cause headaches as upstream libraries evolve and change. A more detailed approach is needed. In this post I outline current and potential work on a path towards a more complete inspection of requirements based on APIs and dynamic pinning of libraries.
Recently I've been thinking about operational risk (op. risk). Operational risks arise from failures of processes, for instance a missing email, or an automated software system not running properly. Many commercial institutions are interested in minimizing op. risk, since it is risk that produces no value, as opposed to risks associated with investing. This is also something I think about in my job at Lab49, where I'm a software engineering consultant focusing on financial institutions. I think there is also a good analogy for Conda-Forge, even though we are not a commercial outfit. In this case the risk we incur isn't the potential for lost earnings but frustration for our users and maintainers in the form of bugs and lackluster user experience. In this post I explore three main sources of operational risk for Conda-Forge: Automation, Top-Down Control, and Self-Service Structure.