Skip to main content

2020-10-21 conda-forge core meeting

Zoom link What time is the meeting in my time zone last weeks meeting


* Eric D
* Matt B
* Filipe
* Cheng Lee
* Isuru
* Marius
* Crystal
* Marcelo
* Paul I.
* Marcel B.
* Connor M.
* Bill Batey
* Stephanie
* John K.
* Lori B.
* Wolf
* CJ


Standing items

  • intros for new folks on the call

    • Bill - new to Anaconda. On the distribution team with Cheng and Connor.
  • (CJ) budget

    • current approvals?
    • Whenever updated numbers land, please screenshare and show the budget.
      • Link is in Keybase (numfocus_spreadsheets.txt)
  • (MRB) open votes

    • Chris Burr for Core
    • Won't use polys again because something totally went sideways. Maybe we'll try again?

From previous meeting(s)

  • (MRB) github releases and conda-forge-sparta

    • code and data in the regro github org right now
      • if you want access, LMK
      • I am also happy to move the repos to the conda-forge org but they may slow rerendering since they use a fair bit of github actions
    • how it works
      • all packages that are validated and copied to are also sent to a releases repo via a workflow dispatch event in github actions
      • the github actions CI job (regro/releases)
        • downloads the package
        • computes its repodata information
        • uploads the package and its repodata as a github release
        • pushes the repodata shard to another repo with all of the shards
      • another job on a continuous loop (regro/repodata aka the repoworker)
        • downloads the shards
        • figures out which ones are new
        • applies repodata patching to the shards (will also repatch everything if patches have changed)
        • computes the repodata and channel data
        • computes an index of links that point to where one can download the package
        • uploads all of this data as a github release
        • kicks the heroku server to download the new links
      • the herkou server (
        • gets a request for a package or repodata file at the standard URL for an anaconda server
        • looks up where that package is in the links index
        • issues a 300 to redirect the request to the actual location
      • a final set of cron jobs runs once an hour to sync our repodata shards with (regro/repodata-shards)
        • this job catches any missed releases due to bad CI jobs or manual uploads
          • will update the shards repo and upload to github
        • ensures eventual consistency on ~1-2 hour time scales if things break
    • overall time delay from the initial workflow dispatch to a package being available is 5-10 minutes
      • can go longer if repodata patches are being redone for everything
    • how could this not depend on anaconda if we needed it
      • replace cf-staging and w/ quetz
      • limit traffic to those instances to keep the costs down
      • send the vast majority of download requests to our (possibly beefed up) heroku server and thus github
    • I really don't want to maintain this for production, but we could if we had to.
    • For now, we have a system to help index and archive our data in another spot.
  • (MRB / ED / SC) Roadmap / Funding? Discussion in gitter:

    • (SC) What would CF do as a project if it received funding? Maybe having some kind of whitepapers on the conda-forge website would help. Would conda-forge do something very different if some corporation gave 10k, 100k, or 1M to the project via NumFOCUS?
      • 10k: does it buy enough hosting to do something interesting?
      • 100k - what does it bring.
      • A much larger amount: conda-forge may even be able to have funded people directly working on the project.
      • We can almost certainly find a lot useful stuff that would not hurt getting funding for without necessarily having to get consensus on a roadmap. (something i think would not be controversial, like oh we could use some funding to do some nice web development for better visualization of migrations, or stuff) there are probably a lor of things of that nature.
    • If we make a list, Santa may just show up...
    • see the bloomberg note above...
    • wishlist and ideas:
      • (ED) Can we add initials for whomever submitted the idea? would be useful to track that if there are follow-on questions
      • (MRB) Start chatting on this for 15 minutes in the next core meeting
  • (MvN) tactical things to do about cython?

    • CPython breaks its apis reguarly enough to force a recythoning of much of the python ecosystem.
    • Should we patch our cython so that when the CONDA_BUILD env var is set, we always recythonize and ignore mtime for files?
  • (MvN) noarch: python v2

    • As we increasingly drop compatibility with py27/35/36 should we automatically add python constraints to all noarch: python artifacts using repodata pacthing?
      • The idea would be to pick an epoch date (say 2020-10-01) and have the artifacts built after that date get a repodata patch adding a python>=3.6 constraint iff it only had a python constraint.
    • Additionally we may as well upate the noarch adding bot action that adds noarch to strip all lines containing py27/34/35 selectors from the recipe.
      • This should allow us to move lots of feedstocks that currently depend on some things only for py27 compat to noarch: python cutting down on the growth rate of artifacts.
    • to do
      • ( ) [ ] make a hint
      • ( ) [ ] make an announcement
      • ( ) [ ] make the hint a lint
  • (WV) multisheller

Your new() agenda items

  • (FF) Outreachy, round 2
  • (FF) "Feature parity" between feedstock and staged-recipes. For example: CFEP-05 does not contemplate a dev release on staged recipes when there is no stable release to start a feedstock.
    • No need for a conda-forge.yml, one can add a conda config file
    • The broader issue still is not solved, we cannot do cos7
    • staged recipes needs some work!

Pushed to next meeting

Active votes

Subteam updates









CI infrastructure

Compiler upgrade

CFEP updates

Open PRs

  • cfep-04 X11 and CDT policy

    • INACTIVE - Merge in with some inactive-esque status?
    • Needs new champion. Thanks for your work on this pkgw! Has unaddressed comments from pkgw as from Jan 10, 2020
  • cfep-06 Staged-recipes review lifecycle

    • INACTIVE - Merge in with some inactive-esque status?
    • Lingering comment from @saraedum. @jakirkham, can you reply? Has unadressed comment from @saraedum from Jan 8, 2020
    • (MRB) The stalebot has solved the worst of the issues here. I think we could defer this one permanently.
  • cfep-10 Feedstock statuses, unmaintained

    • INACTIVE - Merge in with some inactive-esque status?
    • Needs another review. Has unaddressed updates from pkgw as of Jan 11, 2020
  • cfep-12 Removing packages that violate the terms of the source package

    • Stalled since May 26, 2020
    • Active debate about moving to "broken" vs deleting from conda-forge channel
    • Active vote, ends on 2020-03-11
    • What were the results of the vote?
    • Did we hear back from NumFOCUS?
  • cfep-17 Handling pin backports and dependency rebuilds

    • Stalled debate about implementation details between Isuru, CJ and Matt
    • UPDATE 2020-07-22: We in principle have agreement to render the extra pinnings needed directly in the feedstock on a temporary basis (i.e., until the migration has ended).
  • cfep-20


Check in on previous action items

Copy previous action items from last meeting agenda.

This meeting


  • Python 2.7 migration
    • ( ) [ ] make a hint
    • ( ) [ ] make an announcement
    • ( ) [ ] make the hint a lint

Last meeting


  • Make sure to add the NVBug info to the cudatoolkit package that conda-forge makes (if we make one)

2 meetings ago


Move to Issue Tracker


  • (MRB)
    • do libgfortran name change
    • add target platform to hashes
    • do gfortran migration with bot
    • bump pinnings


  • Get a call set up with Jon Mease about the kaleido staged recipes PR
    • Emailed on 2020-09-16
  • (FF) Open up a PR on the python feedstock for python 3.9 and see what fails


2020-08-26 Docker hub

  • (JK) Check in on Azure build workers to see if they have the docker hub limitation.
  • (JK) work with dockerhub to see if we can get OSS status
    • Check in again at some point. We haven't heard back as of 2020-09-23
  • (MRB) start pushing images to quay (


  • (???) build webpage to credit them (and others)

  • If we're adding a logo, will want to make sure that we have permission to use it.

  • Shout-out on twitter at some point. "Thanks forOVHCloud for providing a VM", etc. (maybe after we ship qt on windows with it?)

  • Figure out how to communicate breaking changes to users. Likely should open up an issue immediately for futher discussion. Ping @kkraus, plus capture notes from further up in these meeting notes

  • John K. will update the cuda toolkit feedstock on the git repo to note the NVBug link to the internal NVIDIA issue tracker

  • Jonathan will update docs to note that some non-exhaustive list of packages (like cuda-toolkit, MKL, etc.)

  • Jonathan will review this PR

  • (Kale) schedule conda working group

  • cfep-10 next steps: CJ to call a vote for feedback

  • cfep-06 next steps: Ask staged recipes team to champion this CFEP and move it forward

  • jakirkham & CJ-wright to sync on adding CUDA to the migration bot

  • (Eric) Scheduling Anaconda <-> conda-forge sync on requirements gathering

    • Will try and get this scheduled in the next month.
  • (Anthony) Reach out to NumFocus to figure out legal ramifications of not including licenses in files.

  • (Eric) check internally for funding levels for hotels & flying folks from the community in?

  • (Eric) Figure out finances of conda-forge to support themselves?

  • (jjhelmus) Open up CFEP for which python's we're going to support

  • (jakirkham) write a blog post on CUDA stuff we discussed today

  • (jakirkham) update docs on how to add CUDA support to feedstocks

  • (jakirkham) will open an issue on conda-smithy to investigate Drone issues. (ping the aarch team)

  • (ED) Who we are page? Some combination of a FAQ and a who is everyone. FAQ things like:

    • who's the POC for CF <> Anaconda, CF <> NumFocus, CF <> Azure
    • who's the POC for the various subteams?
    • Informal information: roles, day jobs, bios, the whole nine yards, why you're here, etc.
    • Public or internal? I don't really care either way. Anyone feel strongly one way or the other?
    • opt-in to public bios
    • software carpentry has a large number of instructors and has
    • some concern about "yet another place to keep stuff up to date"
  • (ED) document strategies for reproducible environments using conda-forge

  • (UK) Static libraries stuff

    • Add linting hints to builds to find them
    • Recommend how to package them -> CFEP-18
    • We should write docs saying we don't provide support and this is a bad idea. -> CFEP-18