2020-08-26 conda-forge core meeting

Zoom link What time is the meeting in my time zone last weeks meeting

Attendees

Agenda

Standing items

  • [x] intros for new folks on the call

  • [x] (CJ) budget

    • current approvals are all up to date.

    • screenshare and show the doc monthly?

Your new agenda items

stuff from last time

  • [x] (MRB) shall we merge this (https://github.com/conda-forge/conda-forge.github.io/pull/1139)?

    • PR is on supported python versions

    • waiting on isuru

  • [x] (ED) Sylvain: Any updates from OVH on the windows VM?

    • TODO: keep checking in on this

    • OVH will want their brand on our page somewhere.

    • TODO

      • [ ] (???) build webpage to credit them (and others)

      • If we’re adding a logo, will want to make sure that we have permission to use it.

      • Shout-out on twitter at some point. “Thanks forOVHCloud for providing a VM”, etc. (maybe after we ship qt on windows with it?)

  • [x] (AS) qgpu - GPU build agents.

    • Drone or Azure? Drone is a simple go executable and you can run it in docker. Azure build agent is heavy weight?

    • Pick one and go

    • Waiting on Anthony to have some spare time.

    • JH: Github actions has a runner

    • AS: Happy to try any and all solutions

    • MRB: Rerendering can’t push changes to workflow files in feedstocks. There’s not a feasible workaround for this atm. Currently working with github on trying to fix this, but it’s not going quickly.

new stuff

  • [x] (MRB, John) docker hub changes

    • by default they will limit pulls of our images to a few hundred per hour for anonymous users

      • https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/

      • HN: https://news.ycombinator.com/item?id=24262757

      • https://news.ycombinator.com/item?id=24268336

      Github Docker Registry is a mess and should be avoided at all costs.

      1. It is broken and unusable on Kubernetes and Docker Swarm.

      2. It is flaky often returning 500 type errors.

      3. It is expensive as the amount of pull bandwidth is very limited.

    • this is based on the user pulling the image, not the user hosting the image

    • John put in an application for their open source program. Once we are accepted, they will work with us hopefully.

      • (JK) Check in on Azure build workers and see if they have the docker hub limitation

    • For now, we should start trying to host our images elsewhere

      • Isuru suggested using github packages

      • we have used quay in the past

    • going to take a many pronged approach

      • [ ] (JK) Check in on Azure build workers and see if they have the docker hub limitation

      • [ ] (JK) work with dockerhub

      • [ ] (???) start pushing images to quay (github?)

      • [ ] (???) put changes in smithy to make sure we can use those other image locations

  • [x] (MRB) AWS stuff submitted

    • we asked for 2k total from NumFocus through their pool of credits.

    • working through back and forth…

    • NumFocus is going to put our AWS account under theirs

  • [x] (KK) Conda virtual packages plugin

    • From the NVIDIA / RAPIDS side, desire to build more conda packages on things that depend on drivers with kernel modules, i.e. MOFED

      • (KK) MOFED is basically infiniband drivers. This is not the only case for this

    • Work to do on conda side to build plugin architecture to support this

      • https://github.com/conda/conda/issues/10131 (currently slated for 5.0.0 release)

    • Could potentially explode build matrices for certain projects, what is our stance from the conda-forge side?

Active votes

Subteam updates

Bot

ARM

POWER

CUDA

Docs

staged-recipes

website

security+systems

CI infrastructure

Compiler upgrade

CFEP updates

Open PRs

  • cfep-04 X11 and CDT policy

    • INACTIVE - Merge in with some inactive-esque status?

    • Needs new champion. Thanks for your work on this pkgw! Has unaddressed comments from pkgw as from Jan 10, 2020

  • cfep-06 Staged-recipes review lifecycle

    • INACTIVE - Merge in with some inactive-esque status?

    • Lingering comment from @saraedum. @jakirkham, can you reply? Has unadressed comment from @saraedum from Jan 8, 2020

    • (MRB) The stalebot has solved the worst of the issues here. I think we could defer this one permanently.

  • cfep-10 Feedstock statuses, unmaintained

    • INACTIVE - Merge in with some inactive-esque status?

    • Needs another review. Has unaddressed updates from pkgw as of Jan 11, 2020

  • cfep-12 Removing packages that violate the terms of the source package

    • Stalled since May 26, 2020

    • Active debate about moving to “broken” vs deleting from conda-forge channel

    • Active vote, ends on 2020-03-11

    • What were the results of the vote?

    • Did we hear back from NumFOCUS?

  • cfep-17 Handling pin backports and dependency rebuilds

    • Stalled debate about implementation details between Isuru, CJ and Matt

    • UPDATE 2020-07-22: We in principle have agreement to render the extra pinnings needed directly in the feedstock on a temporary basis (i.e., until the migration has ended).

Discussion

Check in on previous action items

Copy previous action items from last meeting agenda.

This meeting

Docker hub

  • [ ] (JK) Check in on Azure build workers to see if they have the docker hub limitation. Maybe Azure and docker hub

  • [ ] (JK) Check in on Azure build workers and see if they have the docker hub limitation

  • [ ] (JK) work with dockerhub to see if we can get OSS status

  • [ ] (???) start pushing images to quay (github?)

  • [ ] (???) put changes in smithy to make sure we can use those other image locations

OVH

  • [ ] (???) build webpage to credit them (and others)

  • [ ] If we’re adding a logo, will want to make sure that we have permission to use it.

  • [ ] Shout-out on twitter at some point. “Thanks forOVHCloud for providing a VM”, etc. (maybe after we ship qt on windows with it?)

Last meeting

2 meetings ago

  • [ ] Figure out how to communicate breaking changes to users. Likely should open up an issue immediately for futher discussion. Ping @kkraus, plus capture notes from further up in these meeting notes

  • [x] (Eric) TODO: Make strict an option in conda_forge.yaml and turn it on by default. Open issue in conda-smithy

3 meetings ago

  • [x] Eric to add a new page to our docs around how to engage with conda-forge and affiliated in a commercial relationship.

  • [x] Eric will get the NVBug link from Keith and archive it in the conda-forge google drive.

  • [ ] John K. will update the cuda toolkit feedstock on the git repo to note the NVBug link to the internal NVIDIA issue tracker

  • [ ] Jonathan will update docs to note that some non-exhaustive list of packages (like cuda-toolkit, MKL, etc.)

  • [ ] Jonathan will review this PR

Move to Issue Tracker

  • [ ] (Kale) schedule conda working group

  • [ ] cfep-10 next steps: CJ to call a vote for feedback

  • [ ] cfep-06 next steps: Ask staged recipes team to champion this CFEP and move it forward

  • [ ] jakirkham & CJ-wright to sync on adding CUDA to the migration bot

  • [ ] (Eric) Scheduling Anaconda <-> conda-forge sync on anaconda.org requirements gathering

    • Will try and get this scheduled in the next month.

  • [ ] (Anthony) Reach out to NumFocus to figure out legal ramifications of not including licenses in files.

  • [ ] (Eric) check internally for funding levels for hotels & flying folks from the community in?

  • [ ] (Eric) Figure out finances of conda-forge to support themselves?

  • [ ] (jjhelmus) Open up CFEP for which python’s we’re going to support

  • [ ] (jakirkham) write a blog post on CUDA stuff we discussed today

  • [ ] (jakirkham) update docs on how to add CUDA support to feedstocks

  • [ ] (jakirkham) will open an issue on conda-smithy to investigate Drone issues. (ping the aarch team)

    • https://github.com/conda-forge/conda-forge.github.io/issues/954

  • [ ] (ED) Who we are page? Some combination of a FAQ and a who is everyone. FAQ things like:

    • who’s the POC for CF <> Anaconda, CF <> NumFocus, CF <> Azure

    • who’s the POC for the various subteams?

    • Informal information: roles, day jobs, bios, the whole nine yards, why you’re here, etc.

    • Public or internal? I don’t really care either way. Anyone feel strongly one way or the other?

    • opt-in to public bios

    • software carpentry has a large number of instructors and has https://carpentries.org/instructors

    • some concern about “yet another place to keep stuff up to date”

  • [ ] (CJ) Form finance subteam

  • [ ] (ED) document strategies for reproducible environments using conda-forge

  • [ ] (UK) Static libraries stuff

    • [ ] Add linting hints to builds to find them

    • [x] Recommend how to package them -> CFEP-18

    • [x] We should write docs saying we don’t provide support and this is a bad idea. -> CFEP-18