conda-forge core meeting 2022-11-16

Add new agenda items under the Your __new__() agenda items heading

last weeks meeting What time is the meeting in my time zone Meeting info:

  • To join the video meeting, click this link: https://zoom.us/j/9138593505?pwd=SWh3dE1IK05LV01Qa0FJZ1ZpMzJLZz09

  • Otherwise, to join by phone, dial +1 347-384-8597 and enter this PIN: 828 997 153#

  • To view more phone numbers, click this link: https://tel.meet/ijv-qsvm-tvn?hs=5

Attendees

Name

Initials

GitHub ID

Affiliation

Jaime Rodríguez-Guerra

JRG

jaimergp

Quansight/cf

Eric Dill

EDD

ericdill

cf

Daniel Ching

DJC

carterbox

Argonne

Marcel Bargull

MB

mbargull

Bioconda/cf

Matthew R Becker

MRB

beckermr

cf

Dave Clements

DPC

tnabtaf

Anaconda

Cheng H. Lee

CHL

chenghlee

Anaconda/cf

Jannis Leidel

JL

jezdez

Anaconda/cf

Katherine Kinnaman

KK

kathatherine

Anaconda

John Kirkham

JK

jakirkham

NVIDIA/cf

Travis Hathaway

TH

travishathaway

Anaconda :snake:

Marius van Niekerk

MvN

mariusvniekerk

cf

Wolf Vollprecht

WV

wolfv

prefix.dev / cf

Amit Kumar

AK

aktech

Quansight / cirun

18 people total

Standing items

  • [X] intros for new folks on the call

    • Amit Kumar - helping setup GPU server for CF. Jaime pulled me in.

    • Kyle - director of engineering for package building teams at anaconda

  • [X] open votes

    • [x] (JRG) Python 3.10.7 and friends https://github.com/conda-forge/python-feedstock/pull/579#issuecomment-1291847544

      • will circle back to this at the end of the call

      • Action item: Jaime to write announcement

    • [x] (JRG) Status on staged-recipes application(s)?

      • [ ] https://github.com/orgs/conda-forge/teams/core/discussions/12 + another member

      • Action item: FF to take over

From previous meeting(s)

Your new() agenda items

  • [x] (JK) CUDA EULA Amendment

  • [x] (JK) Python debug build ( https://github.com/conda-forge/python-feedstock/pull/597 )

  • [x] (WV) Announcing prefix.dev

    • Issue tracker for ideas: https://github.com/prefix-dev/prefix-dev

  • [x] (WV) Planning for PackagingCon 2023 is (re)-starting (right after this meeting)

  • [x] (WV) Got CZI grant for improving conda-forge infra

    • [ ] https://conda-forge.org/docs/orga/funding/00_intro.html (see submitted proposal here)

  • [x] (MRB) moving to .conda is ready to ship

    • See here: https://github.com/conda-forge/conda-forge.github.io/issues/1586

    • implementation is in the ci-setup package

      • it looks for a key in the conda-forge.yml at build time and if that key exists and has the right value (conda_pkg_format: 2), then it adjusts the condarc file

    • note this is not a statically set thing in conda smithy since conda build doesn’t appear to have a command line switch controlling the package format it outputs

      • Not at this time; code is (partially?) there, but we should probably resurrect this CEP to formalize this feature.

    • I’ve tested the following for .conda packages

      • packages can be uploaded through the webserver, cf-staging, etc.

      • packages can be marked broken and are removed from the repodata

      • packages be marked not broken and are added back to the repodata

      • packages can be patched and the patches appear in the repodata

      • packages are properly picked up and indexed by libcfgraph

    • Note these checks went beyond simply merging the PRs associated with these items after they passed. I actually went into the repodata/graphdata and/or tried to make environments live and checked they did not work or had the correct (possibly modified) deps.

    • The roll out would work as follows

      • make an announcement

      • setup an admin migration to adjust the conda-forge.yaml for every feedstock slowly

      • wait till all feedstocks are adjusted and we see no major breakages

      • adjust default in ci setup package to be .conda

    • isuru:

      • could remove the tarbz2 in repodata

    • TODO

      • [ ] make sure on announcement you mention the minimum conda version (4.7)

      • [ ] check that ci services do not do duplicate uploads

      • [ ] set compression level for big packages

  • [x] (JK) Graal Python

    • https://github.com/oracle/graalpython/issues/254

    • https://github.com/conda-forge/staged-recipes/pull/20882

    • https://github.com/conda-forge/python_abi-feedstock/pull/19

  • [ ] (JRG/AK) GPU Server updates

    • gpu server is hosted by metrostar. uses github actions. whenever a job is triggered, cirun triggers job on openstack. can run 3 parallel jobs with 1 gpu v100. it is ready to run. there’s an example here https://github.com/aktech/cirun-openstack-example

    • JK: how do we plug this into a feedstock? do we need a test feedstock to work with?

    • JR: needs a bit of work on conda-smithy to get this template for cirun. need to figure out a policy for opting-in to the gpu service. shouldn’t be that publicly available at first. need to figure out how we face the load.

    • MRB: How do we affect which repositories have access (at org level)?

    • AK: Can add public API for configuring access

    • test feedstock: https://github.com/conda-forge/cf-autotick-bot-test-package-feedstock

  • [x] (FF) Staged-recipes reviewers vote results. We had 33 people invited to vote, 25 core, 8 emeritus. only 14 voted and they all seem like active core. - @timsnyder: passed - @mfansler: passed

  • [ ] DJC: What is policy for publishing forks of packages?     - [ ] Two pull requests on staged-recipes for forks of a presumably abandoned package; both want to keep the name of the upstream package     - [ ] https://github.com/orgs/conda-forge/teams/staged-recipes/discussions/8     - [ ] https://github.com/conda-forge/staged-recipes/pull/20669     - [ ] https://github.com/conda-forge/staged-recipes/pull/17965     - Marcel: complicated topic. we shuold talk more in-depth about this in an async issue     - will the packages clobber each other if they’re both installed?     guidance:     - must have: dont clobber existing code (run_constrained example)     - options: - rebrand your fork - inject your github user name (where the fork lives) into the packge / feedstock name - talk to upstream and take over the fork / feedstock

    • TODO: adjust PR template to address fork naming

requirements:
  run_constrained:
    sdsl-lite <0.a0

  • [x] (MB) Adding a Perl package maintainers team

Pushed to next meeting

  • results of staged-recipes reviewer vote

    • We had 33 people invited to vote, 25 core, 8 emeritus. only 14 voted and they all seem like active core.

  • [ ] (JRG) Support for abi3 Python packages with abi3audit checks?

    • https://blog.trailofbits.com/2022/11/15/python-wheels-abi-abi3audit/

  • [ ] (DPC) Should we shut down conda-forge@googlegroups.com at the same time we shut the conda google group mailing list (which is the end of the year)?

Active votes

  • staged-recipes reviewers vote is live, i think?

  • cfep-21 license packages for static/header libraries

    • Opened July 25, 2022 ends August 2, 2022

CFEPs

  • cfep-12 Removing packages that violate the terms of the source package

    • Stalled since May 26, 2020

    • Active debate about moving to “broken” vs deleting from conda-forge channel

    • Active vote, ends on 2020-03-11

    • What were the results of the vote?

    • Did we hear back from NumFOCUS? they did the legal seminar which is recorded

TODOs

2022-06-15

  • [ ] (MvN) vs2022 compiler support

    • Initial stab at some pieces https://github.com/conda-forge/vc-feedstock/pull/46

    • This is a new VC version

      • Uses existing UCRT

    • Would need to use the windows-2022 github actions image

    • TODO: follow on to change the name of the runtime to vc_runtime

    • TODO: will need patches for msys epoch for UCRT

    • (CHL) For reference: msys2 environments - pick your own compiler + libstdc++ + Windows runtime adventure

    • https://www.msys2.org/docs/package-naming/

    • (RG) “we need some version of mingw-w64 for SciPy 1.9.0 too, MSVC + gfortran will no longer work. we use an Rtools toolchain in CI now”

  • [ ] (CHL) How does conda-forge use channeldata.json?

    • Only known use by c-f: counting # of package namespaces for “By the numbers” on the webpage; other uses deprecated

    • TODO: CEP to deprecate & remove --use-channeldata from conda-build

    • TODO: CEP to add run exports to repodata.json

2022-04-05

  • TODO: Articulate suggested mechanism for “so you’re thinking about taking a contract to work on conda-forge”. tl;dr is you should work on a design doc with the impacted parties (or working groups, or interested folks) on the conda-forge side. Once you have agreement on the scope, delivery timeline, and anything that’s needed from the conda-forge volunteer side, go ahead and sign that contract with your payer. if you don’t do this first, you may encounter some difficulties finishing delivery of your contractual agreement with your payer.

2022-01-12

  • review Qt PR after logs are uploaded

  • Add cupython and cuquantum to don’t mirror list (MRB done)

  • re: std=c++14, Wait for Kai to comment and merge the PR

  • CJ and Jaime coordinate to Let Maxiconda know that we can’t use their logo 2021-12-01

  • WV: Set up meet-and-greet call with homebrew team?

  • MRB: (repodata patches) make a cron job that runs show_diff.py and posts an issue + commit if it is non-empty

2021-11-03

  • Self-hosting CI TODOs:

    • Change URL from drone.conda-forge.org. Proposals:

      • woodpecker.conda-forge.org

      • ci.conda-forge.org

    • Set up monitoring

      • Quantstack is setting up grafana for the mirror

2021-10-18

2021-09-22

  • [x] (WV): TensorFlow-GPU ready to go, just need to decide if GPU should get prio over CPU?!

    • GPU gets prio

2020-11-18

  • [ ] (IF/MRB/MV) intel oneAPI

    • todo

      • [ ] (Nikolay) licensing for opencl_rt

      • [ ] (Nikolay) intelmpi ABI compat w/ mpich

      • [ ] (MRB/IF) figure out how exactly to package C/C++ compilers

      • [ ] (MRB/IF) think about fortran ABI

      • [x] (MRB) make conda-forge compilers room (add people including keith)

  • [ ] (MB) asking core members to move to “emeritus” status

    • [ ] TODO: Eric to set up quarterly check-in for all core members to see if they’re interested in remaining “active” or if they want to move to emeritus

      • Remove emeritus folks from having access to various credentials (api tokens, twitter password, etc.)? This would require a change to the governance doc.

2020-11-11

  • TODO: Think about bringing in JOSS to provide context around how we might best write papers

2020-11-03

  • TODO: Check on Forrest Watters permissions for core

2020-10-28 2020-10-21

  • [ ] (Marius?) Python 2.7 migration

    • ( ) [ ] make a hint

    • ( ) [ ] make an announcement

    • ( ) [ ] make the hint a lint

2020-10-07

  • [ ] Make sure to add the NVBug info to the cudatoolkit package that conda-forge makes (if we make one)

2020-09-09

  • [ ] (ED) Update governance docs with similar voting model as what got put into conda-tools (+3 with no -1 is a pass)

  • [ ] (SC) Write jinja template to turn institutional partners yaml into a website https://github.com/conda-forge/conda-forge.github.io/blob/master/src/inst_partners.yaml

  • [ ] (SC) Document what needs to be done to create an OVH account and get access

2020-08-26 Docker hub

  • [ ] (JK) Check in on Azure build workers to see if they have the docker hub limitation.

  • [ ] (JK) work with dockerhub to see if we can get OSS status

    • [ ] Check in again at some point. We haven’t heard back as of 2020-09-23

OVH

  • [ ] Shout-out on twitter at some point. “Thanks forOVHCloud for providing a VM”, etc. (maybe after we ship qt on windows with it?)

  • [ ] Figure out how to communicate breaking changes to users. Likely should open up an issue immediately for futher discussion. Ping @kkraus, plus capture notes from further up in these meeting notes

  • [ ] John K. will update the cuda toolkit feedstock on the git repo to note the NVBug link to the internal NVIDIA issue tracker

  • [ ] Jonathan will update docs to note that some non-exhaustive list of packages (like cuda-toolkit, MKL, etc.)

  • [ ] Jonathan will review this PR

  • [ ] (Kale) schedule conda working group

  • [ ] cfep-10 next steps: CJ to call a vote for feedback

  • [ ] cfep-06 next steps: Ask staged recipes team to champion this CFEP and move it forward

  • [ ] jakirkham & CJ-wright to sync on adding CUDA to the migration bot

  • [ ] (Eric) Scheduling Anaconda <-> conda-forge sync on anaconda.org requirements gathering

    • Will try and get this scheduled in the next month.

  • [ ] (Anthony) Reach out to NumFocus to figure out legal ramifications of not including licenses in files.

  • [ ] (Eric) check internally for funding levels for hotels & flying folks from the community in?

  • [ ] (Eric) Figure out finances of conda-forge to support themselves?

  • [ ] (jjhelmus) Open up CFEP for which python’s we’re going to support

  • [ ] (jakirkham) write a blog post on CUDA stuff we discussed today

  • [ ] (jakirkham) update docs on how to add CUDA support to feedstocks

  • [ ] (jakirkham) will open an issue on conda-smithy to investigate Drone issues. (ping the aarch team)

    • https://github.com/conda-forge/conda-forge.github.io/issues/954

  • [ ] (ED) Who we are page? Some combination of a FAQ and a who is everyone. FAQ things like:

    • who’s the POC for CF <> Anaconda, CF <> NumFocus, CF <> Azure

    • who’s the POC for the various subteams?

    • Informal information: roles, day jobs, bios, the whole nine yards, why you’re here, etc.

    • Public or internal? I don’t really care either way. Anyone feel strongly one way or the other?

    • opt-in to public bios

    • software carpentry has a large number of instructors and has https://carpentries.org/instructors

    • some concern about “yet another place to keep stuff up to date”

  • [ ] (ED) document strategies for reproducible environments using conda-forge

  • [ ] (UK) Static libraries stuff

    • [ ] Add linting hints to builds to find them

    • [x] Recommend how to package them -> CFEP-18

    • [x] We should write docs saying we don’t provide support and this is a bad idea. -> CFEP-18