conda-forge core meeting 2022-11-16¶
Add new agenda items under the Your __new__() agenda items
heading
last weeks meeting What time is the meeting in my time zone Meeting info:
To join the video meeting, click this link: https://zoom.us/j/9138593505?pwd=SWh3dE1IK05LV01Qa0FJZ1ZpMzJLZz09
Otherwise, to join by phone, dial +1 347-384-8597 and enter this PIN: 828 997 153#
To view more phone numbers, click this link: https://tel.meet/ijv-qsvm-tvn?hs=5
Attendees¶
Name |
Initials |
GitHub ID |
Affiliation |
---|---|---|---|
Jaime Rodríguez-Guerra |
JRG |
jaimergp |
Quansight/cf |
Eric Dill |
EDD |
ericdill |
cf |
Daniel Ching |
DJC |
carterbox |
Argonne |
Marcel Bargull |
MB |
mbargull |
Bioconda/cf |
Matthew R Becker |
MRB |
beckermr |
cf |
Dave Clements |
DPC |
tnabtaf |
Anaconda |
Cheng H. Lee |
CHL |
chenghlee |
Anaconda/cf |
Jannis Leidel |
JL |
jezdez |
Anaconda/cf |
Katherine Kinnaman |
KK |
kathatherine |
Anaconda |
John Kirkham |
JK |
jakirkham |
NVIDIA/cf |
Travis Hathaway |
TH |
travishathaway |
Anaconda :snake: |
Marius van Niekerk |
MvN |
mariusvniekerk |
cf |
Wolf Vollprecht |
WV |
wolfv |
prefix.dev / cf |
Amit Kumar |
AK |
aktech |
Quansight / cirun |
18 people total
Standing items¶
[X] intros for new folks on the call
Amit Kumar - helping setup GPU server for CF. Jaime pulled me in.
Kyle - director of engineering for package building teams at anaconda
[X] open votes
[x] (JRG) Python 3.10.7 and friends https://github.com/conda-forge/python-feedstock/pull/579#issuecomment-1291847544
will circle back to this at the end of the call
Action item: Jaime to write announcement
[x] (JRG) Status on staged-recipes application(s)?
[ ] https://github.com/orgs/conda-forge/teams/core/discussions/12 + another member
Action item: FF to take over
From previous meeting(s)¶
Your new() agenda items¶
[x] (JK) CUDA EULA Amendment
[x] (JK) Python debug build ( https://github.com/conda-forge/python-feedstock/pull/597 )
[x] (WV) Announcing prefix.dev
Issue tracker for ideas: https://github.com/prefix-dev/prefix-dev
[x] (WV) Planning for PackagingCon 2023 is (re)-starting (right after this meeting)
[x] (WV) Got CZI grant for improving conda-forge infra
[ ] https://conda-forge.org/docs/orga/funding/00_intro.html (see submitted proposal here)
[x] (MRB) moving to .conda is ready to ship
See here: https://github.com/conda-forge/conda-forge.github.io/issues/1586
implementation is in the ci-setup package
it looks for a key in the conda-forge.yml at build time and if that key exists and has the right value (conda_pkg_format: 2), then it adjusts the condarc file
note this is not a statically set thing in conda smithy since conda build doesn’t appear to have a command line switch controlling the package format it outputs
Not at this time; code is (partially?) there, but we should probably resurrect this CEP to formalize this feature.
I’ve tested the following for .conda packages
packages can be uploaded through the webserver, cf-staging, etc.
packages can be marked broken and are removed from the repodata
packages be marked not broken and are added back to the repodata
packages can be patched and the patches appear in the repodata
packages are properly picked up and indexed by libcfgraph
Note these checks went beyond simply merging the PRs associated with these items after they passed. I actually went into the repodata/graphdata and/or tried to make environments live and checked they did not work or had the correct (possibly modified) deps.
The roll out would work as follows
make an announcement
setup an admin migration to adjust the conda-forge.yaml for every feedstock slowly
wait till all feedstocks are adjusted and we see no major breakages
adjust default in ci setup package to be .conda
isuru:
could remove the tarbz2 in repodata
TODO
[ ] make sure on announcement you mention the minimum conda version (4.7)
[ ] check that ci services do not do duplicate uploads
[ ] set compression level for big packages
flag is
--zstd-compression-level
default is 22, set here
[x] (JK) Graal Python
https://github.com/oracle/graalpython/issues/254
https://github.com/conda-forge/staged-recipes/pull/20882
https://github.com/conda-forge/python_abi-feedstock/pull/19
[ ] (JRG/AK) GPU Server updates
gpu server is hosted by metrostar. uses github actions. whenever a job is triggered, cirun triggers job on openstack. can run 3 parallel jobs with 1 gpu v100. it is ready to run. there’s an example here https://github.com/aktech/cirun-openstack-example
JK: how do we plug this into a feedstock? do we need a test feedstock to work with?
JR: needs a bit of work on conda-smithy to get this template for cirun. need to figure out a policy for opting-in to the gpu service. shouldn’t be that publicly available at first. need to figure out how we face the load.
MRB: How do we affect which repositories have access (at org level)?
AK: Can add public API for configuring access
test feedstock: https://github.com/conda-forge/cf-autotick-bot-test-package-feedstock
[x] (FF) Staged-recipes reviewers vote results. We had 33 people invited to vote, 25 core, 8 emeritus. only 14 voted and they all seem like active core. - @timsnyder: passed - @mfansler: passed
[ ] DJC: What is policy for publishing forks of packages? - [ ] Two pull requests on staged-recipes for forks of a presumably abandoned package; both want to keep the name of the upstream package - [ ] https://github.com/orgs/conda-forge/teams/staged-recipes/discussions/8 - [ ] https://github.com/conda-forge/staged-recipes/pull/20669 - [ ] https://github.com/conda-forge/staged-recipes/pull/17965 - Marcel: complicated topic. we shuold talk more in-depth about this in an async issue - will the packages clobber each other if they’re both installed? guidance: - must have: dont clobber existing code (run_constrained example) - options: - rebrand your fork - inject your github user name (where the fork lives) into the packge / feedstock name - talk to upstream and take over the fork / feedstock
TODO: adjust PR template to address fork naming
requirements:
run_constrained:
sdsl-lite <0.a0
[x] (MB) Adding a Perl package maintainers team
Pushed to next meeting¶
results of staged-recipes reviewer vote
We had 33 people invited to vote, 25 core, 8 emeritus. only 14 voted and they all seem like active core.
[ ] (JRG) Support for abi3 Python packages with abi3audit checks?
https://blog.trailofbits.com/2022/11/15/python-wheels-abi-abi3audit/
[ ] (DPC) Should we shut down conda-forge@googlegroups.com at the same time we shut the conda google group mailing list (which is the end of the year)?
Active votes¶
staged-recipes reviewers vote is live, i think?
cfep-21 license packages for static/header libraries
Opened July 25, 2022 ends August 2, 2022
CFEPs¶
cfep-12 Removing packages that violate the terms of the source package
Stalled since May 26, 2020
Active debate about moving to “broken” vs deleting from conda-forge channel
Active vote, ends on 2020-03-11
What were the results of the vote?
Did we hear back from NumFOCUS? they did the legal seminar which is recorded
TODOs¶
[ ] (MvN) vs2022 compiler support
Initial stab at some pieces https://github.com/conda-forge/vc-feedstock/pull/46
This is a new VC version
Uses existing UCRT
Would need to use the windows-2022 github actions image
TODO: follow on to change the name of the runtime to vc_runtime
TODO: will need patches for msys epoch for UCRT
(CHL) For reference: msys2 environments - pick your own compiler + libstdc++ + Windows runtime adventure
https://www.msys2.org/docs/package-naming/
(RG) “we need some version of mingw-w64 for SciPy 1.9.0 too, MSVC + gfortran will no longer work. we use an Rtools toolchain in CI now”
[ ] (CHL) How does conda-forge use channeldata.json?
Only known use by c-f: counting # of package namespaces for “By the numbers” on the webpage; other uses deprecated
TODO: CEP to deprecate & remove
--use-channeldata
fromconda-build
TODO: CEP to add run exports to
repodata.json
TODO: Articulate suggested mechanism for “so you’re thinking about taking a contract to work on conda-forge”. tl;dr is you should work on a design doc with the impacted parties (or working groups, or interested folks) on the conda-forge side. Once you have agreement on the scope, delivery timeline, and anything that’s needed from the conda-forge volunteer side, go ahead and sign that contract with your payer. if you don’t do this first, you may encounter some difficulties finishing delivery of your contractual agreement with your payer.
2022-01-12
review Qt PR after logs are uploaded
Add cupython and cuquantum to don’t mirror list (MRB done)
re:
std=c++14
, Wait for Kai to comment and merge the PRCJ and Jaime coordinate to Let Maxiconda know that we can’t use their logo 2021-12-01
WV: Set up meet-and-greet call with homebrew team?
MRB: (repodata patches) make a cron job that runs show_diff.py and posts an issue + commit if it is non-empty
2021-11-03
Self-hosting CI TODOs:
Change URL from drone.conda-forge.org. Proposals:
woodpecker.conda-forge.org
ci.conda-forge.org
Set up monitoring
Quantstack is setting up grafana for the mirror
2021-10-18
2021-09-22
[x] (WV): TensorFlow-GPU ready to go, just need to decide if GPU should get prio over CPU?!
GPU gets prio
2020-11-18
[ ] (IF/MRB/MV) intel oneAPI
todo
[ ] (Nikolay) licensing for opencl_rt
[ ] (Nikolay) intelmpi ABI compat w/ mpich
[ ] (MRB/IF) figure out how exactly to package C/C++ compilers
[ ] (MRB/IF) think about fortran ABI
[x] (MRB) make conda-forge compilers room (add people including keith)
[ ] (MB) asking core members to move to “emeritus” status
[ ] TODO: Eric to set up quarterly check-in for all core members to see if they’re interested in remaining “active” or if they want to move to emeritus
Remove emeritus folks from having access to various credentials (api tokens, twitter password, etc.)? This would require a change to the governance doc.
2020-11-11
TODO: Think about bringing in JOSS to provide context around how we might best write papers
2020-11-03
TODO: Check on Forrest Watters permissions for core
2020-10-28 2020-10-21
[ ] (Marius?) Python 2.7 migration
( ) [ ] make a hint
( ) [ ] make an announcement
( ) [ ] make the hint a lint
2020-10-07
[ ] Make sure to add the NVBug info to the cudatoolkit package that conda-forge makes (if we make one)
2020-09-09
[ ] (ED) Update governance docs with similar voting model as what got put into conda-tools (+3 with no -1 is a pass)
[ ] (SC) Write jinja template to turn institutional partners yaml into a website https://github.com/conda-forge/conda-forge.github.io/blob/master/src/inst_partners.yaml
[ ] (SC) Document what needs to be done to create an OVH account and get access
2020-08-26 Docker hub
[ ] (JK) Check in on Azure build workers to see if they have the docker hub limitation.
[ ] (JK) work with dockerhub to see if we can get OSS status
[ ] Check in again at some point. We haven’t heard back as of 2020-09-23
OVH
[ ] Shout-out on twitter at some point. “Thanks forOVHCloud for providing a VM”, etc. (maybe after we ship qt on windows with it?)
[ ] Figure out how to communicate breaking changes to users. Likely should open up an issue immediately for futher discussion. Ping @kkraus, plus capture notes from further up in these meeting notes
[ ] John K. will update the cuda toolkit feedstock on the git repo to note the NVBug link to the internal NVIDIA issue tracker
[ ] Jonathan will update docs to note that some non-exhaustive list of packages (like cuda-toolkit, MKL, etc.)
[ ] Jonathan will review this PR
[ ] (Kale) schedule conda working group
[ ] cfep-10 next steps: CJ to call a vote for feedback
[ ] cfep-06 next steps: Ask staged recipes team to champion this CFEP and move it forward
[ ] jakirkham & CJ-wright to sync on adding CUDA to the migration bot
[ ] (Eric) Scheduling Anaconda <-> conda-forge sync on anaconda.org requirements gathering
Will try and get this scheduled in the next month.
[ ] (Anthony) Reach out to NumFocus to figure out legal ramifications of not including licenses in files.
[ ] (Eric) check internally for funding levels for hotels & flying folks from the community in?
[ ] (Eric) Figure out finances of conda-forge to support themselves?
[ ] (jjhelmus) Open up CFEP for which python’s we’re going to support
[ ] (jakirkham) write a blog post on CUDA stuff we discussed today
[ ] (jakirkham) update docs on how to add CUDA support to feedstocks
[ ] (jakirkham) will open an issue on conda-smithy to investigate Drone issues. (ping the aarch team)
https://github.com/conda-forge/conda-forge.github.io/issues/954
[ ] (ED) Who we are page? Some combination of a FAQ and a who is everyone. FAQ things like:
who’s the POC for CF <> Anaconda, CF <> NumFocus, CF <> Azure
who’s the POC for the various subteams?
Informal information: roles, day jobs, bios, the whole nine yards, why you’re here, etc.
Public or internal? I don’t really care either way. Anyone feel strongly one way or the other?
opt-in to public bios
software carpentry has a large number of instructors and has https://carpentries.org/instructors
some concern about “yet another place to keep stuff up to date”
[ ] (ED) document strategies for reproducible environments using conda-forge
[ ] (UK) Static libraries stuff
[ ] Add linting hints to builds to find them
[x] Recommend how to package them -> CFEP-18
[x] We should write docs saying we don’t provide support and this is a bad idea. -> CFEP-18