2020-07-01 conda-forge core meeting

Attendees

Agenda

Your agenda items

• [x] (WV) Quick announce of micromamba (https://gist.github.com/wolfv/fe1ea521979973ab1d016d95a589dcde)

• [x] (WV) Update on standardization of next gen package format from last meeting?

  • https://github.com/TheSnakePit/conda-specs/blob/master/proposed_specs/recipe.md

  • https://github.com/TheSnakePit/boa/blob/master/boa/cli/render.py

• [x] (MRB/IF) cos7 and CDTs plans (https://github.com/conda-forge/conda-forge.github.io/issues/1085)

  • cos7 compilers are working!

  • we have removed any mention of cos6 or cos7 from the sysroot path so that we have one compiler

    • old sysroot: x86_64-conda_cos6-linux-gnu

    • new sysroot: x86_64-conda-linux-gnu

  • we now use the sysroot/glibc/kernel headers packages directly from cos6 or cos7 when building (instead of rebuilding glibc)

  • this has created an issue where CDTs get installed to the wrong path (sym links kind of work, but one would not be able to swap cos6 to cos7 in an env)

  • thus we want to rebuild the CDTs to remove cos6 and cos7 from their paths (and will use the sysroot packages to make sure old-style CDTs, cos6 and cos7 are all mutually exclusive in envs)

  • old-style CDT:

    • needs to have no_hoist in the source sections

    • sysroot dir has conda_cos6 or conda_cos7 in the path

    • needs to have run_constrained entry of sysroot_{subdir} ==99999999999 to prevent it from being co-installed w/ the new compilers or CDTs

  • new-style CDT:

    • needs to have no_hoist in the source sections

    • sysroot dir has conda only in the path

    • needs to have run requirement on the proper version of the sysroot_{subdir} package so that it is only installed with CDTs from the right version of CentOS

      • 2.12 for cos6

      • 2.17 for cos7

  • old plan is to roughly

    1. patch repo data to make current CDTs on conda-forge conflict w/ new compilers (PR issued)

    2. move all of the needed CDTs from defaults to conda-forge so that we can build from conda-forge (in progress)

    3. turn on strict channel priority (or even remove defaults from list of channels for linux) for builds

    4. write a migrator that will issue PRs to change all old-style CDTs to new-style CDTs

    5. remove path shims from new compilers and sysroots

  • which ones to move etc

    • list of needed things here: https://github.com/beckermr/misc/blob/master/work/conda_forge_cdt_scripts/needed_cdt_list.yaml

    • keep

      • selinux, mesa, pciutils etc (IDK what is included in etc here)

      • X11

      • ca-cert is a dep of the java CDTs

      • libxt (use by gnuplot and r-base)

    • maybe keep

      • alsa ones (used by qt but we have this packaged too: https://github.com/conda-forge/alsa-lib-feedstock)

      • libpng-devel (needed for wxpython and javafx-sdk)

      • numactl (used by a lot of feedstocks)

    • not keep

      • zip (but not in list also)

  • new plan!

    1. put run_constrained in defaults and conda-forge

    2. build special repo to build them and upload them outside of feedstocks (both new and old kinds)

    3. build and upload the minimal set to conda-forge

    4. turn on strict channel priority in builds and/or remove defaults from channels

    5. push final builds of the new compilers

• [ ] (FF) Should we do Outreachy as part of an effort to support diversity in tech? Advantages are low cost and high impact. Dissdvantages are the time effort from the mentors.

• [x] (UK) CFEP-18: (Not) Packaging static libraries

• [x] (IF) cf-mark-broken: Marking not broken packages as broken

  • marking packages broken breaks envs

  • repo data patches are always better

  • we might want to allow maintainers to mark things broken when the consumers of them are only themselves

  • the speedy aspect of doing things is good for pushing bug fixes fast

  • to do (MRB)

    • [x] 24 hour time limit

    • [x] add more detail to PR text to help ppl understand what they are doing

    • [x] make it easier to revert

• [x] (KK) CUDA 11 support

  • CUDA 11 dropped CentOS 6 support

  • ties into CentOS 7 migration above

  • cudatoolkit 11 - https://github.com/AnacondaRecipes/cudatoolkit-feedstock/pull/7

    • Make sure we have a glibc run constrain line

  • Add Docker image (need to double check how FROM can be adjusted conditionally)

    • Needs a recent docker version but this should work

      ARG IMAGE=ubuntu:lastest
      FROM ${IMAGE}
      

• [x] (CJ) Anyone want to put in for PyData Global?

  • https://global.pydata.org/pages/cfp.html

  • (submissions not open yet)

  • CJ will put something in under “Lessons from Industry” discussing a risk model view of CF (please let CJ know if you are interested/have a perspective to add)

• [ ] (IF) strict channel priority

  • Needed for PyPy

  • Needed for Cos7

  • Remove packages that we don’t fully support. For eg: there are 3 builds of scipy on windows.

Active votes

Subteam updates

Bot

• [x] (CJ) Status of openjdk, r-java stalled out

Stuff from last week that we didnt get to

ARM

POWER

CUDA

Docs

staged-recipes

website

security+systems

• [ ] still need to finish CFEP-13

CI infrastructure

Compiler upgrade

CFEP updates

Open PRs

• cfep-04 X11 and CDT policy

  • INACTIVE - Merge in with some inactive-esque status?

  • Needs new champion. Thanks for your work on this pkgw! Has unaddressed comments from pkgw as from Jan 10, 2020

• cfep-06 Staged-recipes review lifecycle

  • INACTIVE - Merge in with some inactive-esque status?

  • Lingering comment from @saraedum. @jakirkham, can you reply? Has unadressed comment from @saraedum from Jan 8, 2020

  • (MRB) The stalebot has solved the worst of the issues here. I think we could defer this one permanently.

• cfep-10 Feedstock statuses, unmaintained

  • INACTIVE - Merge in with some inactive-esque status?

  • Needs another review. Has unaddressed updates from pkgw as of Jan 11, 2020

• cfep-12 Removing packages that violate the terms of the source package

  • Active debate about moving to “broken” vs deleting from conda-forge channel

  • Active vote, ends on 2020-03-11

  • What were the results of the vote?

  • Did we hear back from NumFOCUS?

• cfep-18 Packaging static libraries

Discussion

Check in on previous action items

Copy previous action items from last meeting agenda.

Last meeting

• [ ] (CJ) Form finance subteam

• [ ] (MRB) document how users are supposed to interpret broken label

• [ ] (MRB) update how we, as core, are supposed to mark packages as broken

• [ ] (ED) document strategies for reproducible environments using conda-forge

• [ ] (UK) Static libraries stuff

  • [ ] Add linting hints to builds to find them

  • [x] Recommend how to package them -> CFEP-18

  • [x] We should write docs saying we don’t provide support and this is a bad idea. -> CFEP-18

2 meetings ago

• [ ] (ED) Who we are page? Some combination of a FAQ and a who is everyone. FAQ things like:

  • who’s the POC for CF <> Anaconda, CF <> NumFocus, CF <> Azure

  • who’s the POC for the various subteams?

  • Informal information: roles, day jobs, bios, the whole nine yards, why you’re here, etc.

  • Public or internal? I don’t really care either way. Anyone feel strongly one way or the other?

  • opt-in to public bios

  • software carpentry has a large number of instructors and has https://carpentries.org/instructors

  • some concern about “yet another place to keep stuff up to date”

3 meetings ago

Move to Issue Tracker

• [ ] (Kale) schedule conda working group

• [ ] cfep-10 next steps: CJ to call a vote for feedback

• [ ] cfep-06 next steps: Ask staged recipes team to champion this CFEP and move it forward

• [ ] jakirkham & CJ-wright to sync on adding CUDA to the migration bot

• [ ] (Eric) Scheduling Anaconda <-> conda-forge sync on anaconda.org requirements gathering

  • Will try and get this scheduled in the next month.

• [ ] (Anthony) Reach out to NumFocus to figure out legal ramifications of not including licenses in files.

• [ ] (Eric) check internally for funding levels for hotels & flying folks from the community in?

• [ ] (Eric) Figure out finances of conda-forge to support themselves?

• [ ] (jjhelmus) Open up CFEP for which python’s we’re going to support

• [ ] (jakirkham) write a blog post on CUDA stuff we discussed today

• [ ] (jakirkham) update docs on how to add CUDA support to feedstocks

• [ ] (jakirkham) will open an issue on conda-smithy to investigate Drone issues. (ping the aarch team)

  • https://github.com/conda-forge/conda-forge.github.io/issues/954