2017-11-16 compiler meeting notes

Scheduled time: 9 AM central. Meeting link: https://anaconda.webex.com/anaconda-en/j.php?MTID=m11b5ddad66325da22bbe58d7d1c02809

Adopting Anaconda compilers

• Linux: gcc 7.2

  • Prefixed compilers: require activation

  • Common adaptations required for new anaconda compilers:

• Mac: LLVM/clang 4.0.1

  • prefixed compilers: require activation

  • common adaptations required for new anaconda compilers:

• Windows: activation scripts

  • Need adaptation for Appveyor compiler locations

  • common adaptations required:

    • cmake

      • clear CC and/or CXX vars

  import os

  print(“Hello World”)

Compiler flag unification

• GCC

  • toolchain: https://github.com/conda-forge/toolchain-feedstock/blob/master/recipe/activate.sh

  • toolchain3: https://github.com/conda-forge/toolchain3-feedstock/blob/master/recipe/activate.sh

  • anaconda: https://github.com/AnacondaRecipes/aggregate/blob/master/ctng-compilers-activation-feedstock/recipe/conda_build_config.cos6.x86_64.yaml#L41-L54 | Variable | toolchain only | anaconda only | | | ——– | ————————————— | ——————————————————————————————————————————————————— | | | CFLAGS | -m${ARCH} | march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe | | | CPPFLAGS | | -D_FORTIFY_SOURCE=2 -O2 | | | CXXFLAGS | -DBOOST_MATH_DISABLE_FLOAT128 -m${ARCH} | -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe | | | LDFLAGS | -Wl,-rpath,$PREFIX/lib | Wl,-O2 -Wl,–sort-common -Wl,–as-needed -Wl,-z,relro -Wl,-z,now | |

• LLVM/clang

  • toolchain: https://github.com/conda-forge/toolchain-feedstock/blob/master/recipe/activate.sh

  • toolchain3: https://github.com/conda-forge/toolchain3-feedstock/blob/master/recipe/activate.sh

  • anaconda: https://github.com/AnacondaRecipes/aggregate/blob/master/clang/build.sh | Variable | toolchain only | anaconda only | | ———- | ———————————————————————– | —————————————————————————————————————————————————————————- | | CPPFLAGS | | -mmacosx-version-min=${MACOSX_VERSION_MIN} | | CFLAGS | -mmacosx-version-min=${MACOSX_VERSION_MIN} -m${ARCH} | -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe | | CXXFLAGS | -mmacosx-version-min=${MACOSX_VERSION_MIN} -m${ARCH} | -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 | | LDFLAGS | -mmacosx-version-min=${MACOSX_VERSION_MIN} -lc++ -Wl,-rpath,$PREFIX/lib | -pie | | LDFLAGS_CC | | -Wl,-pie -Wl,-headerpad_max_install_names | | | | |

• Fortran

  • toolchain: (not set)

  • toolchain3: (not set)

  • anaconda:

    • Linux: https://github.com/AnacondaRecipes/aggregate/blob/master/ctng-compilers-activation-feedstock/recipe/conda_build_config.cos6.x86_64.yaml#L46

    • Mac: https://github.com/AnacondaRecipes/aggregate/blob/master/gfortran-feedstock/recipe/build.sh#L5

• Windows: flags that affect some things (cmake) | Variable | anaconda value | | ————– | ——————————- | | CFLAGS | %CFLAGS% -MD -GL | | CXXFLAGS | %CXXFLAGS% -MD -GL | | LDFLAGS_SHARED | %LDFLAGS_SHARED% -LTCG ucrt.lib |

Overall: everyone receptive to new compilers. Mike to provide way of keeping host and build prefix separate, even when not cross compiling. This would avoid need for things like “always_include_files” and would facilitate conda-forge keeping their llvmdev recipe as is (for cling usage).

Filipe: this is really little more than a vendor change. We already depend on other vendors for compilers (RH for devtoolset2; apple for existing clang), we’re only switching to a different vendor, not fundamentally changing what we do.

Need to maintain llvm with cling patches, but this will not be the default compiler.

Conda-build 3: strategy for moving

• install and use with c-b-a (no cb3 matrix)

  • Waiting on feedback from issue https://github.com/conda-tools/conda-build-all/issues/94 but mostly seems OK.

• Mike: needs to fix —skip-existing. Concern is that re-rendering should not generate new packages when only some dependency has changed (bugfix bump?)

  • Jonathan to explore ways to skip uploads when only hash has changed as a temporary workaround.

• replace c-b-a with cb3 matrix support

  • replace pinning script with central conda_build_config.yaml

    • Re-render installs from conda-forge central config package, uses that config

    • Each recipe can have its own conda_build_config.yaml alongside its meta.yaml file to override anything

  • Where/how to store intermediate files and distribute CI jobs

    • John recommended committing these to the feedstock repo during re-rendering

    • Jonathan wondered about committing the full conda_build_config.yaml to the repo, or pulling it in as a dependency at build time, but then reducing it using environment variables.

    • Mike wondered about CONDA_VARIANT_* as a pattern for environment variables that CB might recognize, so that we keep the current CI scheme. This probably also integrates with Jonathan’s idea of reducing the matrix on a per-job basis. Conda-smithy would create the set of jobs, each with different env vars to reduce the overall matrix for each job.

• use run_exports and use either c-b-a or cb3

  • People generally interested, but needs to be implemented and proven over time. Good experience so far with defaults.

Fortran support on Windows

• gfortran (msys2) / Flang

• Timeline for adding either

• Mike asked that whatever is done is done with community approval, for the sake of maintaining a high quality user experience.

OpenMP behavior

• Currently, needs extra package on mac, but is included on Linux (not active in flags, though)

• What is desirable default behavior?