numpy.distutils changes and distinguishing Windows compilers

[rgommers]

MingwPy related PRs to Numpy that got reverted (in numpy/numpy#6534) are numpy/numpy#5614 and numpy/numpy#6354. The issues seemed to come from how to detect which compiler is used. That's a bit of a mess right now. Plus mingw32ccompiler.py could use a cleanup as well.

What would be good to start with is to decide/document what situations on Windows need distinguishing and how. First thought:

• Compilers:
  • MSVC 32-bit and 64-bit
  • MinGW 3.4.5 32-bit (don't break this for a while, there are external users like BioPython).
  • MingwPy 32-bit and 64-bit
  • Other MinGW-w64 builds (unsupported by Numpy, but shouldn't break them in numpy.distutils).
  • Cygwin (?)
• How to detect compilers:
  • In numpy.distutils.misc_util, implement a class that figures out what is what, so we don't get ad-hoc is_win64-like functions inside compiler classes.
  • That class should allow cleanly checking what compiler is used.
  • Maybe this needs a better way than parsing gcc --version for mingwpy?
• C/C++ code:
  • There's now stuff like #if defined(__MINGW32__), defined(__MINGW64__),defined(_WIN64)and#if (defined(_MSC_VER)`` in numpy.
  • It's not always clear if that treats mingw 3.4.5 and mingwpy 32-bit, or mingwpy 64-bit and MSVC, the same or not - and whether that's correct or not.
  • Starting with a doc page here on how it's recommended to check for each compiler and bit-ness would be useful.

[carlkl]

not the best time to answer, the countdown runs ....

the main problem was indeed incomplete compiler detection, that has to be handled in numpy.distutils.misc_util.mingw32(). Maybe this function should return a string describing the mingw flavour or None.

I will later add a document describing compiler detection during configure (not needed by numpy), C-compile and numpy distutils.

Other question is about correct compiler flag settings: should this be handled by numpy.distutils or by setting OPT, FOPT?

[rgommers]

the main problem was indeed incomplete compiler detection, that has to be handled in numpy.distutils.misc_util.mingw32(). Maybe this function should return a string describing the mingw flavour or None.

Instead of a string, I was thinking a class instance with attributes like:

is_mingw32
is_mingw_w64
is_mingwpy
is_msvc
is_cygwin
is_64bit
is_32bit
version   # namedtuple(major, minor, micro) for all but msvc
version_string

[rgommers]

Other question is about correct compiler flag settings: should this be handled by numpy.distutils or by setting OPT, FOPT?

Having the correct flags in numpy.distutils is more user-friendly and more robust I think. It's useful however to describe in the MingwPy docs how to override those hardcoded flags, because that's sometimes necessary.

[carlkl]

maybe its the best to add another function numpy.distutils.misc_util.windows_compiler() for this action and let numpy.distutils.misc_util.mingw32() just use this new function.

[rgommers]

that sounds about right

[rgommers]

I can have a go at that, plus a bit of necessary cleanup in those files.

[techtonik]

SCons also needs a revamp for detection of compiler toolchain. Is it possible to describe the needs of mingwpy project in more formal "user story" format so that they can be acted upon by people not familiar with numpy/mingwpy specifics?