Python import errors for MA27 and MA57

[lambe]

(This is the same issue I mentioned to @syarra privately earlier this week.)

The module compiles with no issues and importing MC21, MC29, and MC60 modules in Python is fine. However importing the solvers MA27 and MA57 gives the following error:

from hsl.solvers import _pyma27
ImportError: /home/andrew/anaconda3/envs/nlpy-dev/lib/python2.7/site-packages/hsl-0.1.0.dev0-py2.7-linux-x86_64.egg/hsl/solvers/_pyma27.so: undefined symbol: cblas_idamax

The symbol in question is identified in cblas.h (both in this repo and different versions on my laptop, such as the one included with OpenBLAS) so I suspect a bad include or link of some kind.

Some diagnostics: I compiling the master branch on Ubuntu 16.04, using python 2.7 through the Anaconda distribution (via a virtual environment). If I test the built *.so modules with ld directly, I get

ld: warning: cannot find entry symbol _start; not setting start address
_pyma27.so: undefined reference to `cblas_idamax'
_pyma27.so: undefined reference to `cblas_dcopy'
_pyma27.so: undefined reference to `cblas_daxpy'

and

ld: warning: cannot find entry symbol _start; not setting start address
_pyma57.so: undefined reference to `dgemm_'
_pyma57.so: undefined reference to `dtpsv_'
_pyma57.so: undefined reference to `cblas_dcopy'
_pyma57.so: undefined reference to `idamax_'
_pyma57.so: undefined reference to `dgemv_'

for _pyma27.so and _pyma57.so respectively. Again, everything seems to point to cblas.h not being correctly handled.

[lambe]

After some grep commands in the right places, I can find all of the above symbols listed in different *.a and *.o files in the build directory. It must a broken link issue of some kind, but I still can't find a fix.

[lambe]

I fixed it on my machine, but I'm not ready to say the fix is permanent yet. The fix was to add the blas library explicitly to config.add_library() and config.add_extension for both the MA27 and MA57 modules. (This is available from get_info(blas_opt, 0) in numpy.)

I say this isn't a permanent fix because the link isn't to the same blas library used by numpy, but by a copy in my home folder (checked with ldd). That means something is very strange about how setup.py is finding the libraries it needs. This needs further investigation, so I'll keep the issue open for now.

[syarra]

I tried to reproduce this error on a freshly installed ubuntu 16.04 and cannot reproduce it.

• Python 2.7.12
• gcc 5.3.0
• Numpy 1.11.2

I do not use Anaconda though.

[lambe]

I think it's related to how setup.py searches for libraries. For some reason, the blas version that is linked to my Numpy is not being automatically linked to HSL.py. Is there an easy way to find the default include and library paths that setup.py uses during the install?

[dpo]

Should probably add an option in site.cfg to specify the BLAS/LAPACK libs that we want.

[lambe]

The fix on my machine was to modify the setup.py file to grab the BLAS library and directory explicitly from the blas_info object, then pass those directly to config.add_library() and config.add_extension methods. I can share the changes publically if you like so we can discuss a permanent fix.

[dpo]

Yes it would be good to have a permanent solution.