Installing on Python2.7

[techolga]

I try to install pysparse on ubuntu 16.04.1, numpy version 1.14.5,.
When "sudo pip install pysparse" I get this error:

pysparse/sparse/src/spmatrixmodule.c:4:22: fatal error: spmatrix.h: No such file or directory
    compilation terminated.
    error: Command "x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DLENFUNC_OK=1 -DNO_ATLAS_INFO=1 -DHAVE_CBLAS -Ipysparse/sparse/src -I/tmp/pip-install-zB_RaU/pysparse/pysparse/include -I/usr/local/include -I/usr/include -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/tmp/pip-install-zB_RaU/pysparse/pysparse/include -I/usr/include/python2.7 -c pysparse/sparse/src/spmatrixmodule.c -o build/temp.linux-x86_64-2.7/pysparse/sparse/src/spmatrixmodule.o -MMD -MF build/temp.linux-x86_64-2.7/pysparse/sparse/src/spmatrixmodule.o.d" failed with exit status 1
    
    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-zB_RaU/pysparse/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-pKOp5o/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-zB_RaU/pysparse/

When I try to build it with sudo python setup.py install or sudo python setup.py config_fc --fcompiler=gfortran build i get this:

error: Command "x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DUSE_VENDOR_BLAS=1 -DNO_ATLAS_INFO=1 -DHAVE_CBLAS -Ipysparse/direct/src -Ipysparse/direct/superlu/include -I/usr/local/include -I/usr/include -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/home/jayohh/topy/pysparse/pysparse/include -I/usr/include/python2.7 -c pysparse/direct/superlu/src/util.c -o build/temp.linux-x86_64-2.7/pysparse/direct/superlu/src/util.o -MMD -MF build/temp.linux-x86_64-2.7/pysparse/direct/superlu/src/util.o.d" failed with exit status 1

Hope you can help, I need it for a script called ToPy and i am not very in python.

[dpo]

It could be that the version of pysparse on PyPI is outdated. The following worked for me:

pip install git+https://github.com/PythonOptimizers/pysparse.git

[techolga]

I tried but it gave me an error, i read something about using dedicated numpy version and installed numpy 1.7.1 and changed /usr/local/lib/python2.7/dist-packages/numpy/core/include/numpy/npy_no_deprecated_api.h to #define NPY_NO_DEPRICATED_API NPY_1_7_API_VERSION but that gave me this

/usr/bin/ld: cannot find -lf77blas
    /usr/bin/ld: cannot find -lcblas
    /usr/bin/ld: cannot find -latlas
    collect2: error: ld returned 1 exit status
    /usr/bin/ld: cannot find -lf77blas
    /usr/bin/ld: cannot find -lcblas
    /usr/bin/ld: cannot find -latlas
    collect2: error: ld returned 1 exit status

on the newest version of numpy i get this error:

    pysparse/direct/superlu/src/util.c:266:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
     LUSolveFlops()
     ^
    cc1: some warnings being treated as errors
    error: Command "x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DUSE_VENDOR_BLAS=1 -DSCIPY_MKL_H -DHAVE_CBLAS -Ipysparse/direct/src -Ipysparse/direct/superlu/include -I/usr/local/include -I/usr/include -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/tmp/pip-req-build-3d2UGt/pysparse/include -I/usr/include/python2.7 -c pysparse/direct/superlu/src/util.c -o build/temp.linux-x86_64-2.7/pysparse/direct/superlu/src/util.o -MMD -MF build/temp.linux-x86_64-2.7/pysparse/direct/superlu/src/util.o.d" failed with exit status 1
    
    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-req-build-3d2UGt/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-MhsjRv/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-req-build-3d2UGt/

with many more of the "function declaration isn't a prototype" warnings in util.c and util.h.

[dpo]

I'm using Numpy 1.14.5. Try creating a virtual environment to avoid conflicts with system-wide Python packages. I just did

$ mkvirtualenv test-pysparse
$ workon test-pysparse
$ pip install numpy
$ pip install git+https://github.com/PythonOptimizers/pysparse.git

[techolga]

nop, still same warnings and this error:

Command "/home/username/test-pysparse/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-req-build-xb1Uc4/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-A0Ynuo/install-record.txt --single-version-externally-managed --compile --install-headers /home/username/test-pysparse/include/site/python2.7/pysparse" failed with error code 1 in /tmp/pip-req-build-xb1Uc4/

[dpo]

Does it also fail if you clone the repository and run python setup.py install by hand?

[techolga]

then its compiling some files and ends with nearly the same:

cc1: some warnings being treated as errors
error: Command "x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DUSE_VENDOR_BLAS=1 -DSCIPY_MKL_H -DHAVE_CBLAS -Ipysparse/direct/src -Ipysparse/direct/superlu/include -I/usr/local/include -I/usr/include -I/home/username/.virtualenvs/pysparsetest/include -I/home/username/.virtualenvs/pysparsetest/local/lib/python2.7/site-packages/numpy/core/include -I/home/username/.virtualenvs/pysparsetest/local/lib/python2.7/site-packages/numpy/core/include -I/home/username/pysparse/pysparse/include -I/usr/include/python2.7 -c pysparse/direct/superlu/src/util.c -o build/temp.linux-x86_64-2.7/pysparse/direct/superlu/src/util.o -MMD -MF build/temp.linux-x86_64-2.7/pysparse/direct/superlu/src/util.o.d" failed with exit status 1

[dpo]

Could you please post the entire error message?

[techolga]

 --- Could not run svn info ---
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'summary'
  warnings.warn(msg)
blas_opt_info:
blas_mkl_info:
customize UnixCCompiler
  FOUND:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/usr/local/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/usr/local/include', '/usr/include', '/home/JayOHH/.virtualenvs/pysparsetest/include']

  FOUND:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/usr/local/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/usr/local/include', '/usr/include', '/home/JayOHH/.virtualenvs/pysparsetest/include']

Sparse:: Using BLAS info:
{'libraries': ['mkl_rt', 'pthread'], 'library_dirs': ['/usr/local/lib'], 'define_macros': [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)], 'include_dirs': ['/usr/local/include', '/usr/include', '/home/JayOHH/.virtualenvs/pysparsetest/include']}
Using dflt_lib_dirs =  []
Using dflt_libs =  []
Eigen:: Using BLAS info:
{'libraries': ['mkl_rt', 'pthread'], 'library_dirs': ['/usr/local/lib'], 'define_macros': [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)], 'include_dirs': ['/usr/local/include', '/usr/include', '/home/JayOHH/.virtualenvs/pysparsetest/include']}
lapack_opt_info:
lapack_mkl_info:
customize UnixCCompiler
  FOUND:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/usr/local/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/usr/local/include', '/usr/include', '/home/JayOHH/.virtualenvs/pysparsetest/include']

  FOUND:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/usr/local/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/usr/local/include', '/usr/include', '/home/JayOHH/.virtualenvs/pysparsetest/include']

Eigen:: Using LAPACK info:
{'libraries': ['mkl_rt', 'pthread'], 'library_dirs': ['/usr/local/lib'], 'define_macros': [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)], 'include_dirs': ['/usr/local/include', '/usr/include', '/home/JayOHH/.virtualenvs/pysparsetest/include']}
Direct:: Using BLAS info:
{'libraries': ['mkl_rt', 'pthread'], 'library_dirs': ['/usr/local/lib'], 'define_macros': [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)], 'include_dirs': ['/usr/local/include', '/usr/include', '/home/JayOHH/.virtualenvs/pysparsetest/include']}
Using default UMFPACK and AMD
 If you do not like this, edit the [UMFPACK] and [AMD]
 sections of site.cfg
Using default SuperLU.
 If you do not like this, edit the [SuperLU] section of site.cfg
/home/JayOHH/.virtualenvs/pysparsetest/local/lib/python2.7/site-packages/setuptools/dist.py:398: UserWarning: Normalizing '1.3-dev' to '1.3.dev0'
  normalized_version,
running install
running bdist_egg
running egg_info
running build_src
build_src
building py_modules sources
building extension "pysparse.sparse.spmatrix" sources
building extension "pysparse.eigen.jdsym" sources
building extension "pysparse.direct.umfpack" sources
building extension "pysparse.direct.superlu" sources
building extension "pysparse.itsolvers.krylov" sources
building extension "pysparse.precon.precon" sources
building data_files sources
build_src: building npy-pkg config files
writing requirements to pysparse.egg-info/requires.txt
writing pysparse.egg-info/PKG-INFO
writing top-level names to pysparse.egg-info/top_level.txt
writing dependency_links to pysparse.egg-info/dependency_links.txt
reading manifest file 'pysparse.egg-info/SOURCES.txt'
writing manifest file 'pysparse.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying pysparse/version.py -> build/lib.linux-x86_64-2.7/pysparse
copying build/src.linux-x86_64-2.7/pysparse/__config__.py -> build/lib.linux-x86_64-2.7/pysparse
copying build/src.linux-x86_64-2.7/pysparse/sparse/__config__.py -> build/lib.linux-x86_64-2.7/pysparse/sparse
copying build/src.linux-x86_64-2.7/pysparse/eigen/__config__.py -> build/lib.linux-x86_64-2.7/pysparse/eigen
copying build/src.linux-x86_64-2.7/pysparse/direct/__config__.py -> build/lib.linux-x86_64-2.7/pysparse/direct
copying build/src.linux-x86_64-2.7/pysparse/itsolvers/__config__.py -> build/lib.linux-x86_64-2.7/pysparse/itsolvers
copying build/src.linux-x86_64-2.7/pysparse/precon/__config__.py -> build/lib.linux-x86_64-2.7/pysparse/precon
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'pysparse.direct.superlu' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC

compile options: '-DUSE_VENDOR_BLAS=1 -DSCIPY_MKL_H -DHAVE_CBLAS -Ipysparse/direct/src -Ipysparse/direct/superlu/include -I/usr/local/include -I/usr/include -I/home/JayOHH/.virtualenvs/pysparsetest/include -I/home/JayOHH/.virtualenvs/pysparsetest/local/lib/python2.7/site-packages/numpy/core/include -I/home/JayOHH/.virtualenvs/pysparsetest/local/lib/python2.7/site-packages/numpy/core/include -I/home/JayOHH/pysparse/pysparse/include -I/usr/include/python2.7 -c'
x86_64-linux-gnu-gcc: pysparse/direct/superlu/src/util.c
In file included from pysparse/direct/superlu/src/util.c:23:0:
pysparse/direct/superlu/include/util.h:94:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 extern void    StatFree();
 ^
pysparse/direct/superlu/include/util.h:100:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 extern double  SuperLU_timer_ ();
 ^
pysparse/direct/superlu/src/util.c: In function ‘superlu_abort_and_exit’:
pysparse/direct/superlu/src/util.c:32:5: error: format not a string literal and no format arguments [-Werror=format-security]
     fprintf(stderr, msg);
     ^
pysparse/direct/superlu/src/util.c: At top level:
pysparse/direct/superlu/src/util.c:251:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 StatFree()
 ^
pysparse/direct/superlu/src/util.c:260:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 LUFactFlops()
 ^
pysparse/direct/superlu/src/util.c:266:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 LUSolveFlops()
 ^
cc1: some warnings being treated as errors
In file included from pysparse/direct/superlu/src/util.c:23:0:
pysparse/direct/superlu/include/util.h:94:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 extern void    StatFree();
 ^
pysparse/direct/superlu/include/util.h:100:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 extern double  SuperLU_timer_ ();
 ^
pysparse/direct/superlu/src/util.c: In function ‘superlu_abort_and_exit’:
pysparse/direct/superlu/src/util.c:32:5: error: format not a string literal and no format arguments [-Werror=format-security]
     fprintf(stderr, msg);
     ^
pysparse/direct/superlu/src/util.c: At top level:
pysparse/direct/superlu/src/util.c:251:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 StatFree()
 ^
pysparse/direct/superlu/src/util.c:260:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 LUFactFlops()
 ^
pysparse/direct/superlu/src/util.c:266:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 LUSolveFlops()
 ^
cc1: some warnings being treated as errors
error: Command "x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DUSE_VENDOR_BLAS=1 -DSCIPY_MKL_H -DHAVE_CBLAS -Ipysparse/direct/src -Ipysparse/direct/superlu/include -I/usr/local/include -I/usr/include -I/home/JayOHH/.virtualenvs/pysparsetest/include -I/home/JayOHH/.virtualenvs/pysparsetest/local/lib/python2.7/site-packages/numpy/core/include -I/home/JayOHH/.virtualenvs/pysparsetest/local/lib/python2.7/site-packages/numpy/core/include -I/home/JayOHH/pysparse/pysparse/include -I/usr/include/python2.7 -c pysparse/direct/superlu/src/util.c -o build/temp.linux-x86_64-2.7/pysparse/direct/superlu/src/util.o -MMD -MF build/temp.linux-x86_64-2.7/pysparse/direct/superlu/src/util.o.d" failed with exit status 1

sorry havent read your answer. I also tried with Debian today, same error with "pip install git+https://github.com/PythonOptimizers/pysparse.git"

[dpo]

Ok thanks for your report. Let me try and set up continuous integration for this repository.

[kfedick]

I found the error that is causing the problem. This fprintf statement is being considered a security issue by the compiler and is causing the cc1 error. A temporary fix can be applied by commenting out this line and replacing it with a simple error message or by setting gcc to ignore errors but this isn't an ideal fix.

pysparse/direct/superlu/src/util.c: In function ‘superlu_abort_and_exit’:
pysparse/direct/superlu/src/util.c:32:5: error: format not a string literal and no format arguments [-Werror=format-security]
fprintf(stderr, msg);
^

[dpo]

Thanks for tracking that down! Perhaps we could also tell the compiler to treat format-security as a warning and not an error? Would you like to submit a pull request?

[kfedick]

I don't know if that would be the best solution. The security warning is there to prevent a buffer overflow vulnerability which is probably not a concern here but the best way to fix it would be to properly format the output. I am not sure how the message is formatted though so I am not sure what to use here. Edit: I just realized I wrote buffer overflow vulnerability. What I meant was uncontrolled format string.

…

On Tue, Jan 22, 2019 at 2:19 PM Dominique ***@***.***> wrote: Thanks for tracking that down! Perhaps we could also tell the compiler to treat format-security as a warning and not an error? Would you like to submit a pull request? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMNl9tFQhPKJNII5zO_W_snaDx-bj79aks5vF2QsgaJpZM4U1u6-> .

[ajcoelho]

I found the error that is causing the problem. This fprintf statement is being considered a security issue by the compiler and is causing the cc1 error. A temporary fix can be applied by commenting out this line and replacing it with a simple error message or by setting gcc to ignore errors but this isn't an ideal fix.

pysparse/direct/superlu/src/util.c: In function ‘superlu_abort_and_exit’:
pysparse/direct/superlu/src/util.c:32:5: error: format not a string literal and no format arguments [-Werror=format-security]
fprintf(stderr, msg);
^

That worked for me!
Do you have any idea why that fprintf is causing it?

[kfedick]

Basically, since the fprintf statement doesn't state the format of the output, it is being flagged as an uncontrolled format string by the compiler. This is a security flaw because it is possible to insert malicious code, read the stack, crash a program, and more. The compiler is set up to fail with security warnings. Commenting it out/changing it to a generic message works because it is just an error message and not part of the functions themselves. Setting the compiler to ignore errors would work since the information being outputted is from a trusted source (the library itself) but this would mean any other security issues that may arise and be actual problems will be ignored so it isn't a good idea. https://fedoraproject.org/wiki/Format-Security-FAQ https://en.wikipedia.org/wiki/Uncontrolled_format_string

…

On Wed, Feb 27, 2019 at 12:11 PM ajchc ***@***.***> wrote: I found the error that is causing the problem. This fprintf statement is being considered a security issue by the compiler and is causing the cc1 error. A temporary fix can be applied by commenting out this line and replacing it with a simple error message or by setting gcc to ignore errors but this isn't an ideal fix. pysparse/direct/superlu/src/util.c: In function ‘superlu_abort_and_exit’: pysparse/direct/superlu/src/util.c:32:5: error: format not a string literal and no format arguments [-Werror=format-security] fprintf(stderr, msg); ^ That worked for me! Do you have any idea why that fprintf is causing it? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMNl9uuS9oCQgfknu-Ihkm9dRPMoYufpks5vRrwlgaJpZM4U1u6-> .