Build failure on Linux #267
Description
Prerequisites
Before submitting your issue, please ensure the following:
- I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
- I have carefully read and followed the instructions in the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
Expected Behavior
Attempting to build using the install instructions.
- Installed python deps using uv
- Runing cmake to build the binary
Current Behavior
Builld failed
Environment and Context
Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.
- Physical (or virtual) hardware you are using, e.g. for Linux:
$ lscpu
lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 39 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Vendor ID: GenuineIntel
Model name: 13th Gen Intel(R) Core(TM) i9-13900F
CPU family: 6
Model: 183
Thread(s) per core: 2
Core(s) per socket: 24
Socket(s): 1
Stepping: 1
CPU(s) scaling MHz: 21%
CPU max MHz: 5600.0000
CPU min MHz: 800.0000
BogoMIPS: 3993.60
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl
vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms in
vpcid rdseed adx smap clflushopt clwb intel_pt sha_ni xsaveopt xsavec xgetbv1 xsaves split_lock_detect user_shstk avx_vnni dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req hfi vnmi umip pku ospke waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm md_clear ser
ialize arch_lbr flush_l1d arch_capabilities
Virtualization features:
Virtualization: VT-x
Caches (sum of all):
L1d: 896 KiB (24 instances)
L1i: 1.3 MiB (24 instances)
L2: 32 MiB (12 instances)
L3: 36 MiB (1 instance)
NUMA:
NUMA node(s): 1
NUMA node0 CPU(s): 0-31
Vulnerabilities:
Gather data sampling: Not affected
Ghostwrite: Not affected
Indirect target selection: Not affected
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Not affected
Reg file data sampling: Mitigation; Clear Register File
Retbleed: Not affected
Spec rstack overflow: Not affected
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S
Srbds: Not affected
Tsx async abort: Not affected
- Operating System, e.g. for Linux:
$ uname -a
uname -a
Linux hostname 6.14.10-arch1-1 #1 SMP PREEMPT_DYNAMIC Wed, 04 Jun 2025 18:52:35 +0000 x86_64 GNU/Linux
- SDK version, e.g. for Linux:
$ python3 --version
$ make --version
$ g++ --version
python3 --version && make --version && g++ --version
Python 3.13.3
GNU Make 4.4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
g++ (GCC) 15.1.1 20250425
Copyright (C) 2025 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Failure Information (for bugs)
Please help provide information about the failure / bug.
Steps to Reproduce
Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.
In a fresh clone of the repo:
- uv venv
- uv pip install -r requirements.txt
- cmake -S . -B build -DLLAMA_CUBLAS=ON
- cmake --build build --config Release
Failure Logs
Please include any relevant log snippets or files. If it works under one configuration but not under another, please provide logs for both configurations and their corresponding outputs so it is easy to see where behavior changes.
cmake -S . -B build -DLLAMA_CUBLAS=ON
cmake --build build --config Release
-- The C compiler identification is GNU 15.1.1
-- The CXX compiler identification is GNU 15.1.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found CUDAToolkit: /opt/cuda/targets/x86_64-linux/include (found version "12.8.93")
-- cuBLAS found
-- The CUDA compiler identification is NVIDIA 12.8.93 with host compiler GNU 14.3.1
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /opt/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Using CUDA architectures: 52;61;70
GNU ld (GNU Binutils) 2.44.0
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Configuring done (1.9s)
-- Generating done (0.0s)
-- Build files have been written to: /root/development/AI/PowerInfer/build
[ 1%] Building C object CMakeFiles/ggml.dir/ggml.c.o
In function ‘ggml_op_name’,
inlined from ‘ggml_get_n_tasks’ at /root/development/AI/PowerInfer/ggml.c:16847:17:
/root/development/AI/PowerInfer/ggml.c:2006:24: warning: array subscript 71 is above array bounds of ‘const char *[70]’ [-Warray-bounds=]
2006 | return GGML_OP_NAME[op];
| ~~~~~~~~~~~~^~~~
/root/development/AI/PowerInfer/ggml.c: In function ‘ggml_get_n_tasks’:
/root/development/AI/PowerInfer/ggml.c:1586:21: note: while referencing ‘GGML_OP_NAME’
1586 | static const char * GGML_OP_NAME[GGML_OP_COUNT] = {
| ^~~~~~~~~~~~
[ 2%] Building C object CMakeFiles/ggml.dir/ggml-alloc.c.o
[ 3%] Building C object CMakeFiles/ggml.dir/ggml-backend.c.o
[ 4%] Building C object CMakeFiles/ggml.dir/ggml-quants.c.o
/root/development/AI/PowerInfer/ggml-quants.c: In function ‘ggml_axpy_q4_0_q8_0’:
/root/development/AI/PowerInfer/ggml-quants.c:2457:54: warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual]
2457 | __m256 by = _mm256_loadu_ps((const __m256 *)((char *)vy+i*128));
| ^
/root/development/AI/PowerInfer/ggml-quants.c:2457:37: error: passing argument 1 of ‘_mm256_loadu_ps’ from incompatible pointer type [-Wincompatible-pointer-types]
2457 | __m256 by = _mm256_loadu_ps((const __m256 *)((char *)vy+i*128));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| const __m256 *
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/15.1.1/include/immintrin.h:43,
from /root/development/AI/PowerInfer/ggml-impl.h:74,
from /root/development/AI/PowerInfer/ggml-quants.h:3,
from /root/development/AI/PowerInfer/ggml-quants.c:1:
/usr/lib/gcc/x86_64-pc-linux-gnu/15.1.1/include/avxintrin.h:847:31: note: expected ‘const float *’ but argument is of type ‘const __m256 *’
847 | _mm256_loadu_ps (float const *__P)
| ~~~~~~~~~~~~~^~~
/root/development/AI/PowerInfer/ggml-quants.c:2460:36: warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual]
2460 | _mm256_storeu_ps((__m256*)((char*)vz + i*128), by);
| ^
/root/development/AI/PowerInfer/ggml-quants.c:2460:26: error: passing argument 1 of ‘_mm256_storeu_ps’ from incompatible pointer type [-Wincompatible-pointer-types]
2460 | _mm256_storeu_ps((__m256*)((char*)vz + i*128), by);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| __m256 *
/usr/lib/gcc/x86_64-pc-linux-gnu/15.1.1/include/avxintrin.h:853:26: note: expected ‘float *’ but argument is of type ‘__m256 *’
853 | _mm256_storeu_ps (float *__P, __m256 __A)
| ~~~~~~~^~~
/root/development/AI/PowerInfer/ggml-quants.c:2467:47: warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual]
2467 | by = _mm256_loadu_ps((const __m256 *)((char*)vy+i*128+32));
| ^
/root/development/AI/PowerInfer/ggml-quants.c:2467:30: error: passing argument 1 of ‘_mm256_loadu_ps’ from incompatible pointer type [-Wincompatible-pointer-types]
2467 | by = _mm256_loadu_ps((const __m256 *)((char*)vy+i*128+32));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| const __m256 *
/usr/lib/gcc/x86_64-pc-linux-gnu/15.1.1/include/avxintrin.h:847:31: note: expected ‘const float *’ but argument is of type ‘const __m256 *’
847 | _mm256_loadu_ps (float const *__P)
| ~~~~~~~~~~~~~^~~
/root/development/AI/PowerInfer/ggml-quants.c:2469:36: warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual]
2469 | _mm256_storeu_ps((__m256*)((char*)vz + i*128+32), by);
| ^
/root/development/AI/PowerInfer/ggml-quants.c:2469:26: error: passing argument 1 of ‘_mm256_storeu_ps’ from incompatible pointer type [-Wincompatible-pointer-types]
2469 | _mm256_storeu_ps((__m256*)((char*)vz + i*128+32), by);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| __m256 *
/usr/lib/gcc/x86_64-pc-linux-gnu/15.1.1/include/avxintrin.h:853:26: note: expected ‘float *’ but argument is of type ‘__m256 *’
853 | _mm256_storeu_ps (float *__P, __m256 __A)
| ~~~~~~~^~~
/root/development/AI/PowerInfer/ggml-quants.c:2479:47: warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual]
2479 | by = _mm256_loadu_ps((const __m256 *)((char*)vy+i*128+64));
| ^
/root/development/AI/PowerInfer/ggml-quants.c:2479:30: error: passing argument 1 of ‘_mm256_loadu_ps’ from incompatible pointer type [-Wincompatible-pointer-types]
2479 | by = _mm256_loadu_ps((const __m256 *)((char*)vy+i*128+64));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| const __m256 *
/usr/lib/gcc/x86_64-pc-linux-gnu/15.1.1/include/avxintrin.h:847:31: note: expected ‘const float *’ but argument is of type ‘const __m256 *’
847 | _mm256_loadu_ps (float const *__P)
| ~~~~~~~~~~~~~^~~
/root/development/AI/PowerInfer/ggml-quants.c:2482:36: warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual]
2482 | _mm256_storeu_ps((__m256*)((char*)vz + i*128+64), by);
| ^
/root/development/AI/PowerInfer/ggml-quants.c:2482:26: error: passing argument 1 of ‘_mm256_storeu_ps’ from incompatible pointer type [-Wincompatible-pointer-types]
2482 | _mm256_storeu_ps((__m256*)((char*)vz + i*128+64), by);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| __m256 *
/usr/lib/gcc/x86_64-pc-linux-gnu/15.1.1/include/avxintrin.h:853:26: note: expected ‘float *’ but argument is of type ‘__m256 *’
853 | _mm256_storeu_ps (float *__P, __m256 __A)
| ~~~~~~~^~~
/root/development/AI/PowerInfer/ggml-quants.c:2489:47: warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual]
2489 | by = _mm256_loadu_ps((const __m256 *)((char*)vy+i*128+96));
| ^
/root/development/AI/PowerInfer/ggml-quants.c:2489:30: error: passing argument 1 of ‘_mm256_loadu_ps’ from incompatible pointer type [-Wincompatible-pointer-types]
2489 | by = _mm256_loadu_ps((const __m256 *)((char*)vy+i*128+96));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| const __m256 *
/usr/lib/gcc/x86_64-pc-linux-gnu/15.1.1/include/avxintrin.h:847:31: note: expected ‘const float *’ but argument is of type ‘const __m256 *’
847 | _mm256_loadu_ps (float const *__P)
| ~~~~~~~~~~~~~^~~
/root/development/AI/PowerInfer/ggml-quants.c:2491:36: warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual]
2491 | _mm256_storeu_ps((__m256*)((char*)vz + i*128+96), by);
| ^
/root/development/AI/PowerInfer/ggml-quants.c:2491:26: error: passing argument 1 of ‘_mm256_storeu_ps’ from incompatible pointer type [-Wincompatible-pointer-types]
2491 | _mm256_storeu_ps((__m256*)((char*)vz + i*128+96), by);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| __m256 *
/usr/lib/gcc/x86_64-pc-linux-gnu/15.1.1/include/avxintrin.h:853:26: note: expected ‘float *’ but argument is of type ‘__m256 *’
853 | _mm256_storeu_ps (float *__P, __m256 __A)
| ~~~~~~~^~~
/root/development/AI/PowerInfer/ggml-quants.c:2435:12: warning: unused variable ‘acc’ [-Wunused-variable]
2435 | __m256 acc = _mm256_setzero_ps();
| ^~~
make[2]: *** [CMakeFiles/ggml.dir/build.make:121: CMakeFiles/ggml.dir/ggml-quants.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:778: CMakeFiles/ggml.dir/all] Error 2
make: *** [Makefile:146: all] Error 2