Powerinfer's server-side babble has occurred while the command-side looks good #269
Description
Prerequisites
Before submitting your question, please ensure the following:
- I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
- I have carefully read and followed the instructions in the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
Question Details
Powerinfer's server-side babble has occurred
My configuration is as follows
The commands I run are as follows
./build/bin/server -m /PowerInfer/path/ReluLLaMA-7B.powerinfer.ggufCan someone explain to me why, or is that why the server side is not written right
Additional Context
It is worth noting that my reasoning works fine when running locally, but there is some gibberish on the server.
The commands I run on the command line are as follows
./build/bin/main -m /home/yswang/AI_LLM/PowerInfer/models/GGUF/ReluLLaMA-7B.powerinfer.gguf -n 128 -t 8 -p "Once upon a time"