-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
Description
Add a cobra command to start the serving runtime (vllm in this case) instead of having it automatically be started when the binary is run. The use should be able to run edgellm serve vllm and start the server and edgellm stop vllm to stop the server. This also involves moving over all the flags users can input to the new command. look at the runner internal package to see how the backend server starts.
Reactions are currently unavailable