Skip to content

Update SGLang PD Disagg with sgl router and updated benchmarking#128

Open
gargrahul wants to merge 1 commit intoROCm:developfrom
gargrahul:gargrahul/sgl_pd_disagg
Open

Update SGLang PD Disagg with sgl router and updated benchmarking#128
gargrahul wants to merge 1 commit intoROCm:developfrom
gargrahul:gargrahul/sgl_pd_disagg

Conversation

@gargrahul
Copy link
Collaborator

No description provided.

@@ -1,69 +1,57 @@
#!/bin/bash

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lcskrishna - Any specific need for 2 benchmarking scripts ?
One says _GSP and other normal

if [ "$p_con" -lt 16 ]; then
p_con=16
fi
echo "[RUNNING] prompts $prompts isl $isl osl $osl con $con model ${MODEL_NAME} xP=${xP} yD=${yD} job=${SLURM_JOB_ID}" | tee -a "${LOG}_CONCURRENCY.log"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lcskrishna - I believe these tags were added for benchmark results parsing. Any particular need to remove them ?

echo "${PREFILL_ARGS} are Proxy's Prefill"
echo "${DECODE_ARGS} are Proxy's Decode"
echo "================================================"
echo "Proxy server is waiting for prefile and decode nodes to be ready ..." \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo - prefill

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants