fix: pass num_workers from ccproxy.yaml to litellm CLI#12
Open
npinto wants to merge 1 commit intostarbaser:mainfrom
Open
fix: pass num_workers from ccproxy.yaml to litellm CLI#12npinto wants to merge 1 commit intostarbaser:mainfrom
npinto wants to merge 1 commit intostarbaser:mainfrom
Conversation
The `litellm.num_workers` setting in `ccproxy.yaml` was silently ignored. `start_litellm()` built the litellm command without reading this value, so the proxy always started with litellm's default worker count regardless of what was configured. Read `num_workers` from the `litellm:` section of `ccproxy.yaml` and pass it as `--num_workers` to the litellm subprocess when set.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
The
litellm.num_workerssetting inccproxy.yamlis silently ignored.start_litellm()builds the litellm command aslitellm --config <path>without readingnum_workersfrom the config, so the proxy always starts with litellm's default worker count.Template suggests it works:
Actual command (before fix):
Expected command (after fix):
Fix
Read
num_workersfrom thelitellm:section ofccproxy.yamland append--num_workers Nto the litellm subprocess command when configured. When omitted, litellm uses its own default (CPU count or 4).9 lines in
src/ccproxy/cli.py.Test
One unit test following the existing
@patch("subprocess.run")pattern intest_cli.py: write accproxy.yamlwithnum_workers: 8, callstart_litellm(), assert--num_workers 8appears in the subprocess command.Fails on current HEAD, passes with fix. Zero regressions on existing tests.