Cross Contamination

[marcelinhov2]

FIrst of all, this tool is INSANE, I LOVE IT FROM THE BOTTOM OF MY HEART...

The problem I'm facing here is that I can't grant a "cache thread". I have a multiagent solution that I tried to use with semcache, but as soon as I make the same question using 2 different contexts, it returns the same response.

So maybe it would be good if we can pass some kind of id that grant's the context of the cache, is it something that can be done?

I even tried to pass different id's, but it ignored completely.

Thank you.

[louiscb]

Hi @marcelinhov2 thanks for the kind words and apologies for the delayed response.

I take your issue to mean that you want a single Semcache instance to store different responses to the same prompts, like a namespace. So you don't get key collisions for the same prompts. That's definitely something that could be done and has been discussed in the past.

How are you interacting with Semcache? Using the cache aside with the python sdk I'd want it work something like this:

from semcache import Semcache

cache_a = Semcache(
    base_url="http://localhost:8080",
    namespace="agent-1"
)

cache_a.put("What's your name?", "John")

cache_b = Semcache(
    base_url="http://localhost:8080",
    namespace="agent-2"
)

cache_b.put("What's your name?", "James")

What do you think? And PRs are always welcome.

[marcelinhov2]

Thanks for the response @louiscb .

I'm using as base_url directly into my openai agent definition. This way you made is good or maybe we could add a header to the request as well, idk.

[marcelinhov2]

even passing the namespace in the querystring would be super nice and useful

[louiscb]

I've created a PR implementing the namespace feature using headers. It's not released but I've made a docker image for it you can pull it with: semcache/semcache:namespace

You can set the namespace header with the openai sdk like:

agent_a = OpenAI(
    base_url="http://localhost:8080",  # Semcache endpoint
    api_key=api_key,
    default_headers={"X-Semcache-Namespace": "agent-a"}
)

response = agent_a.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Make up a random one line limerick!"}]
)
print(f"Response: {response.choices[0].message.content}")

agent_b = OpenAI(
    base_url="http://localhost:8080",  # Semcache endpoint
    api_key=api_key,
    default_headers={"X-Semcache-Namespace": "agent-b"}
)

response = agent_b.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Make up a random one line limerick!"}]
)
print(f"Response: {response.choices[0].message.content}")

@marcelinhov2 test it out and lmk if it works for your agentic workflow! Once it looks good I'll go ahead and do a release