LCORE-1216: Bump up to llama-stack 0.4.3

[are-ces]

Description

This is a significant refactoring of all the modules, mostly because the Agents API has been deprecated in favor of the Responses API in llama-stack (already from 0.3.x).

This upgrade is needed to keep lightspeed-providers on par with LCORE

NOTE: run_moderation has not been designed for redaction but to only block the request, thus lightspeed-redactions will block the message if an unauthorized string is detected, as opposed to run_shield where it is possible to redact the original message.

Changes:

• Bump up llama-stack library to 0.4.3
• Refactor agent code to migrate from Agents API to Responses API
• Refactor safety module run_shield, added run_moderation
• Kept temperature override, prioritization to latest used tools, tool fitering

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up service version
• Bump-up dependent library
• Bump-up library or tool used for development (does not change the final image)
• CI configuration change
• Unit tests improvement

Tools used to create PR

Identify any AI code assistants used in this PR (for transparency and review context)

• Partially generated by: Claude

Related Tickets & Documents

• Related Issue # LCORE-1216
• Closes # LCORE-1216

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

I tested manually via curl requests the following:

• Question validity run_shield (valid/invalid questions)
• Question validity run_moderation
• Redaction run_shield (sensitive data redacted)
• Redaction run_moderation (message with sensitive data BLOCKED)
• Tool filtering (11→1 tools)
• min_tools threshold
• Previously called tools persistence
• always_include_tools config
• Temperature override (1.0 for GPT-5)

[tisnik]

I'd say LGTM on my side. But definitely need at least one more reviewer, especially from teams that managed to use provider(s).

[ldjebran]

You are removing the inline::lightspeed_inline_agent we are using in Ansible Lightspeed chatbot, if this PR is merged this will break the chatbot functionality.

[are-ces]

inline::lightspeed_inline_agent still works, the logic has been moved from agent_instance.py  to agents.py

[TamiTakamiya]

@are-ces @ldjebran I could run the updated lightspeed_inline_agent with ansible-chatbot-stack The test setup uses:

• This PR (I created a wheel file from the PR branch)
• Newly generated RAG DB with https://github.com/ansible/aap-rag-content/pull/293
• ansible-chatbot-stack modified for Llama Stack 0.4.3 & this PR https://github.com/TamiTakamiya/ansible-chatbot-stack/tree/TamiTakamiya/AAP-64341/lightspeed-core-0_4_x

The setup is somehow complicated because it's using a number of codes that are not merged to main yet. I will create a memo on my test setup.

Note: My setup does not enable MCP server yet. After writing the memo, I plan to test this with MCP server enabled.

[ldjebran]

why its not the same version 0.4.3 is this intentional ?

[TamiTakamiya]

I tried to use llama-stack-api 0.4.3 for ansible-chatbot-stack and it did not work. I think 0.4.3 is broken.

[are-ces]

Yes, because we updated LCORE to have the api package on 0.4.4 because of a CVE (v0.4.4 shouldn't have breaking changes)

[Jdubrick]

@are-ces since we only consume the safety shield portion for my use case, that part lgtm, fyi

[ldjebran]

@are-ces seems the file https://github.com/lightspeed-core/lightspeed-providers/blob/main/resources/external_providers/inline/agents/lightspeed_inline_agent.yaml

needs to be updated to:

config_class: lightspeed_stack_providers.providers.inline.agents.lightspeed_inline_agent.config.LightspeedAgentsImplConfig
module: lightspeed_stack_providers.providers.inline.agents.lightspeed_inline_agent
api_dependencies: [ inference, safety, tool_runtime, tool_groups, conversations, prompts ]
optional_api_dependencies: [vector_io, files]

The agent lightspeed_inline_agent is passing through the queries and overriding the temperature when configured , unfortunately I was not able to test mcp filtring as seems the lightspeed-stack has a regression as not passing mcp headers received from client by MCP-HEADERS header.

There is a big work done her, @are-ces many thanks for your efforts,
can we wait a little to merge to see comments of the team about mcp headers ?

[ldjebran]

@are-ces many thanks for the work the changes that I proposed in my last comment still valid, tested the mcp but seems the lightspeed_inline_agent is unfortunately not working as expected and breaking when enabling the mcp configuration, I see the mcp returning the list of tools, but the agent seems do not detect that tools and see only 2 instead of more than 300.
this will needs more investigations.

[are-ces]

Hey @ldjebran good catch! I have encountered the same problem, I was handling the tools in a wrong way; basically the MCP servers were not being expanded to their tools so we were counting the MCP servers and comparing them with min_tools.
I have tested it on my side and it works as expected, hopefully the same on your side 😄