Enhance Parsing of Unparsed Function Calls in LLM Output

[rajkstats]

• Replace the single-regex extraction of function call JSON with the more robust extract_json_objects() helper
• Iterate over all JSON objects in the message content to capture multiple function call attempts
• Return the parsed tool calls if any are found; otherwise, return the original message content along with an empty tool calls list.

[jamesmurdza]

For a PR like this, please give the context and situation where it was failing.

If I remember correctly, this was for a specific provider when using LiteLLM.

[rajkstats]

Problem

• Some LLMs (particularly Gemini) return unstructured tool calls as text for example like this (ACTION: type_text\nPARAMS: youtube.com)
• Our JSON parser returns None for these formats
• This causes 'NoneType' object is not iterable errors

Current Parser:

def parse_json(s):
    try:
        return json.loads(s)
    except json.JSONDecodeError:
        print(f"Error decoding JSON for tool call arguments: {s}")
        return None

See the error in screenshot: Error decoding JSON for tool call arguments: ACTION: type_text
PARAMS: youtube.com

Solution

• Added extract_json_objects function to find JSON objects in text
• extract_json_objects function is more robust for handling LLM outputs where JSON might be embedded in explanatory text or other formats.

Here is an example:

Properly Formatted JSON Response:

parsed_args = parse_json('{"name": "type_text", "parameters": {"text": "youtube.com"}}')
# parsed_args = {"text": "youtube.com"}
tool_call = create_tool_call("type_text", parsed_args)  

Unstructured Response

parsed_args = parse_json('ACTION: type_text\nPARAMS: youtube.com')

• parse_json would return None (because it's not valid JSON)

• extract_json_objects successfully extracts the action and parameters, creating a valid tool call that can be executed

Also, added a dummy property if no parameters are provided, because providers like Gemini require a non-empty

            if not properties:
                properties["noop"] = {
                    "type": "string",
                    "description": "Dummy parameter for function with no parameters.",
                }

[jamesmurdza]

Thank you very much for your work on this PR @rajkstats. It looks like this issue applies to LiteLLM, but LiteLLM isn't implemented yet.