fix: removes in-place modification of options in _prepare_workflow_request

[julianolm]

Context

The inplace modification of option object in _prepare_workflow_request is causing a bug.

If an error occurrs after _prepare_workflow_request returns but before new workflows are spawned, then tenacity retries will happen with options dict modified. This is a huge problem when you have any additional_metadata set, because it is going to be a binary in any of the retries (instead of a string).

(I am having some problems in production beacause of it, so it would be very nice to have this resolved asap)

FYI @mmacvicar @banduk @ArthurMor4is @vncsna @cezarguimaraes

To reproduce

Modify AdminClientAioImpl.run_workflows to raise an error after calling _prepare_workflow_request.

Trigger a workflow that uses spawn_workflows and receives some additional_metadata.

In the first attempt, your new error will be raised. In the next attempts the code will fail in _prepare_workflow_request with a
TypeError: Object of type bytes is not JSON serializable.

Trace

Exception trace

Object of type bytes is not JSON serializable
Traceback (most recent call last):

...

return await copy(fn, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 111, in __call__
do = await self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
result = await action(retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
return call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/tenacity/__init__.py", line 398, in
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception

File "lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 114, in __call__
result = await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/hatchet_sdk/context/context.py", line 160, in spawn_workflows
return await self.admin_client.aio.run_workflows(bulk_trigger_workflow_runs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 189, in async_wrapped
return await copy(fn, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 111, in __call__
do = await self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
result = await action(retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
return call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/tenacity/__init__.py", line 398, in
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception

File "lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 114, in __call__
result = await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/hatchet_sdk/clients/admin.py", line 294, in run_workflows
request = self._prepare_workflow_request(workflow_name, input_data, options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "lib/python3.11/site-packages/hatchet_sdk/clients/admin.py", line 98, in _prepare_workflow_request
options["additional_metadata"] = json.dumps(meta).encode("utf-8")
^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/json/__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/json/encoder.py", line 200, in encode
chunks = self.iterencode(o, _one_shot=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/json/encoder.py", line 258, in iterencode
return _iterencode(o, 0)
^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/json/encoder.py", line 180, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

[mrkaye97]

Nice catch, and thanks for the fix! Would you mind bumping the patch version number in the pyproject.toml? Once that's done, happy to ship this 😄

[julianolm]

Nice catch, and thanks for the fix! Would you mind bumping the patch version number in the pyproject.toml? Once that's done, happy to ship this 😄

That's great @mrkaye97 . Thanks for your fast support!
Just upgraded version in pyproject.toml

[mrkaye97]

just released!