Skip to content

Unsafe Rust for improved performance #2

@OutSquareCapital

Description

@OutSquareCapital

Currently, types check happens inside Rust.
for example this function:

#[pyfunction]
#[pyo3(signature = (data, func, *args, **kwargs))]
pub fn for_each_star(
    data: &Bound<'_, PyAny>,
    func: &Bound<'_, PyAny>,
    args: &Bound<'_, PyTuple>,
    kwargs: Option<&Bound<'_, PyDict>>,
) -> PyResult<()> {
    data.try_iter()?.try_for_each(|item| {
        func.concat_star(&item?.cast_into::<PyTuple>()?, args, kwargs)?;
        Ok(())
    })

will check that item is indeed a tuple before passing it to call_func_star.
This give a nice traceback in case of incorrect input:

Traceback (most recent call last):
  File "C:\Users\tibo\python_codes\pyochain\t.py", line 14, in <module>
    y = print(pc.Iter((1, 2)).for_each_star_test(foo))
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "C:\Users\tibo\python_codes\pyochain\src\pyochain\traits\_iterable.py", line 1744, in for_each_star_test
    tls.for_each_star(self.__iter__(), func, *args, **kwargs)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'int' object cannot be cast as 'tuple'

however changing cast_into by cast_into_unchecked (unsafe) lead to small performance gains (+3%), which once compounded across the codebase could become significant if we don't do any runtime type check.

Observations

  1. If I give an incorrect input to the fonction, It will silently crash instead of giving me back the traceback.
  2. Since all python signatures are fully typed, Pylance/ty/wathever would have warned the user of the incorrect signature anyway.
  3. Point 1 is still a problem when interacting with external libs who are not typed.

Conclusion

This remains to be decided, and will surely be revisited during and after the process of migrating the codebase to Rust

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions