Question about memory management in PyOSQPSolver::get_solution()

[XiDianZuoYun]

Question about memory management in PyOSQPSolver::get_solution()

Version

• osqp-python: v1.0.5
• Python: 3.10
• OS: Linux

Background

I'm investigating a memory growth issue in my application and Valgrind is reporting what appears to be a memory leak originating from PyOSQPSolver::get_solution(). However, I suspect I might be misunderstanding the intended design.

Code in Question

In src/bindings.cpp.in (lines 167-171):

PyOSQPSolution& PyOSQPSolver::get_solution() {
    PyOSQPSolution* solution = new PyOSQPSolution(*this->_solver->solution, this->m, this->n);
    return *solution;
}

Combined with the binding (line 492):

.def_property_readonly("solution", &PyOSQPSolver::get_solution, py::return_value_policy::reference)

My Understanding (possibly incorrect)

From my reading:

• Each call to get_solution() allocates a new PyOSQPSolution on the heap
• py::return_value_policy::reference tells pybind11 not to manage this object's lifetime
• I don't see where these objects are deleted

However, the Python layer (interface.py lines 426-429) accesses .solution multiple times per solve:

x=self._solver.solution.x,
y=self._solver.solution.y,
prim_inf_cert=self._solver.solution.prim_inf_cert,
dual_inf_cert=self._solver.solution.dual_inf_cert,

Valgrind Output

My application with ~400k solves shows:

==3130== 50,680,976 bytes in 3,167,559 blocks are definitely lost
==3130==    at 0x483CF83: operator new(unsigned long)
==3130==    by 0x14D53CA21: PyOSQPSolver::get_solution() (bindings.cpp:169)

My Question

Am I missing something about how these objects are managed?

I'm hesitant to conclude this is a bug because:

• This code has been in use for a while
• There might be a cleanup path I'm not seeing
• Perhaps the reference policy has different semantics than I understand

Could someone clarify the intended lifetime management for these PyOSQPSolution objects? Is there a design reason for allocating them on the heap with new rather than returning by value?

Thank you!

[vineetbansal]

@XiDianZuoYun - I think you're absolutely right in your analysis. Since we're allocating a new PyOSQPSolution on every get_solution(), we should be taking ownership of it so it gets deallocated. Since its a property accessed multiple times however, as you noticed, there might be a better way though.

Thanks for your investigative work on this. I'll propose a solution and send a PR over the weekend.

[vineetbansal]

@XiDianZuoYun - I"ve sent a PR for this here. We'll get it reviewed internally but do take a look to make sure it makes sense. If there are unrelated CI errors because of updated github images or cibuildwheel changes (inevitable!), it might take a couple of days to get it merged, but regardless, you can do pip install git+https://github.com/osqp/osqp-python.git@vb/issue198 to build/install from that branch and see if your valgrid test now passes.

I've tested out the solution on my own scripts using valgrind and everything seems okay after this fix. In a future PR we'll make the memory leak check part of our CI.

[XiDianZuoYun]

Thank you for your prompt response and the bug fix. I have verified it in my local environment, and the issue has been resolved.