Add multiprocessing capability to make_image.py#9
Add multiprocessing capability to make_image.py#9jadball wants to merge 2 commits intoFABLE-3DXRD:masterfrom
Conversation
| """ | ||
| Multithreaded version of self.make_images | ||
| """ | ||
| from multiprocessing import Pool |
There was a problem hiding this comment.
multiprocessing was imported above?
There was a problem hiding this comment.
Other than that it looks promising to me. A few things to check:
- Can it run under python27 still ?
- Does it work for both fork and spawn based processes (e.g. linux vs mac+windows) ?
- Could there be a simple testcase script that runs some cases in test/ ands checks the results do not change ? For example, the md5 hashes for the created frames match each other.
If you are confident we can merge now and fix a problem if someone finds one
Thanks!!!
|
Hi @jonwright, I am seeing it now :-) I had a look at @jadball's multiprocessing capability - and from a look at the code it looks fine. Unfortunately, I am quite new to this Github universe and is far from mastering it, so I haven't managed to "checkout" this new version for a test yet. Hopefully I will manage shortly and can thereafter merge @jadball's fork into master. PS: Thanks @jadball for helping out improving the code 🙏 |
osholm
left a comment
osholm
left a comment
There was a problem hiding this comment.
More descriptive error messages are much in line with the original idea of this feedback to the user. So here it comes.
3 participants
Added multiprocessing capability to make_image.py
Moves everything after
for i in no_frames:into a separate function (make_one_frame).As each frame is independent (unless peakshape is 2) the
make_one_framefunction, it usespoolcontextfrom here to run the function on multiple processes.