Skip to content

Parallel download with pthreads#51

Open
Samathy wants to merge 2 commits intomlj:masterfrom
Samathy:parallel-download-pthreads-no-ac-upgrade
Open

Parallel download with pthreads#51
Samathy wants to merge 2 commits intomlj:masterfrom
Samathy:parallel-download-pthreads-no-ac-upgrade

Conversation

@Samathy
Copy link

@Samathy Samathy commented Apr 17, 2021

Downloading the back-issues of podcasts one at a time takes a very long time when there are 1000s of episodes and is trivially parallelisable.
This PR uses pthreads to download everything concurrently, it is much, much faster.

This is a bit hacky and neieve and could probably do with being cleaned up a bit.
But I really only needed this functionality once.
I'm happy to make changes, if anyone is interested in merging this functionality.

Issues:

  • To download N back issues, it just creates N threads, better to specify X threads and split N downloads across them
  • Progress bar doesnt work, disable or fix for parallel downloads.
  • Resuming is presumed to not work, its disabled and I havent tested it.
  • I havent tested catching up, only downloading back issues.
  • I made no effort to ensure there are no race conditions, which means there probably are ( even though it appears to work fine ).
  • Users can't turn off parallel downloads, they should probably be able to do that.

Samathy added 2 commits April 16, 2021 16:05
Spin up as many threads as there are podcasts to download -
which is potentially a LOT of threads.

Also, progress bars don't work.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant