Fix Critical Error While Loading Data#353
Open
AlonzoLeeeooo wants to merge 1 commit intoguoyww:mainfrom
Open
Conversation
Fix critical error in `dataset.py`, which might cause dead loop while running the code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi @huxiuhan ,
I've found some critical issues of this code when debugging it, where I propose this PR to fix it. Details are below:
Previous version of the
__getitem__inanimatediff/data/dataset.py,line 67-73:Fixed version:
The previous version of
dataset.pymight cause dead loop if theWebVid10Mdataset is not downloaded completely. Specifically, ifself.get_batch(idx)does not function well, the data loading process will keep excecutingidx = random.randint(0, self.length-1), untill theidxdata is available.This might cause much inconvenicence for two reasons: (1) If one is attempting to debug this code with a tiny subset of
WebVid, he might not need the entire dataset to do so; (2) The dead loop prevents us to see where the code issue truly lies.I would really appreciate it if you can accept this PR, since the issue has been addressed in a series of issues, particularly in #201, #352, #331. Thank you so much in advance for your precious time and attention from your busy schedule.
Best regards,
Chang