DDP: Implementing DDP for Imagenet by ksreenivasan · Pull Request #32 · ksreenivasan/pruning_is_enough

ksreenivasan · 2022-04-26T02:30:49Z

No description provided.

I think it's just slowing things down and is known to cause memory leaks

…lla prints

this might nearly be all that we need for GM if things work nicely with affineBN and bias

…sion model

otherwise ddp complains about unused parameters. weird.

in that case, the issue might just be tuning in imagenet

ksreenivasan and others added 30 commits April 25, 2022 20:57

adding code to make ddp work

6334d1f

trying out how args work in ddp

9e1f0de

trying out stuff with parser_args in ddp setting

f616ab6

adding ddp_utils

e6b93a6

adding ddp_utils

f202a42

looks like reimporting doesn't hurt the global values

513b8a6

adding more code to implement ddp

02a6627

testing ddp with deepcopy

7f1e0f6

looks like copy.deepcopy causes sync issues

6cc557b

fixing typo in ddp deepcopy

dd0ba73

finishing up ddp for sanity checks and finetune

1d21c5a

minor bugfixes for ddp

691cffa

removing .cuda() from the codebase

ffca648

trying to catch the rank=0 leak

e1c9810

I think ddp works!

64cbf57

skipping checkpoint, trying to get rid of tqdm

eb12a2d

trying to eliminate tqdm and progress meters

f4fbff0

I think it's just slowing things down and is known to cause memory leaks

trainer doesn't see parser_args

69048e0

still removing leftover calls to tqdm and progresmeters

3834b3d

removing more instances of progressmeter and replacing them with vani…

3ba1f93

…lla prints

fixing dir creation bug

77fbe48

adding config for serial debug

1ffc000

stuff works, removing an unnecessary barrier

53b90c0

adding imagenet configs and exec script

10b1ee4

setting find_unused_parameters=False and adding config for sp10

38550af

pushing config for sp15 imagenet

50805e3

minor tweaks to configs for imagenet

fb1eab0

adding port as param for multiple runs

5ccc7a3

updating port to be a str not int

9052870

pushing psutil memory logs

2e17a43

ksreenivasan added 30 commits May 14, 2022 01:40

adding round_all_ones before switch_to_wt

04be2b8

trying gm with switch_to_wt to sanity check the code

d2dbd2b

making sure we save model.module not DDP model itself

910486c

adding switch_to_prune() and trying kaiming normal init for wt training

dc77a55

adding prune(), get_sparsity etc.

f1233e1

this might nearly be all that we need for GM if things work nicely with affineBN and bias

adding model strings to make comparing easier

5b447a2

updating model with bias=False and affine bn

9cf9105

trying to mimic torchvision resnet50

c416362

changing last layer to linear

51594fc

minor bugfixes adding bias to final fc layer

8d52ef8

plugging in subnetconv model into original script to compare

63f38ce

minor bugfixes and adding weight init to builder

0b6cfdc

adding round_model to complete comparison between mymodel and torchvi…

d263960

…sion model

it works! looks like the models are near identical now.

19d7044

changing model back to signed_constant for GM

3a7ad5e

turns out you need to switch_to_wt before instantiating ddp

cfd62cb

otherwise ddp complains about unused parameters. weird.

adding finetune feature

d9743ec

finetuning seems to work well!

188735d

adding save_model before finetune and bugfixes

ccf2867

trying cifar10 on ddp to debug

9d11bb3

changing model to resnet20

7a933c9

adding get_layers for resnet20

873f1a6

minor typo for cifar10 ddp, looks like it works

a276ffd

in that case, the issue might just be tuning in imagenet

print reg_loss only for pruning

782a85f

adding optimizer and results subdir for hyperparam opt

b139623

minor typo with opt

5c35b32

adding mkdir for subfolder

f6116fe

updating run script

e1c68d3

minor changes. for cifar10 testing i think

20b98ce

Merge branch 'ddp' of github.com:ksreenivasan/pruning_is_enough into ddp

6a053b7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DDP: Implementing DDP for Imagenet#32

DDP: Implementing DDP for Imagenet#32
ksreenivasan wants to merge 119 commits intomasterfrom
ddp

ksreenivasan commented Apr 26, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ksreenivasan commented Apr 26, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants