Will the usage of LSTM have an impact on Transformer parallelism?

[chaoyu1999]

[chaoyu1999]

Because LSTM works serially, will the introduction of LSTM affect the speed of Transformer?

[BriansIDP]

Hi. It will, but the influence is very limited because we only inserted LSTM in one or two blocks and the size of the LSTM layer is small as described in our paper. if you run the code and compare the efficiency, the increase in time per minibatch is negligible compared to other computations.