Münchhausen reward not matching paper?

[slerman12]

It looks like the blue color-coded part of Equations (1) and (2) in the main paper are not included in this implementation
https://arxiv.org/pdf/2007.14430.pdf