loss is negative when running SCST

[1301358882]

Hello, when I reproduce scst, loss is negative. Is this normal? The final loss during mle training was around 45. The following is the screenshot of loss. Thank you very much.

[ch3cook-fdu]

Given the formulation of SCST loss, $\mathcal{L} = \sum (R(s_i) - R(b)) \frac{1}{|s_i|}\log P(s_i)$, where $(R(s_i) - R(b))$ and $P(s_i)$ can both be negative. Therefore, it is normal.

[1301358882]

Given the formulation of SCST loss,

𝐿

∑
(
𝑅
(
𝑠
𝑖
)
−
𝑅
(
𝑏
)
)
1
|
𝑠
𝑖
|
log
⁡
𝑃
(
𝑠
𝑖
)
, where
(
𝑅
(
𝑠
𝑖
)
−
𝑅
(
𝑏
)
)
and
𝑃
(
𝑠
𝑖
)
can both be negative. Therefore, it is normal.

Thank you very much.

[1301358882]

May I ask why we multiply it by 5 and then add it？

[ch3cook-fdu]

We chose the loss weight heuristically. Please refer to the loss function in our paper.

[1301358882]

We chose the loss weight heuristically. Please refer to the loss function in our paper.

ok, thank you.