Hello. Let me ask you a question.
I tried to build a BERT-LSTM model for your paper using Movie Reviews data, but I couldn't reproduce the paper results.
My results are as follows.
Training: train 0.925, validation 0.833, test 0.849
prediction:
Performance AURPRC comprehensiveness sufficiency
BERT-LSTM + Attention 0.829 0.463 0.223 0.141
BERT-LSTM + Simple Gradient 0.829 0.469 0.222 0.141
The performance in Table 4 of the paper is 0.974, and my result is 0.829, which is very different.
What I changed from the parameters listed in the README is that the predict batch size is 4 to 2 due to lack of memory.
My environment is as follows:
Memory 65G, GPU NVIDIA Tesla 32GB
Could you tell me if there are any parameter differences or any other differences from the paper experiments?
Hello. Let me ask you a question.
I tried to build a BERT-LSTM model for your paper using Movie Reviews data, but I couldn't reproduce the paper results.
My results are as follows.
Training: train 0.925, validation 0.833, test 0.849
prediction:
Performance AURPRC comprehensiveness sufficiency
BERT-LSTM + Attention 0.829 0.463 0.223 0.141
BERT-LSTM + Simple Gradient 0.829 0.469 0.222 0.141
The performance in Table 4 of the paper is 0.974, and my result is 0.829, which is very different.
What I changed from the parameters listed in the README is that the predict batch size is 4 to 2 due to lack of memory.
My environment is as follows:
Memory 65G, GPU NVIDIA Tesla 32GB
Could you tell me if there are any parameter differences or any other differences from the paper experiments?