Bug description
Evaluator class should return the N scores for N inputs, but it omits an output when an empty input is given.
How to reproduce
- Prepare the python environment
pip install pymarian
- Run the following codes:
from huggingface_hub import hf_hub_download as hf_get
from pymarian import Evaluator
model_id = "marian-nmt/bleurt-20"
model = hf_get(model_id, filename="checkpoints/marian.model.bin")
vocab = hf_get(model_id, filename="vocab.spm")
evaluator = Evaluator.new(model_file=model, vocab_file=vocab, like="bleurt", cpu_threads=8)
for score in evaluator.evaluate(["\t"]):
print(score)
It causes the assertion error:
Traceback (most recent call last):
File "<stdin>", line 10, in <module>
File "<path_to_pymarian>/pymarian/__init__.py", line 142, in evaluate
assert len(scores) == len(batch)
AssertionError
In this example, the len(batch) will be 1, but len(scores) will be 0.
Context
- Python version: Python 3.10.15
- Marian version: v1.12.31 dd3571de
Bug description
Evaluatorclass should return the N scores for N inputs, but it omits an output when an empty input is given.How to reproduce
pip install pymarianIt causes the assertion error:
In this example, the
len(batch)will be 1, butlen(scores)will be 0.Context