to replicate:
import lmppl
scorer = lmppl.MaskedLM("microsoft/deberta-v3-small")
scorer.get_perplexity("to the 3rd")
Result:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
....
RuntimeError: Sizes of tensors must match except in dimension 0. Expected size 6 but got size 7 for tensor number 2 in the list.
to replicate:
Result: