[FEATURE] Unifying classification aggregators interfaces

[alexdremov]

Problem description

I wanted to test quality metrics of several different algorithms from crowdkit.aggregation.classification and found myself writing such kind of function:

def get_scores(model, data, fit=True):
    if fit:
        model.fit(data)
    probas = getattr(model, "probas_", None)
    if probas is not None:
        return probas
    predictor = getattr(model, "predict_score", None)
    if predictor is None:
        predictor = model.predict_proba
    return predictor(data)

That's because different models have different methods for retrieving scores. For example, MMSR has predict_score while almost all others have predict_proba. Some have field probas_ , while others don't.

This seems strange and inconsistent.

Feature description

Unify naming of predict_score functions and presence of probas_ field

[pilot7747]

Thanks for the issue! Will simply adding predict_proba for M-MSR solve this? I assume we can just pass the scores through the softmax but I don't remember what these scores mathematically mean, need to read into the paper.

[alexdremov]

Thanks for the issue! Will simply adding predict_proba for M-MSR solve this? I assume we can just pass the scores through the softmax but I don't remember what these scores mathematically mean, need to read into the paper.

Yes, if all agreggators will have predict_proba, the issue should be considered solved. Also, presence of probas_ field should be made consistent, as it was confusing for me why some models have it, and others don't

Also, KOS aggregator does not have any probas at all. Is it algorithm's limitation or this can be added?

[dustalov]

KOS does not operate with probabilities explicitly, but we can always throw NotImplementedError if we find no reasonable workaround.