Metrics for machine learning

[shark8me]

Hi all,

How are you doing?

I was wondering if we could collaborate on a library for specifically for evaluation metrics in Machine Learning. In the confuse library, we currently have implemented the following binary and multi-class evaluation metrics.

Counts

• True positives
• False positives
• False negatives
• True negatives

Binary class Metrics

• sensitivity / specificity / true positive rate
• specificity / true negative rate
• false positive rate
• false negative rate
• precision
• f1-score
• accuracy
• misclassification-rate
• confusion matrix

Multi-class metrics

• Micro averaged fmeasure/precision/recall
• Macro averaged fmeasure/precision/recall

I observed that cortex implements metrics in a different name space too.

If the idea of moving all evaluation metrics to a separate library is acceptable, we could help migrate & test Cortex code that uses these metric APIs.
I believe it will be helpful for the larger Clojure data science community as there is currently no Clojure library (to my knowledge) that provides these APIs.

Would you like to hear what do you folks think about this.

regards
Kiran

[cnuernber]

This sounds reasonable. We are about to include a solid chunk of metrics that involve bounding boxes of things which could be a nice addition to the library you are talking about.

@CalderBot: FYI.

[cnuernber]

We could perhaps extend this concept to include:

1. Vectorized nms type algorithms:
   https://github.com/kpzhang93/MTCNN_face_detection_alignment/blob/master/code/codes/MTCNNv2/nms.m

Note that nms requires sort and filtering operations.

2. Intersection over union, intersection over min, both used in the the above face detection system.

[shark8me]

Thanks!
Those features would be a useful addition :)