AUC metric

Hello community,

Someone already implemented or tried to create the AUC (ROC CURVE) metric from the NLU models? which steps are necessary to extract that metric, i’ve read about the f1-score, recall and precision but that`s not so clear to me, iam trying to create this kind of metric for a research article.

Thank you, best regards, keep the good work.:slightly_smiling_face:

hi @mgcostaParedes - are you talking about intent classification performance or entity recognition? The ROC curve is usually for binary problems, so if you have more than two intents you will have to come up with a definition that suits you, see for example Receiver Operating Characteristic (ROC) — scikit-learn 0.21.3 documentation