The Ultimate Categorization of Performance Metrics in ML

Regression and classification metrics in a single frame.

Performance metrics are used to assess the performance of a model on a given dataset.

Performance metric overview

They provide quantitative ways to test the effectiveness of a model. They also help in identifying the areas for improvement and optimization.

Why Performance metrics?

Typically, it is difficult to interpret ML models (especially deep learning models). It is difficult to understand the specific patterns identified by the model from the given data.

Performance metrics allow us to determine their performance by providing unseen samples by evaluating the predictions.

This makes them a must-know in ML.

Performance Metrics for Regression

Regression model

1. Mean Absolute Error (MAE):

   1. Measures the average absolute difference between the predicted and actual value.

   2. Provides a straightforward assessment of prediction accuracy.

2. Mean Squared Error (MSE):

   1. Measures the average squared difference between the predicted and actual values.

   2. Larger errors inflate the overall metric.

3. Root Mean Squared Error:

   1. It is the square root of MSE.

4. R-squared:

   1. Represents the proportion of the variance in the target variable explained by the regression model.

5. Adjusted R-squared:

   1. Similar to R-squared.

   2. But accounts for the number of predictors (features) in the model.

   3. Penalizes model complexity.

6. Mean Absolute Percentage Error (MAPE):

   1. Measures the average percentage difference between the predicted and actual values.

   2. Typically used when the scale of the target variable is significant.

Performance Metrics for Classification

Classification models

1. Accuracy:

   1. Measures the proportion of correct predictions, irrespective of the class.

   2. Provides an overall assessment of classification performance.

2. Precision:

   1. Measures the proportion of positive predictions that were correct.

   2. It is also called the accuracy of the model only on the positive predictions.

3. Recall (Sensitivity):

   1. Measures the proportion of correctly classified positive samples in the dataset.

   2. It is also called the accuracy of the model on positive instances in the dataset.

      ---

      Precision → Accuracy on positive predictions.

      Recall → Accuracy on positive instances in the dataset.

      Read by Medium blog to understand more: Precision-Recall Blog.

      ---

4. Specificity:

   1. Opposite of Recall.

   2. Measures the proportion of correctly classified negative samples in the dataset.

   3. It is also called the accuracy of the model on negative instances in the dataset.

5. Negative Predictive Value:

   1. Opposite of Precision.

   2. Measures the proportion of negative predictions that were correct.

   3. It is also called the accuracy of the model only on the negative predictions.

6. Balanced Accuracy:

   1. Computes the average of Recall (accuracy on positive predictions) and specificity (accuracy on negative predictions)

   2. A better and less-misleading measure than Accuracy in the case of an imbalanced dataset.

7. F1-score:

   1. The harmonic mean of precision and recall.

   2. Provides a balanced measure between the two.

8. Matthews Correlation Coefficient (MCC):

   1. Takes into account true positive (TP), true negative (TN), false positive (FP), and false negative (FN) predictions to measure the performance of the binary classifier.

   2. Provides a balanced performance measure unaffected by class imbalance.

      If you struggle to understand TP, TN, FP and FN, read my previous post: Use This Simple Technique To Never Struggle With TP, TN, FP and FN Again

👉 Over to you: What other metrics will you include here?

---

👉 Read what others are saying about this post on LinkedIn and Twitter.

👉 Tell the world what makes this newsletter special for you by leaving a review here :)

👉 If you liked this post, don’t forget to leave a like ❤️. It helps more people discover this newsletter on Substack and tells me that you appreciate reading these daily insights. The button is located towards the bottom of this email.

👉 If you love reading this newsletter, feel free to share it with friends!

👉 Sponsor the Daily Dose of Data Science Newsletter. More info here: Sponsorship details.

---

Find the code for my tips here: GitHub.

I like to explore, experiment and write about data science concepts and tools. You can read my articles on Medium. Also, you can connect with me on LinkedIn and Twitter.