How Zero-inflated Datasets Can Ruin Your Regression Modeling

...and here's how you can prevent it.

The target variable of typical regression datasets is somewhat evenly distributed.

But, at times, the target variable may have plenty of zeros. Such datasets are called zero-inflated datasets.

Zero-inflated target distribution

They may raise many problems during regression modeling. This is because a regression model can not always predict exact “zero” values when, ideally, it should.

For instance, consider simple linear regression. The regression line will output exactly “zero” only once (if it has a non-zero slope).

Simple linear regression fit outputs zero only once

This issue persists:

• Not only in higher dimensions...

• But also in complex models like neural nets for regression.

One great way to solve this is by training a combination of a classification and a regression model.

This goes as follows:

• Mark all non-zero targets as “1” and the rest as “0”.

• Train a binary classifier on this dataset.

Train a binary classifier

• Next, train a regression model only on those data points with a non-zero true target.

Train a regression model

During prediction:

Prediction phase

• If the classifier's output is “0”, the final output is also zero.

• If the classifier's output is “1”, use the regression model to predict the final output.

Its effectiveness over the regular regression model is evident from the image below:

Regression vs. Regression + Classification results

• Linear regression alone underfits the data.

• Linear regression with a classifier performs as expected.

👉 Over to you: What are other ways to train a model on a zero-inflated dataset?

👉 If you liked this post, don’t forget to leave a like ❤️. It helps more people discover this newsletter on Substack and tells me that you appreciate reading these daily insights. The button is located towards the bottom of this email.

Thanks for reading :)

---

Whenever you’re ready, here are a couple of more ways I can help you:

• Get the full experience of the Daily Dose of Data Science. Every week, receive two curiosity-driven deep dives that:

  • Make you fundamentally strong at data science and statistics.

  • Help you approach data science problems with intuition.

  • Teach you concepts that are highly overlooked or misinterpreted.

Daily Dose of Data Science ML articles

• Promote to 31,000 subscribers by sponsoring this newsletter.

---

👉 Tell the world what makes this newsletter special for you by leaving a review here :)

👉 If you love reading this newsletter, feel free to share it with friends!