I’m not really sure that for logistic regression (and linear regression too) feature scaling is actually useful, unless the regression is regularized.
If the regression is not regularized, then the estimates of parameters will be scaled accordingly with the measurement scale of the feature: if I multiply the feature by 10, then the corresponding parameter estimate will be divided by 10; if I add some constant to the feature, then the intercept will change accordingly. Some info about this topic: https://stats.stackexchange.com/questions/48360/is-standardization-needed-before-fitting-logistic-regression
That's true. But this will only happen if you dump the entire data at once. For linear regression, for instance, if you use OLS, it will adjust its parameters and not hurt the accuracy at all. What I am considering here is gradient descent based classifier for logistic regression. Thanks for highlighting this, Enrico :)
What about LDA ? Do we need feature scaling there too?
No, it does not require feature scaling.
Glad to have found your Daily Dose. It helps in consistent revision!
Great 👍
I’m not really sure that for logistic regression (and linear regression too) feature scaling is actually useful, unless the regression is regularized.
If the regression is not regularized, then the estimates of parameters will be scaled accordingly with the measurement scale of the feature: if I multiply the feature by 10, then the corresponding parameter estimate will be divided by 10; if I add some constant to the feature, then the intercept will change accordingly. Some info about this topic: https://stats.stackexchange.com/questions/48360/is-standardization-needed-before-fitting-logistic-regression
That's true. But this will only happen if you dump the entire data at once. For linear regression, for instance, if you use OLS, it will adjust its parameters and not hurt the accuracy at all. What I am considering here is gradient descent based classifier for logistic regression. Thanks for highlighting this, Enrico :)
could you send me the PDF in French. I like this course
Hey. I have this PDF only in english :(
Hello