Questions tagged [ridge-regression]
A regularization method for regression models that shrinks coefficients towards zero.
33 questions
0
votes
0
answers
21
views
Runtime complexity of scikit-learn’s One-vs-Rest LogisticRegression (LBFGS) vs. RidgeClassifier
I’m working through the runtime analysis of scikit-learn’s OneVsRestClassifier for two cases:
LogisticRegression (solver=lbfgs, ...
2
votes
0
answers
72
views
Reverse engineering what stocks are in a dummy ETF using regression (lasso, ridge, etc) in Python
I'm trying to reverse engineer what stocks are in a ETF using python.
In my code, I create a fake ETF that is equal weighted 20 random stocks.
I then try to reverse engineer whats in my ETF using ...
- 29
1
vote
0
answers
52
views
Predicting PGA Tour results with Linear Regression
I have curated a dataset from various online sources that contains information about each PGA player's weekly performance/trends. I'm attempting to predict their finishing positions at the next ...
- 21
0
votes
1
answer
70
views
Scaling and PCA for test data before prediction
I'm fairly new to the world of ML & Data Science. I've completed a certification course in Coursera/IBM and I'm trying to hone my skills using some exercises from Kaggle. The course did not ...
- 1
3
votes
1
answer
290
views
regression model outperform every models
I followed from this question.
Case1:
I have the following task: Train for consecutive 3 days to
predict each fourth day. Each day's data represents one CSV file,
which has dimensions 24x25. Each ...
- 95
7
votes
1
answer
336
views
When the regression models outperforms naive method?
I followed from this question.
Case1:
I have the following task to do: Training by the consecutive 3 days to predict the each 4th day. Each day data represents one CSV file which has dimension 24x25. ...
- 95
1
vote
1
answer
83
views
Overfitting on the validation set
I’m working on a machine learning project where I have two datasets: X (features on individuals) and y (binary predictions: -1 or 1). My goal is to predict y based on the features in X.
Here’s a brief ...
- 11
0
votes
1
answer
94
views
With ridge regression, weights can approach 0 for large values of lambda but will never equal 0 (unlike Lasso). Why?
I've been trying to figure out why Ridge regression has weights approach 0 for large values of lambda but they are never equal to 0, unlike Lasso and Simple Linear Regression.
According to this ...
0
votes
0
answers
88
views
Help with multinomial logistic regression
I am a data science student and have the opportunity to work on an article regrading cardiac arrests in our country. For now I performed the multinomial regression model and I also plan on doing a ...
- 1
1
vote
0
answers
37
views
With infinite observations, would the weights resulting from ridge regression be the same as simple linear regression?
As the number of observations approaches infinity, do the weights of a linear regression approach the weights of a linear regression with L2 penalty?
- 111
0
votes
1
answer
35
views
How to extract MSEP or RMSEP from lassoCV?
I'm doing lasso and ridge regression in R with the package chemometrics. With ridgeCV it is easy to extract the SEP and MSEP values by ...
- 5
0
votes
1
answer
333
views
Why are my ridge regression coefficients completely different from ordinary linear regression coefficients in MATLAB?
I am attempting to implement my own Ridge Regression algorithm and I am trying to achieve similar coefficients found in a MATLAB tutorial on regression.
Specifically, on the MATLAB tutorial page you ...
- 101
1
vote
1
answer
488
views
What's the correct cost function for Linear Regression
As we all know the cost function for linear regression is:
Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
- 203
1
vote
2
answers
143
views
What (linear) model is common practice to use on sample size of 500 with 26 features?
I have a training data set of 500 people and 26 features and I'm trying to develop a regression model. A possibility is to derive more features of course. I'm considering the following models:
Linear ...
- 11
1
vote
1
answer
1k
views
The effect of the λ in the Ridge regression
Why by increasing value of λ in Ridge estimator the slope of the line is decreasing? How exactly λ affects to the y = kx + b?
- 11
3
votes
2
answers
5k
views
Constraining linear regressor parameters in scikit-learn?
I'm using sklearn.linear_model.Ridge to use ridge regression to extract the coefficients of a polynomial.
However, some of the coefficients have physical ...
- 31
1
vote
1
answer
90
views
Why we take $\alpha\sum B_j^2$ as penalty in Ridge Regression?
$$RSS_{RIDGE}=\sum_{i=1}^n(\hat{y_i}-y_i)^2+\alpha\sum_{i=1}^nB_j^2$$
Why we are taking $\alpha\sum B_j^2$ as a penalty here? We are adding this term for minimizing variance in Machine Learning Model. ...
- 19
-1
votes
1
answer
126
views
Lack of standardization in Kaggle jupyter notebooks when using lasso/ridge?
I've recently started using Kaggle, and I've noticed that for a lot of these jupyter notebooks written by others, when they use Ridge/Lasso, they don't standardize the non-categorical numerical ...
1
vote
1
answer
612
views
Do the benefits of ridge regression diminish with larger datasets?
I have a question about ridge regression and about its benefits (relative to OLS) when the datasets are big. Do the benefits of ridge regression disappear when the datasets are larger (e.g. 50,000 vs ...
- 143
1
vote
2
answers
1k
views
What is the meaning of the sparsity parameter
Sparse methods such as LASSO contain a parameter $\lambda$ which is associated with the minimization of the $l_1$ norm. Higher the value of $\lambda$ ($>0$) means that more coefficients will be ...
- 541
2
votes
3
answers
1k
views
how Lasso regression helps to shrinks the coefficient to zero and why ridge regression dose not shrink the coefficient to zero?
How does Lasso regression help with feature selection of model by making the coefficient shrink to zero?
I could see few below with below diagram. Can any please explain in simple terms how to ...
- 1,521
1
vote
0
answers
28
views
what other metrics can i use to estimate quality of the model predicting income range - interval estimation task?
I trained a model that predicts customer's income given the features:
age, declared income
number of oustanding instalment, overdue total amount
active credit limit, total credit limit
total amount
...
- 111
2
votes
2
answers
5k
views
How do standardization and normalization impact the coefficients of linear models?
One benefit of creating a linear model is that you can look at the coefficients the model learns and interpret them. For example, you can see which features have the most predictive power and which do ...
- 318
1
vote
1
answer
153
views
Why is Regularization after PCA or Factor Analysis a bad idea?
I have done Factor Analysis on my data and applied various machine learning models on it. I particularly find it giving high MSE value for Ridge and Lasso Regression compared to other models. I want ...
- 23
1
vote
1
answer
1k
views
How is learning rate calculated in sklearn Lasso regression?
I was applying different regression models to Kaggle Housing dataset for advanced regression. I am planning to test out lasso, ridge and elastic net. However, none of these models have learning rate ...
- 35
1
vote
1
answer
854
views
Is there a reference data set for ridge regression?
In order to test an algorithm, I am looking for a reference data set for ridge regression in research papers. Kind of like the equivalent of MNIST but for regression.
- 23
4
votes
2
answers
2k
views
Extremely high MSE/MAE for Ridge Regression(sklearn) when the label is directly calculated from the features
Edit: Removing TransformedTargetRegressor and adding more info as requested.
Edit2: There were 18K rows where the relation did not hold. I'm sorry :(. After ...
- 197
3
votes
1
answer
726
views
Does ridge regression always reduce coefficients by equal proportions?
Below is an excerpt from the book Introduction to statistical learning in R, (chapter-linear model selection and regularization)
"In ridge regression, each least squares coefficient estimate is ...
- 205
1
vote
1
answer
229
views
Dividing the weights obtained on an already standardized data set by the standard deviation of the features? (Ridge regression)
I'm trying to understand a code snippet from my lecture on Machine Learning (see the code below).
It extracts the mean and standard deviation of the features and uses them to 'normalize' (...
- 133
3
votes
3
answers
9k
views
Can ridge regression be used for feature selection?
I'm trying to figure out whether using Ridge Regression for regularization can be used to cause a more sparse hypothesis however to me it seems like ridge will never actually bring any coefficients to ...
- 197
3
votes
1
answer
226
views
Does it matter whether we put regularization parameter ($C$) with error or weight term in Kernel ridge regression?
Kernel ridge regression associate a regularization parameter $C$ with weight term ($\beta$):
$\text{Minimize}: {KRR}=C\frac{1}{2} \left \|\beta\right\|^{2} + \frac{1}{2}\sum_{i=1}^{\mathcal{N}}\left\|...
- 311
1
vote
1
answer
364
views
How to improve Regression Model with High Training Performance and Low Test Performance
I am performing regression analysis on some data. I keep getting very high training score and low test score. My code is below, what can i do to enhance it? Thank you in advance.
...
- 725
4
votes
2
answers
4k
views
What does a negative coefficient of determination mean for evaluating ridge regression?
Judging by the negative result being displayed from my ridge.score() I am guessing that I am doing something wrong. Maybe someone could point me in the right ...
user29169