Questions tagged [rms]
RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)
342 questions
1
vote
1
answer
52
views
ordParallel() errors in rms::orm models: rfort(theta) : NA/NaN/Inf and argument is of length zero depending on sample size / model complexity
I edited the question to open it again because my main question is how to solve this issue statistically since I cannot interfere with the numerical analysis of the program.
I’m working with ordinal ...
2
votes
2
answers
87
views
Interpreting Shapley values for variance decomposition?
I trained a SVM multiple regression model and want to know how much each feature contributes to the prediction variance (quantified by the RMSE). I got the Shapley values for each feature on data from ...
0
votes
0
answers
25
views
Estimating the contribution of each feature to regression model prediction variance
I have trained a multiregression model using non-linear SVM, and got quite good metrics, with no big differences between test (20% data) and train (80% data) metrics.
The following are the test/train ...
1
vote
0
answers
20
views
How to further reduce MSE/RMSE in CatBoost after tuning, data augmentation, and outlier removal? [duplicate]
I am using the CatBoost model with 100 data points and I have done data augmentation, hyperparameter tuning,cross validation, added isolation forest, and randomizedsearchCV and at the end I could have ...
0
votes
0
answers
51
views
Calculating Standard Deviation of RMSE of an unsupervised algorithm
If there is an ML model, the standard deviation (SD) of the root mean squared error (RMSE) can be calculated using time series splits by fitting the model on different training sets and evaluating it ...
0
votes
0
answers
50
views
How to Compute a Scale-Invariant Reconstruction Error for PCA?
I am working with Principal Component Analysis (PCA) and trying to evaluate reconstruction error. Specifically I am interested in being able to compare the results of PCA on differently scaled data (...
0
votes
1
answer
101
views
How to depict RCS using repeated measured data in linear mixed effect model?
I am using "lmer" package to fit linear mixed-effects model to access the association of Y and X. The response variable of my data is continuous and repeated-measured which requires random ...
4
votes
2
answers
355
views
Concordance index in survival analysis (Gonen and Heller)
I am working on a project to externally validate a clinical prediction model. The original model coefficients were estimated using a Cox model. The model uses the baseline hazard and coefficients to ...
1
vote
1
answer
86
views
Singular information matrix in lrm.fit when only modelling one variable
I am I am trying to fit a logistic regression to my dataset with the variable binary as the response variable of the date ...
0
votes
0
answers
67
views
Is there any value in models that have a larger out of sample RMSE than a standard deviation?
I am predicting y values from x values using various regression models, elastic net and partial least squares regression (PLSR). To quantify performance of models we utilize root mean squared error (...
0
votes
0
answers
118
views
rms::val.surv function estimated the same survival probability for all cases
I have fit a cox regression model, and used val.surv function to plot calibration plot to compare observed survival probability with predicted survival probability.
...
1
vote
0
answers
27
views
Testing the difference between two Root Mean Square Error values for statistical significance [duplicate]
I would like to compare the predictive power of 2 models. The models are meant to model count data and respective probabilities. I am using two metrics as means of comparison:
Root Mean Square Error
...
2
votes
1
answer
74
views
Is Mean Square Prediction Error acceptable to use if predicted values are continuous but actual observed values are discrete?
I would like to compare the predictive power of 2 models. The models are meant to model count data, so the actual observed values are discrete. However both models are designed such that they output ...
1
vote
1
answer
334
views
Regression spline for time to allow for slope changes
Suppose we have a regression / survival model where we would like to model follow-up time using a regression spline. Follow-up time has two phases (first treatment active, and second treatment ...
4
votes
1
answer
305
views
Approximation function for MLP and LSTM
I have total of 6300 samples, 5800 of which are training data, and 500 of which are testing data. We compare the performance of LSTM and multilayer perceptron (MLP) with one hidden layer in terms of ...
0
votes
0
answers
57
views
How to compute the confidence interval of root mean square (e.g., ci of the RMS of the alternating current signal)
If $\bf x$ are $N$ measurement of alternating current signal (sinusoidal), its root mean square is computed using $RMS=\sqrt{\frac{\sum{x_i^2}}{N}}$.
My question is: Does the confidence interval exist ...
6
votes
1
answer
291
views
Valid confidence intervals in GAM’s using shrinkage estimation
In this blog article:
https://www.fharrell.com/post/improve-research/
it states:
“The frequentist paradigm does not provide confidence intervals or p-values when parameters are penalized”.
I was ...
1
vote
0
answers
68
views
Observational pre post design and confounder adjustment
I have observational data with baseline, and two follow-up measures with a binary treatment. Dependant variable is questionnaire scale score ***. I am planning on fitting a linear mixed effects model ...
6
votes
3
answers
539
views
Cox model - unsure of time unit of analysis
I am running a survival analysis (Cox model) on time to event in cancer patients. The start of followup is end of treatment. Tests for the event (recurrence) are performed every 6 months from cancer ...
0
votes
0
answers
89
views
Question about LASSO, RMSE, and Standardization
I have a question about doing LASSO in R using glmnet. It's kind of a conceptual question; I learned that we should interpret RMSE after performing ...
3
votes
1
answer
194
views
Shrinkage of covariates in the Cox model
In a regression model (e.g Cox model) when there are too few events to support modeling all desired covariates / confounders, a possible solution is to apply shrinkage / penalise all but the exposure(...
3
votes
2
answers
483
views
Machine learning benchmarks: MAE, RMSE, and R-squared
I'm working on a machine learning problem, and I'm having trouble interpreting different measures of model performance. I have a single dependent variable (proportion change between two treatments, ...
3
votes
1
answer
434
views
Restricted cubic splines for time-to-event data
I'm kind of new in fitting rcs for cont. variables as a clinican, so I have a few questions:
...
1
vote
0
answers
91
views
How to compute relative error of multi-dimensional time-series?
I have written a python script that uses a variety of different integrators to simulate the gravitational N-body problem. I would like to compare the positions obtained from my simulation to the ...
1
vote
0
answers
41
views
Poor RMSEA/Fit for Simple Poisson Regression
I am running a simple Poisson regression. $X$ = time, $Y$ = count data. This is a huge dataset with many years. There is significance between $X$ and $Y$. But model shows poor fit via high RMSEA value....
1
vote
1
answer
156
views
How to construct an optimal spline model when two continuous independent variables are included
I am interested in evaluating the relationship between age, BMI and lipid level. The lipid level is an outcome in my study. I think that the relationship between lipid level and age and the ...
1
vote
1
answer
74
views
Comparing families of classifiers on large datasets using mixed effect logistic regression models on individual questions
I have a testing dataset of about 6000 images which I am going to try about 25 different neural networks on in a multi-class classification problem. Each network will belong to around 5 families (e.g. ...
1
vote
1
answer
211
views
Can you compare regression models using RMSE when samples have different proportions of zeros?
I am using the ranger package (which implements random forests) in R to build regression models of tree species' basal area, a continuous measure of abundance and ...
0
votes
0
answers
75
views
Compare Root Mean Square Values
I'm trying to compare a regression neural network to a commonly used equation. I have an 80:20 split for my training:test, and I get the root mean square error on the test set from the neural network ...
4
votes
1
answer
275
views
Why isn't there a square root version of the Brier score similar to how RMSE complements MSE?
When computing the mean squared error of a regression model, we get a metric in square units. For ease of interpretation, we can therefore instead compute the root mean squared error, which are in ...
0
votes
0
answers
65
views
RMS residual normalized by standard-deviation
Is there a proper name for the following misfit quantification?
misfit=√{∑[(xi−xi')^2/(n*𝜎i^2)]}
where n is the number of data points xi−xi' is the ith residual, ...
1
vote
1
answer
1k
views
How to interpret interaction effect in ordinal (logistic) regression?
I ran an ordinal regression in R with the polr function from the MASS package as described in this tutorial, which is very good. However, the tutorial does not ...
1
vote
0
answers
302
views
Cox model with ridge term: how to choose value of theta?
The coxph() function in R package "survival" is used to fit the Cox proportional hazards model. This function allows a ridge() term in the formula to penalise selected terms, which requires ...
0
votes
1
answer
135
views
Root Mean Square Error of the addition of two measurements whose RMS Error is known
I am working on a measurement system which tries to measure the distance between two values i.e $\Delta F=F_1-F_2$. Where $F_1, F_2$ are the values I actually measure. I have set up a Monte Carlo ...
1
vote
1
answer
998
views
Can I say that the relative root mean squared error is the averaged percentage error?
RMSE is an error metric in which the mean of the data minimizes its loss function:
$\text{RMSE} = \sqrt{\frac{\sum_{t=1}^{n}(y_t - \hat{y_t})^2}{n}}$
But it gives ...
3
votes
0
answers
198
views
Longitudinal RCT modeling of continuous time
I have data from an intervention study (10 clinics, 5 control, 5 treatment). The outcome is counts, and we have monthly data at baseline, treatment active phase, and post treatment phase. The number ...
1
vote
1
answer
119
views
Should the RMSE of an unrestricted VAR model decrease as compared to a restricted Autoregression model when there is Granger Causality
I have 2 time series, say for instance, T1 and T2. T1 granger causes T2 at lag 2. Should this mean that if I make a VAR model with these two time series, and an autoregression model with just T2, the ...
1
vote
1
answer
128
views
Should the RMSE of the unrestricted (VAR) model for a time series that is being Granger caused by another be lesser than its restricted counterpart?
I have a couple of time series, say, T1 and T2. I have established (using the grangercausalitytest library of Statsmodels in ...
0
votes
1
answer
267
views
Comparing RMSE values across different datasets
I am working on a PV energy production forecasting problem. With various ML models (ANN, RNN, LSTM) I am trying to predict the energy for the following day, based on the historical data.
The ...
1
vote
1
answer
325
views
Is there an error metric that decreases the weight when the target is near zero?
As precipitation prediction models can only predict positive values, they won't be able to undershoot small values by much. When it comes to overshooting, there is no boundary. High precipitation ...
2
votes
1
answer
176
views
RMSE model interpretation
Let's say I train a model and it has an RMSE of 2.5.
Does this mean, that on average, my prediction will be 2.5 away from the true value? Or does some scaling need to be done in oreder to get this ...
0
votes
1
answer
1k
views
Why the results of Cox regression are different between coxph() and cph() in rms package
I found the predicted hazard (the h(t) of Cox regression) through Predict() and cph() in rms package was different from common coxph().
...
0
votes
0
answers
42
views
Obtaining 12 month ahead in sample RMSEs
I am attempting to recreate the results of the paper written by King, Stock and Watson in 1995: Temporal instability in the unemployment inflation relationship.
The paper estimates a VAR model with 12 ...
1
vote
1
answer
316
views
Cox model: how to model treatment variable when timing is unknown
We have cancer medical registry data, including information on date of diagnosis, treatment, and followup e.g. date of death etc. However, we only know type of treatment received for each person. We ...
2
votes
1
answer
1k
views
Restricted cubic spline looks like a linear curve, but p for nonlinear < 0.001
I am analyzing a association between a frailty index and care needs using the cox model. I use R and use rms package to fit restricted cubic spline.
This is my R code.
...
0
votes
0
answers
190
views
ROC with bootstraping
I have a data with 2 variables: diagnosis- yes/no
Score- numeric variable from 0-10.
I need to do ROC analysis for this data and to find the best cut off values.
The problem is the data is too small ...
1
vote
2
answers
195
views
Get the R2/RMSE for each category of a dataset
This might be a dumb question !
I built a model and I'm satisfied enough with the model, given that I have a dataset with categorical variables I wanted to see the R2/RMSE for each of those categories,...
2
votes
1
answer
108
views
How do you interpret the value of RMSE/MSE in English to stakeholders?
For example, if you have a R^2 of 0.95, you can explain this number to stakeholders in a presentation as:
Our model explains 95% of the total variance within the data.
However, if we have a RMSE of 11,...
1
vote
1
answer
218
views
Why RMSE and MAPE changes with the change of axis?
I need your help regarding the information inside the picture. As you know all the information will change with the change of NDVI axis from y-axis to X-axis, except R2 and p-value remain the same? ...
1
vote
1
answer
795
views
How do we relate RMS and standard deviation for continuous signals?
Because the discrete formula for RMS, $\displaystyle X_{RMS}=\sqrt{{1 \over N}(x[1]^2+x[2]^2+...+x[N]^2)}$, is almost the same as the formula for standard deviation (assuming mean zero), except for a ...