My data is RNA-seq data with more than 14000 features and the problem is binary classification. Then the total sample is 50 and p>>n. When I use Elasticnet method with train and test data, the Sensitivity is 0.6 and the accuracy is 0.5. When I use Elasticnet as a feature selection and then fit the random forest or deep learning to the selected features, the result is more better. for example, in random forest modeling, the sensitivity is 0.97 and for deep learning, the sensitivity is 0.84. So, My question is Would it be possible to use regularization methods as a feature selection method and then use machine learning models to analyses data in classification problem?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
First, accuracy is a highly problematic KPI, and sensitivity suffers from the exact same issues.
Second, with just 50 observations an 14,000 features, any variable selection method will be highly variable, and you are almost guaranteed to overfit. (Try re-running your feature selection while removing one observation. I very strongly suspect that your selected features will change quite a bit. Do this by deleting one observation at a time. Then try it deleting two observations.)
There is no magic trick to impart information where there is very little. You will need to either pare down your features based on theory, or collect much more data.
-
1$\begingroup$ Also note that classification is not usually a noble goal - see this. $\endgroup$Frank Harrell– Frank Harrell2024-07-21 12:34:57 +00:00Commented Jul 21, 2024 at 12:34
-
$\begingroup$ Thank you for the information. I did cross validation in all modeling that I considered above. The results are almost the same. Also let me know what is your idea about generating data from real data. Thank in advance for your help and time. $\endgroup$Leila ali– Leila ali2024-07-21 14:44:03 +00:00Commented Jul 21, 2024 at 14:44