5
$\begingroup$

Suppose I have a dataset comprised of garbages. Will a model perform better if I only label the dataset with biodegradable or non-biodegradable?

Or will it be better if I label them with plastics, paper, cardboard, glass, and organic?

In fact, I also plan to further increase the number of labels for example, the plastic label with be comprised of a large number of brands of plastic wrappers, etc.

I think that having a large number of labels is detrimental to computational performance in both training and evaluation since for example, Linear Discriminant Analysis will not lead to much reduced dimensionality due to the nature of the subspace spanned by the centroids on each label.

Neural networks would have a very wide top softmax layer and I am pretty sure that would require a wider architecture.

Aside from model issues, I could easily suffer from class imbalances. Is there any merit to having multiple labels ?

Improve this question
$\endgroup$
2
  • $\begingroup$ What's the purpose of your model? Do you just need to identify if the garbage is biodegradable or not? $\endgroup$ Commented May 5, 2022 at 9:12
  • $\begingroup$ @Lynn yes. It is a biodegradable/non-bio task which I think can be extended to several labels under biodegradable/non-bio. The motivation is that, I think that same plastic wrappers from same brands must be clustered in a Gaussian fashion. I am thinking that classifying these Gaussians would be better than having a binary classification of bio/non-bio $\endgroup$ Commented May 5, 2022 at 10:21

2 Answers 2

Reset to default
1
$\begingroup$

If the data is clustered in the way you describe in your comment, then for the two-class problem you are likely to have a non-linear decision boundary. As linear models can't model non-linear decision boundaries, they are unlikely to perform well here. However, an appropriately sized non-linear model such as a neural network should be able to model this (for more on linear vs non-linear models see Paul Fornia's answer to "Linear vs Nonlinear Machine Learning Algorithms"). If the NN is wide and/or deep enough it will effectively learn the cluster groupings during training.

Correction (in response to @kjetil's comment): You could still use a linear model if the features can be transformed to a higher-dimensional space in which the decision boundary is linear (e.g. by using feature interactions or squaring, cubing etc features).

Transforming your data to a multi-class problem may give you multiple linear decision boundaries, which should help improve a linear model, if for some reason you are restricted to using linear models.

Your point on class imbalances prompted another thought. By separating the classes into sub-classes you may lose information that is common between the subclasses. For instance, consider classes of 'hard plastic' and 'soft plastic', which probably have similar values for some attributes. If they are separate classes the model may treat these shared attributes as "not useful" and ignore them. Conversely the model may place too much importance on differences between 'hard plastic' and 'soft plastic' which are not useful for bio/non-bio classification.

Improve this answer
$\endgroup$
2
2
$\begingroup$

Generally speaking, having classes that are not naturally ordered creates a more difficult prediction/probability estimation problem. When a binary problem turns into a multinomial/polytomous problem the task is more difficult as more model parameters exist, the effective sample size decreases, and the sample size required to achieve a given predictive reliability increases.

If some of the classes are connected, i.e., types of plastic, you can gain information and predictive reliability by using a Bayesian model that has prior distributions for parameters such that you borrow information (assume some similarity of effects) across connected categories. This reduces the effective number of parameters. Sometimes this can be done with Bayesian hierarchical random effects models.

When you can assume that all levels of the outcome variable are ordered, everything changes. The effective sample size increases and the number of parameters that matter does not increase with increasing number of outcome categories. Models such as the proportional odds ordinal logistic model come into play.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Good day professor, may I ask what you meant by "ordered classes"? For example, if the dataset are images of coins (the front and back of the coin have separate labels), are they more "unordered" compared to dataset of different animals? $\endgroup$ Commented Aug 20, 2022 at 11:34
  • 2
    $\begingroup$ Unlike your original setup, ordered classes would classes for which there is consensus among experts that the characteristic of interest is ordered by severity of outcome, impact of outcome, size, darkness, or some other metric. Many ordinal analysis methods exist for such dependent variables. $\endgroup$ Commented Aug 20, 2022 at 11:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.