1
$\begingroup$

I am working on a domain adaptation problem, where the default is a classification problem. I have worked exclusively with regression problems until now, so I am kind of thrown for a loop when it comes to understanding how ML algorithms handle the labels in classification problems. In regression, we use MSE loss with the output from our CNN and compare with the numeric label, how does this work in classification problems? The model outputs an array of numbers, so how cross entropy, and its variants, compare this array with a string label such as 'car', or 'unhealthy'. Is there a specific call to add to my CNN output to make this make sense?

For reference, i am trying to classify images into 3 categories.

Improve this question
$\endgroup$
1
  • $\begingroup$ in classification, the model outputs are probabilities of each class (look up softmax for neural nets), so your target might be [0,1,0] and the model predicts [0.2, 0.5,0.3] $\endgroup$ Commented Sep 2, 2022 at 15:40

1 Answer 1

Reset to default
2
$\begingroup$

All machine learning models need the data provided as numbers because what they do is build mathematical models. This means that you need somehow to encode all the non-numerical data (images, words, categories, sound recordings, etc) to numbers, this applies both to the features and the labels. To learn more, search for feature engineering. In the case of labels, we encode them as binary variables (binary, -1/+1), whereas for multiple categories we can use something like one-hot-encoding.

As for training the model, it's the same as regression: the model predicts some score and it is compared to the encoded label using a loss function. Squared error is a perfectly fine loss here, but there are also specialized ones like cross-entropy loss.

If this is new to you, I'd recommend that you start with some introductory course or handbook on machine learning to catch up.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you! I have set up a function that outputs strings of 'moderate', 'unhealthy', and 'hazardous' depending on a numeric value input to the function. If instead of the strings, i were to output 0, 1, or 2 respectively would this be sufficient? Or is there extra benefit of using the sklearn LabelEncoder for labels of 0,1,2? $\endgroup$ Commented Sep 2, 2022 at 14:58
  • $\begingroup$ @Scott no, such labels probably won't be a good approach in most cases. Is "unhealthy" higher by one "moderate", and "hazardous" two times higher than "unhealthy"? Unlikely the numerical values make sense. The feature is ordinal or categorical. It's too long to answer in a comment. As said, you should check an ML course or a handbook, because doing this before understanding the basics would be a bad idea. $\endgroup$ Commented Sep 2, 2022 at 15:08
  • $\begingroup$ Thank you! I'll dive into the sources you provided. I guess I was merely making sure that feeding strings into a loss functions was indeed incorrect despite most of the examples I see not showing how they handle the string nature of the labels. $\endgroup$ Commented Sep 2, 2022 at 15:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.