Taking into account instance cost in learning?

We can redefine the loss so that each predicted probability is “distorted” by an exponential function of the associated costs, then optimize via gradient-based methods. Let

$$ p_i = \hat{y}_i = \sigma(\mathbf{x}_i^\top \boldsymbol{\theta}) $$

be the standard logistic prediction for instance $i$, with

$$ \sigma(u) = \frac{1}{1 + e^{-u}}. $$

Let

$$ \alpha = e^{-c_{1,0}}, \quad \beta = e^{-c_{1,1}}, \quad \gamma = e^{-c_{0,1}}, \quad \delta = e^{-c_{0,0}}. $$

We define the following “cost-distorted” loss:

$$ \mathcal{L}(\boldsymbol{\theta}) = -\frac{1}{N}\sum_{i=1}^{N} \Big[y_i\big(\alpha \log(p_i) + \beta \log(1 - p_i)\big) + (1 - y_i)\big(\gamma \log(p_i) + \delta \log(1 - p_i)\big)\Big]. $$

Minimizing this corresponds to simultaneously favoring predictions $p_i$ that yield lower exponential cost factors while still maintaining a log-based penalty. We can implement this by customizing the gradient descent step. The gradient with respect to $\boldsymbol{\theta}$ is:

$$ \nabla_{\boldsymbol{\theta}} \mathcal{L} = -\frac{1}{N} \sum_{i=1}^{N} \left[ y_i\left(\alpha \frac{1}{p_i} - \beta \frac{1}{1 - p_i}\right) + (1 - y_i)\left(\gamma \frac{1}{p_i} - \delta \frac{1}{1 - p_i}\right) \right] p_i (1 - p_i) \mathbf{x}_i. $$

Below is a minimal scikit-learn compatible code snippet illustrating how one might implement this custom loss for logistic regression:

import numpy as np
from sklearn.base import BaseEstimator, ClassifierMixin

class CostDistortedLogisticRegression(BaseEstimator, ClassifierMixin):
    def __init__(self, alpha, beta, gamma, delta, lr=0.01, max_iter=1000):
        self.alpha = alpha
        self.beta = beta
        self.gamma = gamma
        self.delta = delta
        self.lr = lr
        self.max_iter = max_iter

    def sigmoid(self, z):
        return 1.0 / (1.0 + np.exp(-z))

    def fit(self, X, y):
        self.theta_ = np.zeros(X.shape[1])
        for _ in range(self.max_iter):
            p = self.sigmoid(X @ self.theta_)
            grad = np.zeros_like(self.theta_)
            for i in range(len(y)):
                grad += ( - ( y[i] * ( self.alpha / p[i] - self.beta / (1 - p[i]) )
                              + (1 - y[i]) * ( self.gamma / p[i] - self.delta / (1 - p[i]) ) )
                          * p[i] * (1 - p[i]) ) * X[i]
            grad /= len(y)
            self.theta_ -= self.lr * grad
        return self

    def predict_proba(self, X):
        p = self.sigmoid(X @ self.theta_)
        return np.column_stack((1 - p, p))

    def predict(self, X):
        return (self.predict_proba(X)[:, 1] >= 0.5).astype(int)

In this construction, choosing $\alpha, \beta, \gamma, \delta$ as exponentials of the respective misclassification costs “tilts” the log loss so that errors whose costs are higher receive a proportionally larger penalty, creating a “differentiable distortion” of the usual logistic objective. In practice, one would plug in $e^{-c_{1,0}}, e^{-c_{1,1}}, e^{-c_{0,1}}, e^{-c_{0,0}}$ or other monotone mappings of $c_{1,0}, c_{1,1}, c_{0,1}, c_{0,0}$ and proceed with the same gradient-based optimization. This method, despite looking similar on the surface, is fundamentally different from standard class weighting: the exponential weighting yields a different geometry for the loss surface and can adapt more flexibly if one extends the approach to instance-specific or output-dependent costs, yet it remains compatible with scikit-learn pipelines and follows all the usual cross-validation or hyperparameter-tuning procedures.