All metrics will be weighted by the variable {{mlTaskDesign.weight.sampleWeightVariable}}, both for hyperparameters optimization and final evaluation of the model.
The settings define how each row in the train set is weighted:
- No weighting: all rows will be considered equally.
- Sample weights: row weights are defined by a column of the dataset (They must be positive).
- Class weights: row weights are defined as inversely proportional to the cardinality of its target class.
- Class and sample weights: row weights are defined as the product of sample weights and class weights.
In classification tasks, class weights is the default weighting method.
It is meant to ensure that despite class imbalance, the trained model takes each class equally into account.
In regression tasks, using no weighting is the default setting.
Averaging method for one-vs-all metrics
The weighted method for averaging metrics across classes may not be adequate.
Class weights are used to train the model, suggesting equal importance of classes. However, the one-vs-all metrics will be the weighted average across classes, with weights proportional to the cardinality of each class, suggesting unequal importance of classes.
The unweighted method for averaging metrics across classes may not be adequate.
Class weights are not used to train the model, suggesting unequal importance of classes. However, the one-vs-all metrics will be the arithmetic average across classes, suggesting equal importance of classes.
This setting determines how one-vs-all metrics are averaged across classes:
- Unweighted class average: all classes have equal weight. Better suited for classes of equal importance.
- Weighted class average: classes are weighted by the number of rows for each class, or the sum of their sample weights if sample weights are specified.