Metrics

Hyperparameter optimization and model evaluation

See

Sample weights

All metrics will be weighted by the variable {{mlTaskDesign.weight.sampleWeightVariable}}, both for hyperparameters optimization and final evaluation of the model.

You can revert to using unweighted metrics by disabling the sample weights option.

Averaging method for one-vs-all metrics

{{mlTaskDesign.modeling.metrics.classAveragingMethod == 'WEIGHTED' ? 'Weighted' : 'Unweighted'}}
You can change this setting in the weighting strategy section.

One-vs-all metrics are computed for each class, the overall metric can be computed using the following methods:

  • Unweighted class average: all classes have equal weight. Better suited for classes of equal importance.
  • Weighted class average: classes are weighted by the number of rows for each class, or the sum of their sample weights if sample weights are specified.
The weighted method for averaging metrics across classes may not be adequate. Class weights are used to train the model, suggesting equal importance of classes. However, the one-vs-all metrics will be the weighted average across classes, with weights proportional to the cardinality of each class, suggesting unequal importance of classes.
The unweighted method for averaging metrics across classes may not be adequate. Class weights are not used to train the model, suggesting unequal importance of classes. However, the one-vs-all metrics will be the arithmetic average across classes, suggesting equal importance of classes.
You can change this setting in the weighting strategy section.
%

Cost matrix

The cost matrix evaluates a "gain" brought by this model based on the following hypothesis