module prediction.evaluation.prediction_evaluator

Predictive Model Evaluator.

This module contains an evaluator class for online predictive models.

Copyright (C) 2022 Johannes Haug.

class PredictionEvaluator

Online prediction evaluator class.

Attributes:

  • measure_funcs (List[Callable]): List of evaluation measure functions.
  • decay_rate (float |None): If this parameter is not None, the measurements are additionally aggregated with the specific decay/fading factor.
  • window_size (int | None): If this parameter is not None, the measurements are additionally aggregated in a sliding window.
  • kwargs (dict): A dictionary containing additional and specific keyword arguments, which are passed to the evaluation functions.
  • testing_comp_times (list): List of computation times per testing iteration.
  • training_comp_times (list): List of computation times per training iteration.
  • memory_changes (list): Memory changes (in GB RAM) per training iteration of the online feature selection model.
  • result (dict): The raw and aggregated measurements of each evaluation measure function.

method PredictionEvaluator.__init__

__init__(
    measure_funcs: List[Callable],
    decay_rate: Optional[float] = None,
    window_size: Optional[float] = None,
    **kwargs
)

Inits the prediction evaluation object.

Args:

  • measure_funcs: List of evaluation measure functions.
  • decay_rate: If this parameter is not None, the measurements are additionally aggregated with the specific decay/fading factor.
  • window_size: If this parameter is not None, the measurements are additionally aggregated in a sliding window.

kwargs: A dictionary containing additional and specific keyword arguments, which are passed to the evaluation functions.

method PredictionEvaluator.run

run(
    y_true: Union[numpy._array_like._SupportsArray[numpy.dtype], numpy._nested_sequence._NestedSequence[numpy._array_like._SupportsArray[numpy.dtype]], bool, int, float, complex, str, bytes, numpy._nested_sequence._NestedSequence[Union[bool, int, float, complex, str, bytes]]],
    y_pred: Union[numpy._array_like._SupportsArray[numpy.dtype], numpy._nested_sequence._NestedSequence[numpy._array_like._SupportsArray[numpy.dtype]], bool, int, float, complex, str, bytes, numpy._nested_sequence._NestedSequence[Union[bool, int, float, complex, str, bytes]]],
    X: Union[numpy._array_like._SupportsArray[numpy.dtype], numpy._nested_sequence._NestedSequence[numpy._array_like._SupportsArray[numpy.dtype]], bool, int, float, complex, str, bytes, numpy._nested_sequence._NestedSequence[Union[bool, int, float, complex, str, bytes]]],
    predictor: float.prediction.base_predictor.BasePredictor,
    rng: numpy.random._generator.Generator
)

Updates relevant statistics and computes the evaluation measures.

Args:

  • y_true: True target labels.
  • y_pred: Predicted target labels.
  • X: Array/matrix of observations.
  • predictor: Predictor object.
  • rng: A numpy random number generator object.

Raises:

  • TypeError: If the calculation of a measure runs an error.

This file was automatically generated via lazydocs.