`whylogs.experimental.performance_estimation.estimators`#

Module Contents#

Classes#

`PerformanceEstimator`	Base class for performance estimators.
`AccuracyEstimator`	Estimates the accuracy of a model based on the confusion matrix of the reference result set and the target result set.

Attributes#

logger

whylogs.experimental.performance_estimation.estimators.logger#

class whylogs.experimental.performance_estimation.estimators.PerformanceEstimator(reference_result_set: whylogs.api.logger.result_set.SegmentedResultSet)#

Bases: abc.ABC

Base class for performance estimators.

Parameters: reference_result_set (whylogs.api.logger.result_set.SegmentedResultSet) –

abstract estimate(target_result_set: whylogs.api.logger.result_set.SegmentedResultSet) → whylogs.experimental.performance_estimation.estimation_results.EstimationResult#

Parameters: target_result_set (whylogs.api.logger.result_set.SegmentedResultSet) –
Return type: whylogs.experimental.performance_estimation.estimation_results.EstimationResult

class whylogs.experimental.performance_estimation.estimators.AccuracyEstimator(reference_result_set: whylogs.api.logger.result_set.SegmentedResultSet)#

Bases: PerformanceEstimator

Estimates the accuracy of a model based on the confusion matrix of the reference result set and the target result set.

The estimation is based on the individual performance of each segment of the reference result set. The reference accuracies for each segment are weighted by the number of records in each segment in the target result set. The estimated accuracy is the sum of the weighted accuracies of each segment.

Expects a single partition with mutually exclusive segments, and same set of segments in the reference and target result sets.

Supported for binary classification only.

Examples

Create target and reference dataframes:

from whylogs.experimental.performance_estimation import AccuracyEstimator
from whylogs.api.logger.result_set import SegmentedResultSet

estimator = AccuracyEstimator(reference_result_set: SegmentedResultSet = reference_results)

estimator_result = estimator.estimate(target_result_set: SegmentedResultSet = target_results)
print("Estimated accuracy: ", estimator_result.accuracy)

Parameters: reference_result_set (whylogs.api.logger.result_set.SegmentedResultSet) –

estimate(target: whylogs.api.logger.result_set.SegmentedResultSet) → whylogs.experimental.performance_estimation.estimation_results.EstimationResult#

Estimate the overall accuracy of a target result set based on a reference result set.

Assumes the presence of a segmented column in the reference and target result sets.

returns: An EstimationResult object containing the estimated accuracy, partition’s id and the profiles’ dataset timestamp.

Parameters: target (whylogs.api.logger.result_set.SegmentedResultSet) –
Return type: whylogs.experimental.performance_estimation.estimation_results.EstimationResult

whylogs.experimental.performance_estimation.estimators#

Module Contents#

Classes#

Attributes#

`whylogs.experimental.performance_estimation.estimators`#