whylogs.experimental.extras.nlp_metric

Module Contents

Classes

SvdMetricConfig
SvdMetric	non-updating SVD metric
UpdatableSvdMetric	updating SVD metric
NlpConfig	If you pass in an UpdatableSvdMetric, the SVD will be updated along with the
BagOfWordsMetric	Natural language processing metric -- treat document as a bag of words
LsiMetric	Natural language processing -- latent sematic indexing metric
ResolverWrapper
NlpLogger

class whylogs.experimental.extras.nlp_metric.SvdMetricConfig

Bases: whylogs.core.metrics.metrics.MetricConfig

k: int = 100

decay: float = 1.0

class whylogs.experimental.extras.nlp_metric.SvdMetric

Bases: whylogs.core.metrics.metrics.Metric

non-updating SVD metric

property namespace: str

Return type

str

k: whylogs.core.metrics.metric_components.IntegralComponent

decay: whylogs.core.metrics.metric_components.FractionalComponent

U: whylogs.experimental.extras.matrix_component.MatrixComponent

S: whylogs.experimental.extras.matrix_component.MatrixComponent

residual(vector: whylogs.core.stubs.np.ndarray) → float

Retruns the residual of the vector given the current approximate SVD: residual = || U S S^{+} U’ x - x || / || x || where x is the vector

Parameters

vector (whylogs.core.stubs.np.ndarray) –

Return type

float

merge(other: SvdMetric) → SvdMetric

Parameters

other (SvdMetric) –

Return type

SvdMetric

to_summary_dict(cfg: Optional[whylogs.core.configs.SummaryConfig] = None) → Dict[str, Any]

Parameters

cfg (Optional[whylogs.core.configs.SummaryConfig]) –

Return type

Dict[str, Any]

columnar_update(data: whylogs.core.preprocessing.PreprocessedColumn) → whylogs.core.metrics.metrics.OperationResult

Parameters

data (whylogs.core.preprocessing.PreprocessedColumn) –

Return type

whylogs.core.metrics.metrics.OperationResult

classmethod zero(cfg: Optional[whylogs.core.metrics.metrics.MetricConfig] = None) → SvdMetric

Instances created with zero() will be useless because they’re not updatable.

Parameters

cfg (Optional[whylogs.core.metrics.metrics.MetricConfig]) –

Return type

SvdMetric

class whylogs.experimental.extras.nlp_metric.UpdatableSvdMetric

Bases: SvdMetric

updating SVD metric

property namespace: str

Return type

str

k: whylogs.core.metrics.metric_components.IntegralComponent

decay: whylogs.core.metrics.metric_components.FractionalComponent

U: whylogs.experimental.extras.matrix_component.MatrixComponent

S: whylogs.experimental.extras.matrix_component.MatrixComponent

merge(other: SvdMetric) → UpdatableSvdMetric

Parameters

other (SvdMetric) –

Return type

UpdatableSvdMetric

columnar_update(data: whylogs.core.preprocessing.PreprocessedColumn) → whylogs.core.metrics.metrics.OperationResult

Parameters

data (whylogs.core.preprocessing.PreprocessedColumn) –

Return type

whylogs.core.metrics.metrics.OperationResult

classmethod zero(cfg: Optional[SvdMetricConfig] = None) → UpdatableSvdMetric

Instances created with zero() will be useless because they’re not updatable.

Parameters

cfg (Optional[SvdMetricConfig]) –

Return type

UpdatableSvdMetric

residual(vector: whylogs.core.stubs.np.ndarray) → float

Retruns the residual of the vector given the current approximate SVD: residual = || U S S^{+} U’ x - x || / || x || where x is the vector

Parameters

vector (whylogs.core.stubs.np.ndarray) –

Return type

float

to_summary_dict(cfg: Optional[whylogs.core.configs.SummaryConfig] = None) → Dict[str, Any]

Parameters

cfg (Optional[whylogs.core.configs.SummaryConfig]) –

Return type

Dict[str, Any]

class whylogs.experimental.extras.nlp_metric.NlpConfig

Bases: whylogs.core.metrics.metrics.MetricConfig

If you pass in an UpdatableSvdMetric, the SVD will be updated along with the NlpMetric’s residual distribution. A non-updatable SvdMetric will update the residual distribution, but it will not update the SVD as new term vectors are processed.

Note that the [Updatable]SvdMetric is not [de]serialized with the NlpMetric. You’ll have to manage that yourself.

svd: SvdMetric

class whylogs.experimental.extras.nlp_metric.BagOfWordsMetric

Bases: whylogs.core.metrics.multimetric.MultiMetric

Natural language processing metric – treat document as a bag of words

property namespace: str

Return type

str

fi_disabled: bool = False

columnar_update(data: whylogs.core.preprocessing.PreprocessedColumn) → whylogs.core.metrics.metrics.OperationResult

Parameters

data (whylogs.core.preprocessing.PreprocessedColumn) –

Return type

whylogs.core.metrics.metrics.OperationResult

classmethod zero(cfg: Optional[whylogs.core.metrics.metrics.MetricConfig] = None) → BagOfWordsMetric

Parameters

cfg (Optional[whylogs.core.metrics.metrics.MetricConfig]) –

Return type

BagOfWordsMetric

classmethod from_protobuf(msg: whylogs.core.proto.MetricMessage) → BagOfWordsMetric

Parameters

msg (whylogs.core.proto.MetricMessage) –

Return type

BagOfWordsMetric

class whylogs.experimental.extras.nlp_metric.LsiMetric

Bases: whylogs.core.metrics.multimetric.MultiMetric

Natural language processing – latent sematic indexing metric

property namespace: str

Return type

str

svd: SvdMetric

merge(other: LsiMetric) → LsiMetric

Parameters

other (LsiMetric) –

Return type

LsiMetric

columnar_update(data: whylogs.core.preprocessing.PreprocessedColumn) → whylogs.core.metrics.metrics.OperationResult

Parameters

data (whylogs.core.preprocessing.PreprocessedColumn) –

Return type

whylogs.core.metrics.metrics.OperationResult

classmethod zero(cfg: Optional[whylogs.core.metrics.metrics.MetricConfig] = None) → LsiMetric

Parameters

cfg (Optional[whylogs.core.metrics.metrics.MetricConfig]) –

Return type

LsiMetric

classmethod from_protobuf(msg: whylogs.core.proto.MetricMessage) → LsiMetric

Parameters

msg (whylogs.core.proto.MetricMessage) –

Return type

LsiMetric

class whylogs.experimental.extras.nlp_metric.ResolverWrapper(resolver: whylogs.core.resolvers.Resolver)

Bases: whylogs.core.resolvers.Resolver

Parameters

resolver (whylogs.core.resolvers.Resolver) –

resolve(name: str, why_type: whylogs.core.datatypes.DataType, column_schema: whylogs.core.schema.ColumnSchema) → Dict[str, whylogs.core.metrics.metrics.Metric]

Parameters

• name (str) –

• why_type (whylogs.core.datatypes.DataType) –

• column_schema (whylogs.core.schema.ColumnSchema) –

Return type

Dict[str, whylogs.core.metrics.metrics.Metric]

class whylogs.experimental.extras.nlp_metric.NlpLogger(svd_class: Optional[type] = None, svd_config: Optional[SvdMetricConfig] = None, svd_state: Optional[whylogs.core.proto.MetricMessage] = None, schema: Optional[whylogs.core.DatasetSchema] = None, column_prefix: str = 'nlp')

Parameters

• svd_class (Optional[type]) –

• svd_config (Optional[SvdMetricConfig]) –

• svd_state (Optional[whylogs.core.proto.MetricMessage]) –

• schema (Optional[whylogs.core.DatasetSchema]) –

• column_prefix (str) –

log(terms: Optional[Union[Dict[str, List[str]], List[str]]] = None, vector: Optional[Union[Dict[str, whylogs.core.stubs.np.ndarray], whylogs.core.stubs.np.ndarray]] = None) → whylogs.api.logger.result_set.ResultSet

Parameters

• terms (Optional[Union[Dict[str, List[str]], List[str]]]) –

• vector (Optional[Union[Dict[str, whylogs.core.stubs.np.ndarray], whylogs.core.stubs.np.ndarray]]) –

Return type

whylogs.api.logger.result_set.ResultSet

get_svd_state() → whylogs.core.proto.MetricMessage

Return type

whylogs.core.proto.MetricMessage

get_profile() → whylogs.api.logger.result_set.ResultSet

Return type

whylogs.api.logger.result_set.ResultSet