whylogs.app.config

Classes/functions for configuring the whylogs app

Module Contents

Classes

WriterType

Generic enumeration.

TransportParameterConfig

TransportParameterConfigSchema

Marshmallow schema for WriterConfig class.

WriterConfig

Config for whylogs writers

MetadataConfig

Config for whylogs metadata

SessionConfig

Config for a whylogs session.

WriterConfigSchema

Marshmallow schema for WriterConfig class.

MetadataConfigSchema

Marshmallow schema for MetadataConfig class.

SessionConfigSchema

Marshmallow schema for SessionConfig class.

Functions

load_config(path_to_config: str = None)

Load logging configuration, from disk and from the environment.

Attributes

SUPPORTED_WRITERS

WHYLOGS_YML

ALL_SUPPORTED_FORMATS

Supported output formats for whylogs writer configuration

SegmentTag

SegmentTags

class whylogs.app.config.WriterType

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

local
s3
whylabs
mlflow
whylogs.app.config.SUPPORTED_WRITERS
whylogs.app.config.WHYLOGS_YML = .whylogs.yaml
whylogs.app.config.ALL_SUPPORTED_FORMATS

Supported output formats for whylogs writer configuration

whylogs.app.config.SegmentTag
whylogs.app.config.SegmentTags
class whylogs.app.config.TransportParameterConfig(endpoint_url: str, aws_access_key_id: str, aws_secret_access_key: str, region_name: str, verify: str)
class whylogs.app.config.TransportParameterConfigSchema

Bases: marshmallow.Schema

Marshmallow schema for WriterConfig class.

endpoint_url
aws_access_key_id
aws_secret_access_key
region_name
verify
make_writer(self, data, **kwargs)
class whylogs.app.config.WriterConfig(type: str, formats: Optional[List[str]] = None, output_path: Optional[str] = None, path_template: Optional[str] = None, filename_template: Optional[str] = None, data_collection_consent: Optional[bool] = None, transport_parameters: Optional[TransportParameterConfig] = None)

Config for whylogs writers

See also:

Parameters
to_yaml(self, stream=None)

Serialize this config to YAML

Parameters

stream – If None (default) return a string, else dump the yaml into this stream.

static from_yaml(stream, **kwargs)

Load config from yaml

Parameters

  • stream (str, file-obj) – String or file-like object to load yaml from

  • kwargs – ignored

Returns

config – Generated config

Return type

WriterConfig

class whylogs.app.config.MetadataConfig(type: str, output_path: str, input_path: Optional[str] = '', path_template: Optional[str] = None)

Config for whylogs metadata

See also:

Parameters

  • type (str) – Destination for the writer output, e.g. ‘local’ or ‘s3’

  • output_path (str) – Prefix of where to output files. A directory for type = ‘local’, or key prefix for type = ‘s3’

  • input_path (str) – Path to search for pre-calculated segment files. Paths separated by ‘:’.

  • path_template (str, optional) – Templatized path output using standard python string templates. Variables are accessed via $identifier or ${identifier}. See whylogs.app.writers.Writer.template_params() for a list of available identifers. Default = whylogs.app.metadata_writer.DEFAULT_PATH_TEMPLATE

to_yaml(self, stream=None)

Serialize this config to YAML

Parameters

stream – If None (default) return a string, else dump the yaml into this stream.

static from_yaml(stream, **kwargs)

Load config from yaml

Parameters

  • stream (str, file-obj) – String or file-like object to load yaml from

  • kwargs – ignored

Returns

config – Generated config

Return type

WriterConfig

class whylogs.app.config.SessionConfig(project: str, pipeline: str, writers: List[WriterConfig], metadata: Optional[MetadataConfig] = None, verbose: bool = False, with_rotation_time: str = None, cache_size: int = 1, report_progress: bool = False)

Config for a whylogs session.

See also SessionConfigSchema

Parameters

  • project (str) – Project associated with this whylogs session

  • pipeline (str) – Name of the associated data pipeline

  • writers (list) – A list of WriterConfig objects defining writer outputs

  • metadata (MetadataConfig) – A MetadataConfiguration object. If none, will replace with default.

  • verbose (bool, default=False) – Output verbosity

  • with_rotation_time (str, default = None, to rotate profiles with time, takes values of overall rotation interval,) – “s” for seconds “m” for minutes “h” for hours “d” for days

  • cache_size (int default =1, sets how many dataprofiles to cache in logger during rotation) –

to_yaml(self, stream=None)

Serialize this config to YAML

Parameters

stream – If None (default) return a string, else dump the yaml into this stream.

static from_yaml(stream)

Load config from yaml

Parameters

stream (str, file-obj) – String or file-like object to load yaml from

Returns

config – Generated config

Return type

SessionConfig

class whylogs.app.config.WriterConfigSchema

Bases: marshmallow.Schema

Marshmallow schema for WriterConfig class.

class Meta
unknown
type
formats
output_path
path_template
filename_template
transport_parameters
make_writer(self, data, **kwargs)
class whylogs.app.config.MetadataConfigSchema

Bases: marshmallow.Schema

Marshmallow schema for MetadataConfig class.

type
output_path
input_path
path_template
make_metadata(self, data, **kwargs)
class whylogs.app.config.SessionConfigSchema

Bases: marshmallow.Schema

Marshmallow schema for SessionConfig class.

project
pipeline
with_rotation_time
cache
verbose
writers
metadata
make_session(self, data, **kwargs)
whylogs.app.config.load_config(path_to_config: str = None)

Load logging configuration, from disk and from the environment.

Config is loaded by attempting to load files in the following order. The first valid file will be used

  1. Path set in WHYLOGS_CONFIG environment variable

  2. Current directory’s .whylogs.yaml file

  3. ~/.whylogs.yaml (home directory)

  4. /opt/whylogs/.whylogs.yaml path

Returns

config – Config for the logger, if a valid config file is found, else returns None.

Return type

SessionConfig, None