whylogs.api.writer.s3#

Module Contents#

Classes#

S3Writer

A WhyLogs writer to upload DatasetProfileViews onto Amazon S3.

Attributes#

logger

whylogs.api.writer.s3.logger#
class whylogs.api.writer.s3.S3Writer(s3_client: Optional[botocore.client.BaseClient] = None, base_prefix: Optional[str] = None, bucket_name: Optional[str] = None, object_name: Optional[str] = None)#

Bases: whylogs.api.writer.Writer

A WhyLogs writer to upload DatasetProfileViews onto Amazon S3.

>**IMPORTANT**: In order to correctly connect to your Amazon S3 bucket, make sure you have the following environment variables set: [AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY]

Parameters

  • s3_client (BaseClient, optional) – The s3 client used to authenticate and perform operations on the s3 bucket. Should be a BaseClient from the boto3 library

  • base_prefix (str, optional) – The base file prefix for s3, in order to organize. A placeholder ‘profile’ will take place if None is provided.

  • bucket_name (str, optional) – The name of the bucket to connect to. Made optional so the user can also access it via the option method

  • object_name (str, optional) – The s3’s object name. It basically states the location where the file goes to. Also made optional, so it can be defined through the option method

Returns

Return type

None

Examples

An example usage of this method can be represented with the simple code above. Here we are assuming the user has already assigned a pandas.DataFrame to the df variable.

```python import whylogs as why

profile = why.log(pandas=df) profile.writer(“s3”).option(bucket_name=”my_bucket”).write() ```

write(file: whylogs.api.writer.writer._Writable, dest: Optional[str] = None, **kwargs: Any) → Tuple[bool, Union[str, List[Tuple[bool, str]]]]#
Parameters

  • file (whylogs.api.writer.writer._Writable) –

  • dest (Optional[str]) –

  • kwargs (Any) –

Return type

Tuple[bool, Union[str, List[Tuple[bool, str]]]]

option(**kwargs: Any) → whylogs.api.writer.Writer#

bucket_name: str S3 bucket to write to object_name: str Object name to create s3_client: BaseClient S3 client

Parameters

kwargs (Any) –

Return type

whylogs.api.writer.Writer