whylogs.util.protobuf

Functions for interacting with protobuf

Module Contents

Functions

message_to_json(x: google.protobuf.message, **kwargs)	A wrapper for google.protobuf.json_format.MessageToJson
message_to_dict(x: google.protobuf.message)	Convert a protobuf message to a dictionary
_varint_delim_reader(fp)
_varint_delim_iterator(f)	Return an iterator to read delimited protobuf messages. The iterator will
multi_msg_reader(f, msg_class)	Return an iterator to iterate through protobuf messages in a multi-message
read_multi_msg(f, msg_class)	Wrapper for multi_msg_reader() which reads all the messages and
_encode_one_msg(msg: google.protobuf.message)
_write_multi_msg(msgs: list, fp)
write_multi_msg(msgs: list, f)	Write a list (or iterator) of protobuf messages to a file.
repr_message(x: google.protobuf.message.Message, indent=2, display=True)	Print or generate string preview of a protobuf message. This is mainly
_repr_message(x, level=0, msg='', display=True, indent=2)

whylogs.util.protobuf.message_to_json(x: google.protobuf.message, **kwargs)

A wrapper for google.protobuf.json_format.MessageToJson

Currently a very thin wrapper…x and kwargs are just passed to MessageToJson

whylogs.util.protobuf.message_to_dict(x: google.protobuf.message)

Convert a protobuf message to a dictionary

A thin wrapper around the google built-in function.

whylogs.util.protobuf._varint_delim_reader(fp)

whylogs.util.protobuf._varint_delim_iterator(f)

Return an iterator to read delimited protobuf messages. The iterator will return protobuf messages one by one as raw bytes objects.

whylogs.util.protobuf.multi_msg_reader(f, msg_class)

Return an iterator to iterate through protobuf messages in a multi-message protobuf file.

See also: write_multi_msg()

Parameters

• f (str, file-object) – Filename or open file object to read from

• msg_class (class) – The Protobuf message class, gets instantiated with a call to msg_class()

Returns

Iterator which returns protobuf messages

Return type

msg_iterator

whylogs.util.protobuf.read_multi_msg(f, msg_class)

Wrapper for multi_msg_reader() which reads all the messages and returns them as a list.

whylogs.util.protobuf._encode_one_msg(msg: google.protobuf.message)

whylogs.util.protobuf._write_multi_msg(msgs: list, fp)

whylogs.util.protobuf.write_multi_msg(msgs: list, f)

Write a list (or iterator) of protobuf messages to a file.

The multi-message file format is a binary format with:

<varint MessageBytesSize><message>

Which is repeated, where the len(message) in bytes is MessageBytesSize

Parameters

• msgs (list, iterable) – Protobuf messages to write to disk

• f (str, file-object) – Filename or open binary file object to write to

whylogs.util.protobuf.repr_message(x: google.protobuf.message.Message, indent=2, display=True)

Print or generate string preview of a protobuf message. This is mainly to get a preview of the attribute names and structure of a protobuf message class.

Parameters

• x (google.protobuf.message.Message) – Message to preview

• indent (int) – Indentation

• display (bool) – If True, print the message and return None. Else, return a string.

Returns

msg – If display == False, return the message, else return None.

Return type

str, None

whylogs.util.protobuf._repr_message(x, level=0, msg='', display=True, indent=2)