WhyLabsWriter refactor

[richard-rogers]

Description

Writable represents output to be serialized by whylogs. It will write itself to 1 or more temporary files. A Writer takes the temporary file(s) and sends them to their intended destination. The interface allows any Writer to handle any Writable, but in practice some Writers are only interested in certain types of Writables.

WhyLabsWriter acts as the main entry point for sending data to WhyLabs. It makes use of a new WhyLabsClient interface for interacting with WhyLabs REST APIs. WhyLabsWriter has some deprecated methods that duplicate WhyLabsClient functionality the sake of backwards compatibility. WhyLabsWriter delegates to the WhyLabsTransactionWriter, WhyLabsBatchWriter, and WhyLabsReferenceWriter classes according to the WhyLabsWriter::write() use case. The 3 classes correspond to the 3 REST API endpoints for uploading profiles (transaction, batch/async, and reference).

WhyLabsWriterBase implements a few utility methods shared by the various WhyLabs*Writer classes. In particular, _prepare_view_for_upload() handles processing required before uploading a profile (uncompounding, custom performance metric tagging). _send_writable_to_whylabs() serializes a profile in either V0 or V1 format and uploads it to WhyLabs. _upload_view() is a convenience method that just calls _prepare_view_for_upload() then _send_writable_to_whylabs(). WhyLabsWriter::write() accepts a variety of data structures representing a profile: ViewResultSet, ProfileResultSet, SegmentedResultSet, DatasetProfile, DatasetProfileView, and SegmentedDatasetProfileView. WhyLabsWriterBase::_get_view_of_writable() converts all of those except SegmentedResultSet to either DatasetProfileView or SegmentedDatasetProfileView, which represent a single profile/segment to be uploaded to WhyLabs. The WhyLabs*Writer classes generally iterate over SegmentedResultSet uploading each segment.

• Transactions do not support zipped profiles or reference profiles
• Segmented batch or reference profiles can be zipped by adding zip=True argument to write()

Changes

TODO: describe API changes

Related

zipped batch profiles

• [ ] I have reviewed the Guidelines for Contributing and the Code of Conduct.