Cloud Storage Integration

[abduldjafar]

Hi,

Thank you so much for an excellent project! May I know if you have plans to integrate with S3 or GCS?

[Delgan]

Hi @abduldjafar.

Integration with S3 or GCS is not built-in. However, as Loguru is quite flexible, it should be easy for the user to define and configure a personalized custom handler that works with any cloud service provider. I'm not a user of these services myself, therefore the following examples are untested.

From the GCP documentation, I think google.cloud.logging should combine with a PropagateHandler quite well.

import google.cloud.logging
from loguru import logger

class PropagateHandler(logging.Handler):
    def emit(self, record: logging.LogRecord) -> None:
        logging.getLogger(record.name).handle(record)

logger.add(PropagateHandler(), format="{message}")

client = google.cloud.logging.Client()
client.setup_logging()

logger.info("Message sent to GCP immediately")

It is also certainly possible to get rid of the PropagateHandler and client.setup_logging() entirely, by creating a dedicated sink directly.

Regarding Amazon S3, I imagine you want to upload the entire log file. In such a case, maybe you can configure a regular file handler and upload it at the end using a custom compression function:

from loguru import logger
import boto3

def upload_to_s3(file_path):
    s3_client = boto3.client("s3")
    s3_client.upload_file(file_path, "bucket-name", "log-file.log")

logger.add("app.log", compression=upload_to_s3)

logger.info("Message logged to a file, then the file will be sent to S3")

Did you have something else in mind regarding Cloud Storage integration? I don't plan to make anything built-in, as Loguru is relatively agnostic towards handlers. However, one could imagine enhancing the documentation with useful snippets.