dify Timestamp in Logs is 8 Hours Behind Despite Correct Timezone Settings

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.11

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I deployed Dify using Docker, and have set the timezone to UTC+8 on the web interface. I also added TZ: Asia/Shanghai to the docker-compose. yaml file. The time inside the container is correct, but the timestamps in the logs are still 8 hours behind. How can this be fixed?

The time inside the container is correct. Can the backend service timezone be configured to match the container timezone?

✔️ Expected Behavior

The log timestamp for the back-end container is set to Zone 8

❌ Actual Behavior

The log timestamp of the back-end container can only be utc time

Jun 20 '24 04:06 ZimaBlueee

Hey, @ZimaBlueee! 👋 I'm here to help you with bug fixes, answering questions, and becoming a contributor. Let's squash those bugs together!

To ensure that the log timestamps match the container's timezone, you need to configure both the container and the backend service correctly. Here are the steps to achieve this:

Set the Container Timezone: Ensure that the container's timezone is set correctly in your docker-compose.yml file.

version: '3.8'

services:
  web:
    image: your_image_name
    environment:
      - TZ=Asia/Shanghai
    # Other service configurations...

Configure the Backend Service Timezone: Modify the backend service to use the specified timezone for logging. This involves setting the LOG_TZ configuration in the api/configs/feature/__init__.py file and ensuring the logging configuration uses this timezone.

In api/configs/feature/__init__.py:

class LoggingConfigs(BaseModel):
    """
    Logging configs
    """

    LOG_LEVEL: str = Field(
        description='Log output level, default to INFO.'
                    'It is recommended to set it to ERROR for production.',
        default='INFO',
    )

    LOG_FILE: Optional[str] = Field(
        description='logging output file path',
        default=None,
    )

    LOG_FORMAT: str = Field(
        description='log format',
        default='%(asctime)s.%(msecs)03d %(levelname)s [%(threadName)s] [%(filename)s:%(lineno)d] - %(message)s',
    )

    LOG_DATEFORMAT: Optional[str] = Field(
        description='log date format',
        default=None,
    )

    LOG_TZ: Optional[str] = Field(
        description='specify log timezone, eg: America/New_York',
        default='Asia/Shanghai',
    )

Update the Logging Configuration: Ensure the logging configuration in api/app.py uses the specified timezone.

In api/app.py:

def create_app() -> Flask:
    app = create_flask_app_with_configs()

    app.secret_key = app.config['SECRET_KEY']

    log_handlers = None
    log_file = app.config.get('LOG_FILE')
    if log_file:
        log_dir = os.path.dirname(log_file)
        os.makedirs(log_dir, exist_ok=True)
        log_handlers = [
            RotatingFileHandler(
                filename=log_file,
                maxBytes=1024 * 1024 * 1024,
                backupCount=5
            ),
            logging.StreamHandler(sys.stdout)
        ]

    logging.basicConfig(
        level=app.config.get('LOG_LEVEL'),
        format=app.config.get('LOG_FORMAT'),
        datefmt=app.config.get('LOG_DATEFORMAT'),
        handlers=log_handlers,
        force=True
    )
    log_tz = app.config.get('LOG_TZ')
    if log_tz:
        from datetime import datetime
        import pytz
        timezone = pytz.timezone(log_tz)

        def time_converter(seconds):
            return datetime.utcfromtimestamp(seconds).astimezone(timezone).timetuple()

        for handler in logging.root.handlers:
            handler.formatter.converter = time_converter

    initialize_extensions(app)
    register_blueprints(app)
    register_commands(app)

    return app

By following these steps, you should be able to ensure that the log timestamps match the container's timezone ^[1]^[2].

To continue talking to Dosu, mention @dosu.

Jun 20 '24 04:06 dosubot[bot]

I use the official docker image, no manual compilation, can not modify the code inside the container

Jun 20 '24 04:06 ZimaBlueee

if os.name == "nt": os.system('tzutil /s "China Standard Time"') else: os.environ['TZ'] = 'Asia/Shanghai' time.tzset()

Jul 08 '24 03:07 MSZheng20

The method mentioned above, using os.environ['TZ']= 'Asia/Shanghai' is effective, but it is not recommended to modify it this way. The timezone designed by dify is UTC, and it is only when displaying on the frontend web that dayjs is used to convert it to the current timezone. This is an approach that supports internationalization.

There are two types of logs: one is the built-in log of Flask, and the other is the built-in log of Gunicorn. Docker environments default to using Gunicorn, but you can switch by setting the DEBUG environment variable: https://github.com/langgenius/dify/blob/603a89055cb6746650171969fbb59a310e717a8e/api/docker/entrypoint.sh#L29-L39

use flask run xxx
- .env file can add LOG_TZ=Asia/Shanghai
- docker-compose.yaml add LOG_TZ: ${LOG_TZ:-Asia/Shanghai} at &shared-api-worker-env
- then the flask log can change the timezone： https://github.com/langgenius/dify/blob/603a89055cb6746650171969fbb59a310e717a8e/api/app.py#L127-L139
usegunicorn xxx
- the flask logs are no longer valid because gunicorn has its own logging system, which is strongly associated with the project's timezone. Currently, the system is set to os.environ['TZ'] = 'UTC'
- I haven't found a good solution yet, the times in the database are all in UTC, and using UTC for the logs would also be acceptable

update: The database uses timestamp for storage time. Upon a general inspection, it appears that the database handles both the writing and updating of timestamps, and time retrieval is also conducted after converting to UTC. Therefore, if there are no operations related to local time in Flask, it seems that os.environ['TZ']= 'Asia/Shanghai' could be made.

Aug 15 '24 07:08 nadirvishun

@nadirvishun Can I change the time in the database to 'Asia/Shanghai'?

Nov 08 '24 07:11 junjiem

Can I change the time in the database to 'Asia/Shanghai'?

I personally think that it should not be modified. I have checked that the created_at field in the database uses the timestamp type instead of timestamptz, which means that the write operation depends on the timezone of the database location, it will cause the timestamp conversion to be based on the Asia/Shanghai timezone. The frontend uses dayjs.unix(value).tz(timezone).format(format) to display the time in different timezones. The value corresponds to the timestamp in different timezones mentioned above, different values result in different displayed times.

Nov 10 '24 08:11 nadirvishun

if os.name == "nt": os.system('tzutil /s "China Standard Time"') else: os.environ['TZ'] = 'Asia/Shanghai' time.tzset() 你这个是什么

Mar 28 '25 08:03 LocalObject

嘿，！👋 我在这里帮助您修复错误、回答问题并成为贡献者。让我们一起粉碎这些错误！

要确保日志时间戳与容器的时区匹配，您需要正确配置容器和后端服务。以下是实现此目的的步骤：
设置容器时区：确保在文件中正确设置容器的时区。docker-compose.yml version: '3.8'

services: web: image: your_image_name environment: - TZ=Asia/Shanghai # Other service configurations...
配置后端服务时区：修改后端服务以使用指定的时区进行日志记录。这涉及在文件中设置 configuration 并确保 logging configuration 使用此时区。LOG_TZ``api/configs/feature/__init__.py 在：api/configs/feature/__init__.py class LoggingConfigs(BaseModel): """ Logging configs """
LOG_LEVEL: str = Field(
    description='Log output level, default to INFO.'
                'It is recommended to set it to ERROR for production.',
    default='INFO',
)

LOG_FILE: Optional[str] = Field(
    description='logging output file path',
    default=None,
)

LOG_FORMAT: str = Field(
    description='log format',
    default='%(asctime)s.%(msecs)03d %(levelname)s [%(threadName)s] [%(filename)s:%(lineno)d] - %(message)s',
)

LOG_DATEFORMAT: Optional[str] = Field(
    description='log date format',
    default=None,
)

LOG_TZ: Optional[str] = Field(
    description='specify log timezone, eg: America/New_York',
    default='Asia/Shanghai',
)
更新日志记录配置：确保中的日志记录配置使用指定的时区。api/app.py 在：api/app.py def create_app() -> Flask: app = create_flask_app_with_configs()
app.secret_key = app.config['SECRET_KEY']

log_handlers = None
log_file = app.config.get('LOG_FILE')
if log_file:
    log_dir = os.path.dirname(log_file)
    os.makedirs(log_dir, exist_ok=True)
    log_handlers = [
        RotatingFileHandler(
            filename=log_file,
            maxBytes=1024 * 1024 * 1024,
            backupCount=5
        ),
        logging.StreamHandler(sys.stdout)
    ]

logging.basicConfig(
    level=app.config.get('LOG_LEVEL'),
    format=app.config.get('LOG_FORMAT'),
    datefmt=app.config.get('LOG_DATEFORMAT'),
    handlers=log_handlers,
    force=True
)
log_tz = app.config.get('LOG_TZ')
if log_tz:
    from datetime import datetime
    import pytz
    timezone = pytz.timezone(log_tz)

    def time_converter(seconds):
        return datetime.utcfromtimestamp(seconds).astimezone(timezone).timetuple()

    for handler in logging.root.handlers:
        handler.formatter.converter = time_converter

initialize_extensions(app)
register_blueprints(app)
register_commands(app)

return app
通过执行这些步骤，您应该能够确保日志时间戳与容器的时区匹配[1][2].

要继续与 Dosu 交谈，请提及。

Hello, I followed your settings, and the call logs in the workflow are still 8 hours apart

Mar 28 '25 08:03 LocalObject