[Feature]: Always write server debug logs to a file

[Egor-S]

Problem

In case of a problem with the dstack server, it's hard to investigate without debug logs, which are not always turned on. Re-running the server with a debug log level takes time and may not reproduce the issue.

Solution

Add additional logs handler, writing to ~/.dstack/server/logs/debug.log debug logs. We should use log rotation to limit disk usage.

Benefit

You can always get debug logs for recent events.

Alternatives

Always print debug logs to stdout

Would you like to help contributing this feature?

Yes

[peterschmidt85]

This issue is stale because it has been open for 30 days with no activity.

[peterschmidt85]

This issue is stale because it has been open for 30 days with no activity.

[peterschmidt85]

This issue was closed because it has been inactive for 14 days since being marked as stale. Please reopen the issue if it is still relevant.