zulip
backup: Use the default zulip backup system
Zulip backup/restore system now works fine in a docker environment. So we don't need this custom system anymore.
Related https://chat.zulip.org/#narrow/stream/31-production-help/topic/docker.20zulip
I think we should probably still add some documentation with the precise commands needed to use ./manage.py backup and scripts/setup/restore-backup, perhaps in a new section in README.md? Spelling out details like how to get a backup tarball (which you may have created on another machine) in place to be accessible inside the container etc. seem important.
@timabbott I have added the instructions. Also verified that the instructions work correctly.
One thing I noticed is that after the restore is completed, the log is filled with events like this and they keep on going and going. I had to ctrl + c and do docker-compose up again for them to stop.
ulip_1 | 2020-08-28 18:57:07,879 INFO spawned: 'zulip_events_outgoing_webhooks' with pid 1003
zulip_1 | 2020-08-28 18:57:07,880 INFO success: zulip_events_email_senders entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:08,106 INFO success: zulip_events_user_activity_interval entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:08,106 INFO success: zulip_events_digest_emails entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:08,107 INFO exited: zulip_events_embed_links (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:08,151 INFO spawned: 'zulip_events_embed_links' with pid 1005
zulip_1 | 2020-08-28 18:57:08,311 INFO success: zulip_events_missedmessage_mobile_notifications entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:08,311 INFO exited: zulip_events_error_reports (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:08,317 INFO spawned: 'zulip_events_error_reports' with pid 1006
zulip_1 | 2020-08-28 18:57:08,318 INFO exited: zulip_deliver_enqueued_emails (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:08,681 INFO spawned: 'zulip_deliver_enqueued_emails' with pid 1009
zulip_1 | 2020-08-28 18:57:08,682 INFO success: zulip_events_email_mirror entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:08,682 INFO success: zulip_events_missedmessage_emails entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:08,835 INFO exited: zulip_events_invites (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:08,842 INFO spawned: 'zulip_events_invites' with pid 1012
zulip_1 | 2020-08-28 18:57:08,848 INFO exited: zulip_events_email_senders (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:08,902 INFO success: zulip_events_outgoing_webhooks entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:08,904 INFO spawned: 'zulip_events_email_senders' with pid 1013
zulip_1 | 2020-08-28 18:57:09,092 INFO exited: zulip_events_user_presence (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:09,132 INFO spawned: 'zulip_events_user_presence' with pid 1014
zulip_1 | 2020-08-28 18:57:09,134 INFO exited: zulip_events_user_activity (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:09,163 INFO spawned: 'zulip_events_user_activity' with pid 1015
zulip_1 | 2020-08-28 18:57:09,166 INFO success: zulip_events_embed_links entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:09,295 INFO exited: zulip_events_digest_emails (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:09,298 INFO spawned: 'zulip_events_digest_emails' with pid 1017
zulip_1 | 2020-08-28 18:57:09,299 INFO exited: zulip_events_deferred_work (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:09,527 INFO spawned: 'zulip_events_deferred_work' with pid 1019
zulip_1 | 2020-08-28 18:57:09,528 INFO success: zulip_events_error_reports entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:09,538 INFO exited: zulip_events_embedded_bots (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:09,589 INFO spawned: 'zulip_events_embedded_bots' with pid 1020
zulip_1 | 2020-08-28 18:57:09,590 INFO exited: zulip_events_signups (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:09,595 INFO spawned: 'zulip_events_signups' with pid 1021
zulip_1 | 2020-08-28 18:57:09,789 INFO success: zulip_deliver_enqueued_emails entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:09,904 INFO success: zulip_events_invites entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:09,904 INFO success: zulip_events_email_senders entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:09,905 INFO exited: zulip_events_email_mirror (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:09,963 INFO spawned: 'zulip_events_email_mirror' with pid 1024
zulip_1 | 2020-08-28 18:57:09,965 INFO exited: zulip_events_user_activity_interval (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:10,052 INFO spawned: 'zulip_events_user_activity_interval' with pid 1027
zulip_1 | 2020-08-28 18:57:10,052 INFO exited: zulip_events_missedmessage_mobile_notifications (exit status 1; not expected)
zulip_1 | 2020-08-28 18:57:10,178 INFO success: zulip_events_user_presence entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:10,178 INFO success: zulip_events_user_activity entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1 | 2020-08-28 18:57:10,180 INFO spawned: 'zulip_events_missedmessage_mobile_notifications' with pid 1029
zulip_1 | 2020-08-28 18:57:10,342 INFO success: zulip_events_digest_emails entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
^CGracefully stopping... (press Ctrl+C again to force)
That needs investigation in ls -ltr /var/log/zulip/* -- sounds like those queue workers are constantly crashing.
I will look into this again. I didn't spend much time investigating last time.
The standard backup procedure is storing the default uploads directory. Since in Docker this is a link it will just backup the link, without the actual data. Is this intended?
I know the uploads dir is already on the host filesystem, but if backups are meant to be backups they should include it. Filing a new issue
