SafeLine icon indicating copy to clipboard operation
SafeLine copied to clipboard

[Bug] Postgres data using too much disk space and doesn't cleaning automatic

Open pap1zi opened this issue 5 months ago • 3 comments
trafficstars

What happened?

Hi. I`m using safeline CE docker installation version 8.4.0 and my postgres data using too much disk space I have 5 active domains and ~1.5m-2m requests per day I set keeping logs for 1 day in safeline UI. But it didn't help, my pg data using 28 GB on disk.

How we reproduce?

  1. Set clean data 1 day in web UI
  2. Checking postgres tables size
  3. All tables are cleaning, except pg_toast_16416

Expected behavior

Postgres using less disk space

Error log

My settings: Image

I also ran SQL query:

SELECT relname, relpages
FROM pg_class
ORDER BY relpages DESC;

and got this results:

                    relname                     | relpages 
------------------------------------------------+----------
 pg_toast_16416                                 |  2439830
 mgt_detect_log_detail                          |   617415
 mgt_detect_log_basic                           |   209362
 idx_mgt_detect_log_detail_event_id             |    50358
 idx_mgt_detect_log_basic_event_id              |    50123
 pg_toast_16416_index                           |    33296
 mgt_detect_log_detail_pkey                     |    14915
 mgt_detect_log_basic_pkey                      |    14849
 idx_detect_log_basic_src_ip_host_dst_port      |     6541
 idx_mgt_detect_log_basic_socket_ip             |     6272
 idx_mgt_detect_log_detail_timestamp            |     5306
 idx_mgt_detect_log_basic_created_at            |     5284
 idx_mgt_detect_log_basic_timestamp             |     5283
 idx_mgt_detect_log_basic_action                |     4890
 idx_mgt_detect_log_basic_attack_type           |     4810
 idx_mgt_detect_log_basic_risk_level            |     4805
 mgt_event                                      |     1681
 statistics_ips                                 |     1158
 resources                                      |      816
 statistics_ips_pkey                            |      414
 mgt_statistics_agg                             |      405
 mgt_event_pkey                                 |      337
 idx_statistics_ips_key                         |      269
 pg_toast_16696                                 |      218
 idx_mgt_event_ip                               |      210
 idx_statistics_ips_day_time                    |      159
 statistics_basics                              |      146
 idx_statistics_ips_time                        |      141
 idx_mgt_event_finished                         |       99
 statistics_geos_pkey                           |       98
 pg_attribute                                   |       96
 pg_proc                                        |       96
 idx_statistics_geo_time_country                |       91
 idx_mgt_event_host                             |       91
 idx_mgt_event_dst_port                         |       84
 qps                                            |       78
 statistics_geos                                |       73
 mgt_statistics_agg_pkey                        |       67
 pg_toast_2618                                  |       64
 statistics_basics_pkey                         |       55
 pg_statistic                                   |       55
 statistics_sessions                            |       54
 pg_toast_2619                                  |       52
 siteid_path_method                             |       45
 pg_description                                 |       44
 idx_statistics_agg_type_time                   |       37
 idx_resources_path                             |       34
 qps_pkey                                       |       33
 pg_proc_proname_args_nsp_index                 |       31
 idx_statistics_sessions_key                    |       30
 pg_description_o_c_o_index                     |       28
 idx_statistics_basic_type_time                 |       27
 idx_statistics_basics_site_id                  |       27
 statistics_sessions_pkey                       |       27
 idx_mgt_statistics_agg_day_time                |       25
 idx_mgt_statistics_agg_hour_time               |       23
 pg_attribute_relid_attnam_index                |       23
 pg_depend                                      |       20

table pg_toast_16416 is very big and seem like it doesn't cleaning at all

pap1zi avatar May 24 '25 14:05 pap1zi