openmaptiles-tools icon indicating copy to clipboard operation
openmaptiles-tools copied to clipboard

Importing diff ends with a PostgreSQL error, retries and continues to fail

Open ingarpedersen opened this issue 5 years ago • 0 comments

I have imported a country, and the initial generation works just fine. The next day I'm updating using diffs, but now the update fails:

Starting map-update_postgres_1 ... done
docker-compose run --rm -u 1001:1001 openmaptiles-tools sh -c 'pgwait && import-update'
[Oct  1 10:05:14] [INFO] [  1m0s] C:   77000/s (4642703) N:     500/s (32009) W:       0/s (0) R:      0/s (0)
[Oct  1 10:06:07] [INFO] [diff] Parsing changes, updating cache and removing elements took: 1m53.168368631s
[Oct  1 10:06:07] [INFO] [ 1m53s] C:   78000/s (5165068) N:     500/s (39150) W:   15100/s (590221) R:   3020/s (25845)
[Oct  1 10:07:07] [INFO] [  1m0s] C:       0/s (0) N:       0/s (0) W:    3200/s (158359) R:   2580/s (7702)
[Oct  1 10:08:07] [INFO] [  2m0s] C:       0/s (0) N:       0/s (0) W:    3000/s (332858) R:   2580/s (7702)
[Oct  1 10:09:07] [INFO] [  3m0s] C:       0/s (0) N:    5800/s (73334) W:    2900/s (356748) R:   2580/s (7702)
[Oct  1 10:09:13] [INFO] [PostGIS] Updating generalized tables took: 43.070865217s
[Oct  1 10:09:13] [INFO] [diff] Processing /import/000/002/941.osc.gz took: 4m59.534247281s
[Oct  1 10:09:13] [INFO] importing #2941 till 2020-10-01 00:00:00 +0000 UTC took: 4m59.53537228s
[Oct  1 10:09:13] [ERR] pq: Could not complete operation in a failed transaction
[Oct  1 10:09:13] [INFO] retrying in 2s

After the retrying there seems to be 2 processes working. After a while the same error, with 4s delay, then there are 3 processes, and so on.

I have deleted everything (all docker containers, volumes etc), and imported again, the same thing happens with the first diff update.

I have checked and there is no issue with disk space or RAM.

Any clues where to look??

Best regards Ingar F Pedersen

ingarpedersen avatar Oct 01 '20 10:10 ingarpedersen