Fatal error when publishing more than 1 repository to S3 concurrently

[michel-ea]

Hi,

Using the latest version (1.6.2) we get a fatal go exception when publishing multiple repositories to S3 concurrently. This is using the REST API /publish/s3:S3:<repo>/<distro> endpoint.

Exception:

2025-06-24 14:53:13.754	fatal error: concurrent map writes
2025-06-24 14:53:13.757	
2025-06-24 14:53:13.757	goroutine 4073 [running]:
2025-06-24 14:53:13.760	github.com/aptly-dev/aptly/s3.(*PublishedStorage).LinkFromPool(0xc003612f80, {0xc003626108, 0x18}, {0xc008df0b80?, 0x0?}, {0xc003877ad0, 0x23}, {0x55b37ea1da60, 0xc00265e6a0}, {0xc002464000, ...}, ...)
2025-06-24 14:53:13.760		/__w/aptly/aptly/s3/public.go:390 +0x88d
2025-06-24 14:53:13.760	github.com/aptly-dev/aptly/deb.(*Package).LinkFromPool(0xc0016e93f0, {0x55b37ea1f9a0, 0xc003612f80}, {0x55b37ea1da60, 0xc00265e6a0}, {0xc003626108, 0x18}, {0xc008df0b80, 0x1b}, 0x0)
2025-06-24 14:53:13.760		/__w/aptly/aptly/deb/package.go:698 +0x1f6
2025-06-24 14:53:13.760	github.com/aptly-dev/aptly/deb.(*PublishedRepo).Publish.func2(0xc0016e93f0)
2025-06-24 14:53:13.760		/__w/aptly/aptly/deb/publish.go:942 +0x745
2025-06-24 14:53:13.760	github.com/aptly-dev/aptly/deb.(*PackageList).ForEachIndexed(0xc000ef1400?, 0xc006719c10)
2025-06-24 14:53:13.760		/__w/aptly/aptly/deb/list.go:183 +0x70
2025-06-24 14:53:13.760	github.com/aptly-dev/aptly/deb.(*PublishedRepo).Publish(0xc00003e240, {0x55b37ea1da60, 0xc00265e6a0}, {0x55b37ea00f48?, 0xc000d7c000?}, 0xc000180320, {0x0, 0x0}, {0x55b37ea1fb60, 0xc003e40860}, ...)
2025-06-24 14:53:13.760		/__w/aptly/aptly/deb/publish.go:913 +0xd25
2025-06-24 14:53:13.760	github.com/aptly-dev/aptly/api.apiPublishUpdateSwitch.func1({0x55b37ea1fb60, 0xc003e40860}, 0x0?)
2025-06-24 14:53:13.760		/__w/aptly/aptly/api/publish.go:493 +0x2f6
2025-06-24 14:53:13.760	github.com/aptly-dev/aptly/api.runTaskInBackground.func1({0x55b37ea1fb60?, 0xc003e40860?}, 0xc003e40870?)
2025-06-24 14:53:13.760		/__w/aptly/aptly/api/api.go:169 +0x137
2025-06-24 14:53:13.760	github.com/aptly-dev/aptly/task.(*List).consumer.func1()
2025-06-24 14:53:13.760		/__w/aptly/aptly/task/list.go:54 +0x44
2025-06-24 14:53:13.761	created by github.com/aptly-dev/aptly/task.(*List).consumer in goroutine 2576
2025-06-24 14:53:13.761		/__w/aptly/aptly/task/list.go:53 +0x26

Note: this is from 1.6.1 but the issue still occurs on 1.6.2.

This is noticeable with publish jobs that take a bit of time, in this case 2 repositories published had approximately 8,300 and 9,100 packages respectively. The first publish request started a few minutes before the second one, and the second one only ran very briefly before the exception occurred and aptly as a whole exited.

If it is relevant, this is publishing snapshots, request body:

{ 
  "Snapshots": [ {
     "Name": "${snapshot_name}", 
     "Component": "main" 
   }], 
  "Signing": {  
    "Skip": true 
  }
}