thanos gateway/compact: Upgrade from 0.26 to 0.28 results in Azure BlobNotFound errors

gateway/compact: Upgrade from 0.26 to 0.28 results in Azure BlobNotFound errors

Open maartengo opened this issue 3 years ago • 2 comments

Thanos, Prometheus and Golang version used:

Thanos: from 0.26 -> 0.28 (docker.io/bitnami/thanos:0.26.0 -> docker.io/bitnami/thanos:0.28.0-scratch-r0)
- Deployed using the bitnami chart
Prometheus: 2.37.0 (quay.io/prometheus/prometheus:v2.37.0)
- Deployed using prometheus-community chart

Object Storage Provider: Azure, @vglafirov

What happened: When upgrading from 0.26 to 0.28 we get errors for the Thanos compactor and the Thanos storage gateway. Sometimes the gateway fails to come online, but other times it comes online while logging warnings. The errors are shown below. The compactor gets stuck in a crashloopbackoff on startup.

What you expected to happen: No errors to occur and business as usual

How to reproduce it (as minimally and precisely as possible): It happens for us by:

Deploying 0.26
Upgrading to 0.28

Full logs to relevant components:

Thanos Compactor:

Logs

level=info ts=2022-09-05T08:55:22.450035066Z caller=factory.go:50 msg="loading bucket configuration"
level=info ts=2022-09-05T08:55:22.62906077Z caller=compact.go:245 msg="deduplication.replica-label specified, enabling vertical compaction" dedupReplicaLabels=prometheus_replica
level=info ts=2022-09-05T08:55:22.629106374Z caller=compact.go:250 msg="vertical compaction is enabled" compact.enable-vertical-compaction=true
level=info ts=2022-09-05T08:55:22.629355691Z caller=compact.go:389 msg="retention policy of raw samples is enabled" duration=2160h0m0s
level=info ts=2022-09-05T08:55:22.629377892Z caller=compact.go:396 msg="retention policy of 5 min aggregated samples is enabled" duration=2160h0m0s
level=info ts=2022-09-05T08:55:22.629393293Z caller=compact.go:399 msg="retention policy of 1 hour aggregated samples is enabled" duration=2160h0m0s
level=info ts=2022-09-05T08:55:22.630720184Z caller=compact.go:617 msg="starting compact node"
level=info ts=2022-09-05T08:55:22.630753186Z caller=intrumentation.go:56 msg="changing probe status" status=ready
level=info ts=2022-09-05T08:55:22.631215217Z caller=intrumentation.go:75 msg="changing probe status" status=healthy
level=info ts=2022-09-05T08:55:22.631237519Z caller=http.go:73 service=http/server component=compact msg="listening for requests and metrics" address=0.0.0.0:10902
level=info ts=2022-09-05T08:55:22.631411531Z caller=compact.go:1287 msg="start sync of metas"
level=info ts=2022-09-05T08:55:22.63140623Z caller=tls_config.go:195 service=http/server component=compact msg="TLS is disabled." http2=false
level=error ts=2022-09-05T08:55:22.911353514Z caller=runutil.go:100 msg="function failed. Retrying in next tick" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [799b3802-ba04-432e-59e8-5d2f96c1cbda]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:55:22 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:55:21 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [799b3802-ba04-432e-59e8-5d2f96c1cbda]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a5151894-d01e-005a-7205-c13298000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:55:22.921552009Z caller=intrumentation.go:67 msg="changing probe status" status=not-ready reason="syncing metas: incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [816f7f8b-22e0-499d-78c3-b54b798ecc55]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:55:22 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:55:21 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [816f7f8b-22e0-499d-78c3-b54b798ecc55]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [744484ed-201e-0071-5905-c186bc000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=info ts=2022-09-05T08:55:22.921602513Z caller=http.go:84 service=http/server component=compact msg="internal server is shutting down" err="syncing metas: incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [816f7f8b-22e0-499d-78c3-b54b798ecc55]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:55:22 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:55:21 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [816f7f8b-22e0-499d-78c3-b54b798ecc55]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [744484ed-201e-0071-5905-c186bc000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=error ts=2022-09-05T08:55:22.922279659Z caller=compact.go:499 msg="retriable error" err="compaction: sync: BaseFetcher: iter bucket: cannot list hierarchy blobs with prefix  (iteration #1): Get \"https://REDACTED.blob.core.windows.net/thanosstoragecontainer?comp=list&delimiter=%2F&restype=container&timeout=61\": context canceled"
level=info ts=2022-09-05T08:55:22.922803095Z caller=http.go:103 service=http/server component=compact msg="internal server is shutdown gracefully" err="syncing metas: incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [816f7f8b-22e0-499d-78c3-b54b798ecc55]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:55:22 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:55:21 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [816f7f8b-22e0-499d-78c3-b54b798ecc55]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [744484ed-201e-0071-5905-c186bc000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=info ts=2022-09-05T08:55:22.922863299Z caller=intrumentation.go:81 msg="changing probe status" status=not-healthy reason="syncing metas: incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [816f7f8b-22e0-499d-78c3-b54b798ecc55]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:55:22 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:55:21 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [816f7f8b-22e0-499d-78c3-b54b798ecc55]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [744484ed-201e-0071-5905-c186bc000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=error ts=2022-09-05T08:55:22.922976006Z caller=main.go:158 err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [816f7f8b-22e0-499d-78c3-b54b798ecc55]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:55:22 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:55:21 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [816f7f8b-22e0-499d-78c3-b54b798ecc55]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [744484ed-201e-0071-5905-c186bc000000]\n   X-Ms-Version: [2020-04-08]\n\n\n\nsyncing metas\nmain.runCompact.func6\n\t/app/cmd/thanos/compact.go:410\ngithub.com/thanos-io/thanos/pkg/runutil.Repeat\n\t/app/pkg/runutil/runutil.go:74\nmain.runCompact.func12\n\t/app/cmd/thanos/compact.go:539\ngithub.com/oklog/run.(*Group).Run.func1\n\t/go/pkg/mod/github.com/oklog/[email protected]/group.go:38\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1571\ncompact command failed\nmain.main\n\t/app/cmd/thanos/main.go:158\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1571"

Thanos storage gateway, a running instance:

Logs

level=info ts=2022-09-05T07:13:52.722232847Z caller=factory.go:50 msg="loading bucket configuration"
level=info ts=2022-09-05T07:13:57.812528217Z caller=inmemory.go:174 msg="created in-memory index cache" maxItemSizeBytes=131072000 maxSizeBytes=262144000 maxItems=maxInt
level=info ts=2022-09-05T07:13:57.812911422Z caller=options.go:26 protocol=gRPC msg="disabled TLS, key and cert must be set to enable"
level=info ts=2022-09-05T07:13:57.81437574Z caller=store.go:447 msg="starting store node"
level=info ts=2022-09-05T07:13:57.814506242Z caller=store.go:364 msg="initializing bucket store"
level=info ts=2022-09-05T07:13:57.814537942Z caller=intrumentation.go:75 msg="changing probe status" status=healthy
level=info ts=2022-09-05T07:13:57.814570642Z caller=http.go:73 service=http/server component=store msg="listening for requests and metrics" address=0.0.0.0:10902
level=info ts=2022-09-05T07:13:57.814897746Z caller=tls_config.go:195 service=http/server component=store msg="TLS is disabled." http2=false
level=info ts=2022-09-05T07:13:58.025495838Z caller=fetcher.go:470 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=210.939396ms duration_ms=210 cached=43 returned=26 partial=0
level=info ts=2022-09-05T07:13:58.031001706Z caller=bucket.go:580 msg="loaded new block" elapsed=5.356166ms id=01GC5V6MHC2B45MBFZVRMCD30V
level=info ts=2022-09-05T07:13:58.031357411Z caller=bucket.go:580 msg="loaded new block" elapsed=5.61867ms id=01GC5SEB1T36XBAMJBW2PDR94D
level=info ts=2022-09-05T07:13:58.054790899Z caller=bucket.go:580 msg="loaded new block" elapsed=5.976874ms id=01GC5JJKSSJGW5N1P0MGJP6PTB
level=info ts=2022-09-05T07:13:58.171724638Z caller=bucket.go:580 msg="loaded new block" elapsed=116.880038ms id=01GC5SEB0D6R6X0CJEN3FBCC42
level=info ts=2022-09-05T07:13:58.393568168Z caller=bucket.go:580 msg="loaded new block" elapsed=7.431391ms id=01GC5JJKRFA11W4SF0MJ6NQQF9
level=info ts=2022-09-05T07:13:58.519934624Z caller=bucket.go:580 msg="loaded new block" elapsed=279.839644ms id=01GC5JJKT84MR2XVDKV5KE1Z7P
level=info ts=2022-09-05T07:13:58.543129709Z caller=bucket.go:580 msg="loaded new block" elapsed=149.48664ms id=01GC5V8W3M89TQ3V7Q6Y2S1Y1E
level=info ts=2022-09-05T07:13:58.553384235Z caller=bucket.go:580 msg="loaded new block" elapsed=418.566351ms id=01GC0XBCAE0ZHW8M2CYMCVTSVV
level=info ts=2022-09-05T07:13:58.574670197Z caller=bucket.go:580 msg="loaded new block" elapsed=297.309159ms id=01G8SXKXQ21M8434MWJXNNWMYF
level=info ts=2022-09-05T07:13:58.590177688Z caller=bucket.go:580 msg="loaded new block" elapsed=559.112181ms id=01G8FNWR87DXPG7VTV7XAHTPD8
level=info ts=2022-09-05T07:13:58.593167425Z caller=bucket.go:580 msg="loaded new block" elapsed=159.577364ms id=01GC5SEB2CVPG51FQBB3FZ0GRV
level=info ts=2022-09-05T07:13:58.594925947Z caller=bucket.go:580 msg="loaded new block" elapsed=569.310707ms id=01G6WMZ292CM8B9EXHBSJKJ7SC
level=info ts=2022-09-05T07:13:58.599933708Z caller=bucket.go:580 msg="loaded new block" elapsed=366.127006ms id=01G8MHYV6H5G95HT57Y7CYMZJ1
level=info ts=2022-09-05T07:13:58.608888919Z caller=bucket.go:580 msg="loaded new block" elapsed=523.129539ms id=01G6WP1SCT0TPCJ5PARRCCZHX2
level=info ts=2022-09-05T07:13:58.617193421Z caller=bucket.go:580 msg="loaded new block" elapsed=531.242739ms id=01G7BCCC6RA81D7BMYP98832AJ
level=info ts=2022-09-05T07:13:58.617784128Z caller=bucket.go:580 msg="loaded new block" elapsed=547.802542ms id=01G8FN1YRK5G8467SMG7ZGB5W2
level=info ts=2022-09-05T07:13:58.618399836Z caller=bucket.go:580 msg="loaded new block" elapsed=578.965726ms id=01G6WT2SXDCEZGEXR6BPJP02HS
level=info ts=2022-09-05T07:13:58.63659836Z caller=bucket.go:580 msg="loaded new block" elapsed=500.577461ms id=01G7BBM66X6E6SFDJ1EDF17RK6
level=info ts=2022-09-05T07:13:58.646320179Z caller=bucket.go:580 msg="loaded new block" elapsed=620.668139ms id=01GAQQBRKTTEZZJ8M1NRYP8P4A
level=info ts=2022-09-05T07:13:58.65860363Z caller=bucket.go:580 msg="loaded new block" elapsed=627.202319ms id=01G6WSVCYHES56J3KND0RG2NSX
level=info ts=2022-09-05T07:13:58.664390002Z caller=bucket.go:580 msg="loaded new block" elapsed=492.592363ms id=01G8FM7ESEKWEPFHKCYHJF7NMT
level=info ts=2022-09-05T07:13:58.665766619Z caller=bucket.go:580 msg="loaded new block" elapsed=620.112033ms id=01G6WS0ECE5GECTKX507P9JBR2
level=info ts=2022-09-05T07:13:58.667141836Z caller=bucket.go:580 msg="loaded new block" elapsed=532.413953ms id=01G9KEPNB4KWXPCDN67W294WN0
level=info ts=2022-09-05T07:13:58.667710343Z caller=bucket.go:580 msg="loaded new block" elapsed=641.884101ms id=01G6WQZR07EZ9MXVT1X8NDHT6Q
level=info ts=2022-09-05T07:13:58.670519477Z caller=bucket.go:580 msg="loaded new block" elapsed=535.866795ms id=01GBVS0M4CWB7SCSBG45YJ2YXA
level=info ts=2022-09-05T07:13:58.699049228Z caller=bucket.go:580 msg="loaded new block" elapsed=178.968702ms id=01G7BD8V3QSJ6ZE6NMTNSA71B7
level=info ts=2022-09-05T07:13:58.699370332Z caller=store.go:370 msg="bucket store ready" init_duration=884.82029ms
level=info ts=2022-09-05T07:13:58.699553434Z caller=intrumentation.go:56 msg="changing probe status" status=ready
level=info ts=2022-09-05T07:13:58.699677936Z caller=grpc.go:131 service=gRPC/server component=store msg="listening for serving gRPC" address=0.0.0.0:10901
level=info ts=2022-09-05T07:13:58.737799005Z caller=fetcher.go:470 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=38.385072ms duration_ms=38 cached=43 returned=26 partial=0
level=info ts=2022-09-05T07:16:58.926270721Z caller=fetcher.go:470 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=225.830237ms duration_ms=225 cached=43 returned=26 partial=0
level=info ts=2022-09-05T07:19:58.890196372Z caller=fetcher.go:470 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=190.666513ms duration_ms=190 cached=43 returned=26 partial=0
level=info ts=2022-09-05T07:22:58.841743594Z caller=fetcher.go:470 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=141.617293ms duration_ms=141 cached=43 returned=26 partial=0
level=info ts=2022-09-05T07:25:58.868688738Z caller=fetcher.go:470 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=169.154398ms duration_ms=169 cached=43 returned=26 partial=0
level=info ts=2022-09-05T07:28:58.91099943Z caller=fetcher.go:470 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=210.658223ms duration_ms=210 cached=43 returned=26 partial=0
level=info ts=2022-09-05T07:31:59.041279169Z caller=fetcher.go:470 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=341.143276ms duration_ms=341 cached=43 returned=26 partial=0
level=warn ts=2022-09-05T07:34:59.12301001Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [a01023ce-a807-498c-5789-9b8794508885]\n   X-Ms-Date: [Mon, 05 Sep 2022 07:34:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 07:34:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [a01023ce-a807-498c-5789-9b8794508885]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d8380063-d01e-005a-5ffa-c03298000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T07:37:59.072228693Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6b5301e6-88f1-473c-72d2-d4e784602ceb]\n   X-Ms-Date: [Mon, 05 Sep 2022 07:37:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 07:37:57 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6b5301e6-88f1-473c-72d2-d4e784602ceb]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [0a06f748-301e-003c-2bfa-c08ab6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T07:40:58.872445673Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [f101bbe7-03bb-46bb-5486-551cf8cecc36]\n   X-Ms-Date: [Mon, 05 Sep 2022 07:40:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 07:40:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [f101bbe7-03bb-46bb-5486-551cf8cecc36]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [37bc0a88-401e-001f-6cfa-c06683000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T07:43:58.907755176Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [b5d2741b-c096-4b7b-5900-9c2cbcc58324]\n   X-Ms-Date: [Mon, 05 Sep 2022 07:43:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 07:43:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [b5d2741b-c096-4b7b-5900-9c2cbcc58324]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [0a1226a3-301e-003c-74fb-c08ab6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T07:46:58.908268234Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1ed8014d-bf5c-471d-68ae-3544e1bd1516]\n   X-Ms-Date: [Mon, 05 Sep 2022 07:46:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 07:46:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1ed8014d-bf5c-471d-68ae-3544e1bd1516]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8534b6ea-701e-0040-49fb-c08caf000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T07:49:58.84606764Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [b3c3f3a5-e73d-46fd-77a0-e756a26f34f1]\n   X-Ms-Date: [Mon, 05 Sep 2022 07:49:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 07:49:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [b3c3f3a5-e73d-46fd-77a0-e756a26f34f1]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [60ee9a65-a01e-0055-5cfc-c0bb87000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T07:52:59.002041358Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [41ade815-5aa8-45c0-7970-d66a4b9ae93e]\n   X-Ms-Date: [Mon, 05 Sep 2022 07:52:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 07:52:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [41ade815-5aa8-45c0-7970-d66a4b9ae93e]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [589098a8-701e-006c-20fc-c0e985000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T07:55:58.931177566Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [afb9692b-2425-4389-50ea-2e5a2883651b]\n   X-Ms-Date: [Mon, 05 Sep 2022 07:55:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 07:55:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [afb9692b-2425-4389-50ea-2e5a2883651b]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [63f705b3-401e-0067-34fc-c0cc92000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T07:58:58.928575459Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [11ae3c17-5c5c-4d2e-5375-fb8018e4bff3]\n   X-Ms-Date: [Mon, 05 Sep 2022 07:58:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 07:58:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [11ae3c17-5c5c-4d2e-5375-fb8018e4bff3]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d2170e32-201e-0019-7ffd-c09c8f000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=info ts=2022-09-05T08:01:59.233836855Z caller=bucket.go:580 msg="loaded new block" elapsed=321.402354ms id=01GC60A29S7C5BGTW0H2912MMD
level=info ts=2022-09-05T08:01:59.235369874Z caller=bucket.go:580 msg="loaded new block" elapsed=323.093275ms id=01GC60A28DDY2EZSCS3XHWXKG0
level=info ts=2022-09-05T08:01:59.288391943Z caller=bucket.go:580 msg="loaded new block" elapsed=376.201345ms id=01GC622AJQ3C100XNNHV8AK2PS
level=info ts=2022-09-05T08:01:59.332459099Z caller=bucket.go:580 msg="loaded new block" elapsed=420.116199ms id=01GC60A2A9Y02TB7VKB31Y8Y66
level=warn ts=2022-09-05T08:01:59.332599001Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [d9bb4fb7-ea19-46c0-4dee-44e5ba61a72f]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:01:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:01:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [d9bb4fb7-ea19-46c0-4dee-44e5ba61a72f]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [aded70a1-301e-0068-07fd-c0458d000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:04:58.823108096Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6fd552cd-e23d-4e26-4465-c53c06a065f5]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:04:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:04:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6fd552cd-e23d-4e26-4465-c53c06a065f5]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [5a1d14c5-001e-0027-64fe-c01f83000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:07:58.890830245Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [ba249cab-de74-489c-4d89-db0f9bbe2f1d]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:07:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:07:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [ba249cab-de74-489c-4d89-db0f9bbe2f1d]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [6c5844ff-201e-0009-2efe-c02cad000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:10:58.963631707Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [a90da3da-d3bc-41e7-5cb2-6d2fe99c3676]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:10:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:10:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [a90da3da-d3bc-41e7-5cb2-6d2fe99c3676]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d3b0fdb9-301e-0068-38ff-c0458d000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:13:59.00034582Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [128467b7-044c-4063-483e-8c532e955fd0]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:13:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:13:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [128467b7-044c-4063-483e-8c532e955fd0]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [7510739c-f01e-0058-7eff-c0649c000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:16:58.896463366Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7f5a3222-79b1-4017-6017-c5ceb9132ff5]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:16:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:16:57 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7f5a3222-79b1-4017-6017-c5ceb9132ff5]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [c9945805-e01e-0039-38ff-c00dbc000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:19:58.895053683Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [96a0e47a-e9d4-4233-775b-5b7ee1fc4082]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:19:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:19:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [96a0e47a-e9d4-4233-775b-5b7ee1fc4082]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [941e6ee9-d01e-0022-3900-c19889000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:22:58.833904124Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [89bcf0c3-59ad-401e-77e0-408c6cdef99d]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:22:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:22:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [89bcf0c3-59ad-401e-77e0-408c6cdef99d]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d886d3a1-d01e-005a-6700-c13298000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:25:58.895616686Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [06b14aa3-8416-4b95-6311-e86192f9e318]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:25:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:25:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [06b14aa3-8416-4b95-6311-e86192f9e318]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [0e677c2b-b01e-005c-6401-c1c894000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:28:58.907718489Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8560fec6-e43b-4763-754f-38d0405ed6d7]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:28:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:28:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8560fec6-e43b-4763-754f-38d0405ed6d7]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [188341b7-b01e-0060-5d01-c11d9c000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:31:59.097816647Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e0e77648-39bb-4684-7320-d4bda5d92a6b]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:31:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:31:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [e0e77648-39bb-4684-7320-d4bda5d92a6b]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [7ced6caa-301e-002c-6601-c13a94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:34:58.899224943Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e9d339c7-6011-4693-5dab-5c087ae87220]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:34:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:34:57 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [e9d339c7-6011-4693-5dab-5c087ae87220]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [59ccfc89-101e-0046-4702-c176a3000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:37:58.917387827Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [4f3772f7-7b5c-44f7-6259-70fe86428786]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:37:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:37:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [4f3772f7-7b5c-44f7-6259-70fe86428786]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [762a5b90-601e-0049-2f02-c1ffbc000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:40:58.892704536Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [46f9decd-f5c5-4fab-77fb-e28183fa419c]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:40:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:40:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [46f9decd-f5c5-4fab-77fb-e28183fa419c]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [58dce8cd-201e-0071-2a03-c186bc000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:43:58.887077088Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [9ca2bf37-ed30-4b5e-4abe-415725bf59ae]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:43:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:43:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [9ca2bf37-ed30-4b5e-4abe-415725bf59ae]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [115d9dfe-901e-0036-1703-c184a3000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:46:58.885337761Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6c7159b8-96ab-4f80-6ded-33a12ed33b9f]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:46:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:46:57 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6c7159b8-96ab-4f80-6ded-33a12ed33b9f]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d7ab1770-d01e-0066-0904-c1e790000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:49:58.892696919Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [b4e87b66-f542-4d0f-52f8-c15f0e74fa19]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:49:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:49:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [b4e87b66-f542-4d0f-52f8-c15f0e74fa19]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [169b5798-601e-000d-4904-c180a5000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:52:58.862515069Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [39015f94-3885-4443-7887-b9b7d5a215ef]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:52:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:52:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [39015f94-3885-4443-7887-b9b7d5a215ef]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d53bfd97-101e-006a-2d04-c11389000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"
level=warn ts=2022-09-05T08:55:58.872623914Z caller=store.go:375 msg="syncing blocks failed" err="incomplete view: meta.json file exists: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: cannot get properties for Azure blob, address: 01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://REDACTED.blob.core.windows.net/thanosstoragecontainer/01GC5JJKSSJGW5N1P0MGJP6PTB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [9d1f092b-a628-4f71-7ce8-1cb3ba22b455]\n   X-Ms-Date: [Mon, 05 Sep 2022 08:55:58 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 05 Sep 2022 08:55:58 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [9d1f092b-a628-4f71-7ce8-1cb3ba22b455]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a1cb9405-801e-0003-6105-c122b8000000]\n   X-Ms-Version: [2020-04-08]\n\n\n"

Anything else we need to know: We skipped 0.27 because of the race condition bug in the Thanos Querier

The blob that gives the errors looks as follows:

Sep 05 '22 09:09 maartengo

we also had quite a lot of azure issues since 0.28:

level=warn ts=2022-09-06T13:14:07.754975677Z caller=intrumentation.go:67 msg="changing probe status" status=not-ready reason="could not sync metas: incomplete view: 8 errors: meta.json file exists: 01GC2BJQ3GCAQCA6BCKR3M4BBT/meta.json: cannot get properties for Azure blob, address: 01GC2BJQ3GCAQCA6BCKR3M4BBT/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://1aargoststne.privatelink.blob.core.windows.net/platform-argos-amadeus-com/01GC2BJQ3GCAQCA6BCKR3M4BBT/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e984428b-03d2-4993-5584-81b8255d5cc3]\n   X-Ms-Date: [Tue, 06 Sep 2022 13:14:05 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.
[...]

Sep 06 '22 13:09 ahurtaud

I also saw this in my deployment. I think the deletion-mark is a sign that compactor was going to delete the blob. Manually deleting the blob resolved the issue for me.

Edit: This issue kept happening after the manual cleanup. Downgraded to 0.27 and the issue went away. Also on Azure.

Sep 13 '22 14:09 dsg-anthony

Having similar issue with thanos store, but in our example it falters at two months old containers that for some reason are missing meta.json

{"address":"0.0.0.0:10901","caller":"grpc.go:131","component":"store","level":"info","msg":"listening for serving gRPC","service":"gRPC/server","ts":"2022-10-10T05:23:11.667115507Z"}
{"caller":"http.go:84","component":"store","err":"bucket store initial sync: sync block: incomplete view: 84 errors: meta.json file exists: 01G9XPA9M2RQ7B54SKHVM0NCJZ/meta.json: cannot get properties for Azure blob, address: 01G9XPA9M2RQ7B54SKHVM0NCJZ/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9XPA9M2RQ7B54SKHVM0NCJZ/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [f24a384e-aa1d-471f-450a-b6c4dfa6639b]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [f24a384e-aa1d-471f-450a-b6c4dfa6639b]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a1c-101e-0059-5e68-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9Z6CA5SE71KZ2CQYRJ2RCZ1/meta.json: cannot get properties for Azure blob, address: 01G9Z6CA5SE71KZ2CQYRJ2RCZ1/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9Z6CA5SE71KZ2CQYRJ2RCZ1/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [f6384e66-8987-41e1-6bf6-6b115c817c27]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [f6384e66-8987-41e1-6bf6-6b115c817c27]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a1e-101e-0059-6068-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9Y41NWGEFTBGZJ1DFKTYZP9/meta.json: cannot get properties for Azure blob, address: 01G9Y41NWGEFTBGZJ1DFKTYZP9/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9Y41NWGEFTBGZJ1DFKTYZP9/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [51d3aafe-b3cf-4e30-7fa6-617c4d82ae67]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [51d3aafe-b3cf-4e30-7fa6-617c4d82ae67]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a21-101e-0059-6368-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9YZGJXTA3W9D9490ZQNW3BT/meta.json: cannot get properties for Azure blob, address: 01G9YZGJXTA3W9D9490ZQNW3BT/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9YZGJXTA3W9D9490ZQNW3BT/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7705d09f-6558-4ca9-7488-b47bac6835eb]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7705d09f-6558-4ca9-7488-b47bac6835eb]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a2f-101e-0059-7168-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9YAXFC2SB69S33JJ7MTSF71/meta.json: cannot get properties for Azure blob, address: 01G9YAXFC2SB69S33JJ7MTSF71/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9YAXFC2SB69S33JJ7MTSF71/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8b8afa09-c148-4237-5755-a595dfa79b85]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8b8afa09-c148-4237-5755-a595dfa79b85]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a0b-b01e-0039-4c68-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA08PYDW283XJXDB5QMH4AXF/meta.json: cannot get properties for Azure blob, address: 01GA08PYDW283XJXDB5QMH4AXF/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA08PYDW283XJXDB5QMH4AXF/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [f39c15c8-e4ed-45af-68c2-8253424629f0]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [f39c15c8-e4ed-45af-68c2-8253424629f0]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [6228954c-f01e-0058-3b68-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA2M7Y4DD9Q3T2DEAC58M3VE/meta.json: cannot get properties for Azure blob, address: 01GA2M7Y4DD9Q3T2DEAC58M3VE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA2M7Y4DD9Q3T2DEAC58M3VE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e7d3ad1e-0e31-4170-4d72-896642ded771]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [e7d3ad1e-0e31-4170-4d72-896642ded771]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbde1-701e-0060-7368-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9XFEJC2VHT53WNR1DAYRMB9/meta.json: cannot get properties for Azure blob, address: 01G9XFEJC2VHT53WNR1DAYRMB9/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9XFEJC2VHT53WNR1DAYRMB9/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6068be36-cb9a-4945-6eb3-b29b93120917]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6068be36-cb9a-4945-6eb3-b29b93120917]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [62289554-f01e-0058-4168-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9ZM3RNTED3E3B0BXSV01F6P/meta.json: cannot get properties for Azure blob, address: 01G9ZM3RNTED3E3B0BXSV01F6P/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9ZM3RNTED3E3B0BXSV01F6P/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8d20d0a3-7ba8-4cc9-7fcd-b2a75243ca8e]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8d20d0a3-7ba8-4cc9-7fcd-b2a75243ca8e]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbdf8-701e-0060-0568-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9YRMVNSVH4BRCR5TXCRWPNC/meta.json: cannot get properties for Azure blob, address: 01G9YRMVNSVH4BRCR5TXCRWPNC/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9YRMVNSVH4BRCR5TXCRWPNC/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [adbeac35-7b2d-47df-5f4a-5b23723311c9]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [adbeac35-7b2d-47df-5f4a-5b23723311c9]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a17-b01e-0039-5668-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9XX5YMDFH453FNNPZ3CQ4NX/meta.json: cannot get properties for Azure blob, address: 01G9XX5YMDFH453FNNPZ3CQ4NX/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9XX5YMDFH453FNNPZ3CQ4NX/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [48764922-f654-4697-411e-2dd25f7a6b5f]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [48764922-f654-4697-411e-2dd25f7a6b5f]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a3c-101e-0059-7c68-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA38V643FP4ZFA3M0MG0SM0Y/meta.json: cannot get properties for Azure blob, address: 01GA38V643FP4ZFA3M0MG0SM0Y/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA38V643FP4ZFA3M0MG0SM0Y/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [ecc1bb44-5566-444e-4a65-1e56ec80dd08]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [ecc1bb44-5566-444e-4a65-1e56ec80dd08]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9f85c27a-a01e-0004-4d68-dc0b34000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA1RS14EWV9KE2SHAGQSETYC/meta.json: cannot get properties for Azure blob, address: 01GA1RS14EWV9KE2SHAGQSETYC/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA1RS14EWV9KE2SHAGQSETYC/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1c0f13e1-84b6-44e9-724d-e453e5559aae]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1c0f13e1-84b6-44e9-724d-e453e5559aae]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true2022-10-10T05:23:11.668817677Z ]\n   X-Ms-Request-Id: [a7842e34-901e-0007-5c68-dc0833000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA2V3QEPZAFZJFF74FVWX1ZC/meta.json: cannot get properties for Azure blob, address: 01GA2V3QEPZAFZJFF74FVWX1ZC/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA2V3QEPZAFZJFF74FVWX1ZC/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [99c01cb6-0561-4174-7a4b-bd05e429bf9b]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [99c01cb6-0561-4174-7a4b-bd05e429bf9b]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2955-d01e-0000-4368-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA2DC6WGXZCZH6ZKN02AT5ZT/meta.json: cannot get properties for Azure blob, address: 01GA2DC6WGXZCZH6ZKN02AT5ZT/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA2DC6WGXZCZH6ZKN02AT5ZT/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [810271c5-0a85-4aba-5054-f64b1f12587d]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [810271c5-0a85-4aba-5054-f64b1f12587d]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [58d5b528-e01e-0047-2268-dc21dd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA01V75TAED4G99MBGBX6CC1/meta.json: cannot get properties for Azure blob, address: 01GA01V75TAED4G99MBGBX6CC1/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA01V75TAED4G99MBGBX6CC1/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [9dcacf22-cc21-4e20-6d9c-a914df4d38ca]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [9dcacf22-cc21-4e20-6d9c-a914df4d38ca]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [90bff55d-001e-002b-4568-dc8a0e000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA0PECWH35X9SKFC91AZ56T0/meta.json: cannot get properties for Azure blob, address: 01GA0PECWH35X9SKFC91AZ56T0/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA0PECWH35X9SKFC91AZ56T0/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [0996f21f-3797-4397-63b1-6ee3a5a3e9ab]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [0996f21f-3797-4397-63b1-6ee3a5a3e9ab]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2964-d01e-0000-4e68-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9YHS6M2TS082AG8P34W41TE/meta.json: cannot get properties for Azure blob, address: 01G9YHS6M2TS082AG8P34W41TE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9YHS6M2TS082AG8P34W41TE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [59ccd192-f0c6-4f69-6f6e-2681718f5472]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [59ccd192-f0c6-4f69-6f6e-2681718f5472]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a19-b01e-0039-5868-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA0XA455ATQTK8AZQ2P9RCG9/meta.json: cannot get properties for Azure blob, address: 01GA0XA455ATQTK8AZQ2P9RCG9/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA0XA455ATQTK8AZQ2P9RCG9/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6b588fe1-9d7a-4c11-5dda-5dcd80a21e5c]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6b588fe1-9d7a-4c11-5dda-5dcd80a21e5c]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a41-101e-0059-0168-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA3XEASTBFPY3CWQ80GRXKPW/meta.json: cannot get properties for Azure blob, address: 01GA3XEASTBFPY3CWQ80GRXKPW/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA3XEASTBFPY3CWQ80GRXKPW/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [99f4d2dd-e83e-4343-58bd-7f3e0d744b72]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [99f4d2dd-e83e-4343-58bd-7f3e0d744b72]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a5a87dd6-d01e-000b-4668-dce6c2000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA0FJNMP0YXE1V7RXGDS0ZPW/meta.json: cannot get properties for Azure blob, address: 01GA0FJNMP0YXE1V7RXGDS0ZPW/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA0FJNMP0YXE1V7RXGDS0ZPW/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [87299801-ce62-4fdd-594f-b419d9233dcf]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [87299801-ce62-4fdd-594f-b419d9233dcf]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbdfe-701e-0060-0b68-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9ZD81CS12222EQ6Z6A1GKZA/meta.json: cannot get properties for Azure blob, address: 01G9ZD81CS12222EQ6Z6A1GKZA/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9ZD81CS12222EQ6Z6A1GKZA/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [5565a2c7-a204-4fb2-5606-e82e971e7865]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [5565a2c7-a204-4fb2-5606-e82e971e7865]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [62289558-f01e-0058-4568-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA4ZRZ1VCGPNJDZTCWYF0HB3/meta.json: cannot get properties for Azure blob, address: 01GA4ZRZ1VCGPNJDZTCWYF0HB3/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA4ZRZ1VCGPNJDZTCWYF0HB3/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [64fbbfae-e2cd-484c-4789-1139bc5e727d]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [64fbbfae-e2cd-484c-4789-1139bc5e727d]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [90bff567-001e-002b-4b68-dc8a0e000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA26GHW3J2EPZMPBHDGNSMP6/meta.json: cannot get properties for Azure blob, address: 01GA26GHW3J2EPZMPBHDGNSMP6/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA26GHW3J2EPZMPBHDGNSMP6/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [2f9c2923-8e5d-4041-4076-b53256b24a97]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [2f9c2923-8e5d-4041-4076-b53256b24a97]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [222c6d74-a01e-0069-7368-dca11a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA4B5S9TTCDB7VBV8XQHXKH9/meta.json: cannot get properties for Azure blob, address: 01GA4B5S9TTCDB7VBV8XQHXKH9/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA4B5S9TTCDB7VBV8XQHXKH9/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [3864d7f7-dd88-49c2-483a-b794fbbca898]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [3864d7f7-dd88-49c2-483a-b794fbbca898]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [fdceb77c-a01e-002d-1768-dc7d76000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA1ZMT5W2PGA9CSX6JVBJ495/meta.json: cannot get properties for Azure blob, address: 01GA1ZMT5W2PGA9CSX6JVBJ495/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA1ZMT5W2PGA9CSX6JVBJ495/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [718cd10c-fcd8-4c8f-78a7-f45a2eaf2561]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [718cd10c-fcd8-4c8f-78a7-f45a2eaf2561]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Reques2022-10-10T05:23:11.668852978Z t-Id: [9f85c287-a01e-0004-5668-dc0b34000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA145VD5SF4AW3NGN0JNAHAT/meta.json: cannot get properties for Azure blob, address: 01GA145VD5SF4AW3NGN0JNAHAT/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA145VD5SF4AW3NGN0JNAHAT/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [88f6792e-aa42-4654-6edf-c4c796f9b249]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [88f6792e-aa42-4654-6edf-c4c796f9b249]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2969-d01e-0000-5368-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA4RX7T6FHZ6QC7JDT22NJ3X/meta.json: cannot get properties for Azure blob, address: 01GA4RX7T6FHZ6QC7JDT22NJ3X/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA4RX7T6FHZ6QC7JDT22NJ3X/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [933e5692-6c68-4b17-75da-c03cb760a21f]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [933e5692-6c68-4b17-75da-c03cb760a21f]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [58d5b532-e01e-0047-2968-dc21dd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA31ZEW2Z766F1QDRQ27152T/meta.json: cannot get properties for Azure blob, address: 01GA31ZEW2Z766F1QDRQ27152T/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA31ZEW2Z766F1QDRQ27152T/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [bbf3b06d-be36-4a9c-5170-381f2bcab735]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [bbf3b06d-be36-4a9c-5170-381f2bcab735]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a7842e43-901e-0007-6968-dc0833000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA6FV0XAF6H4K18CWJRSHS8X/meta.json: cannot get properties for Azure blob, address: 01GA6FV0XAF6H4K18CWJRSHS8X/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA6FV0XAF6H4K18CWJRSHS8X/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [2877312f-3245-43a1-6469-ac35a8e9ae36]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [2877312f-3245-43a1-6469-ac35a8e9ae36]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbe01-701e-0060-0e68-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9ZTZFXTRMJJDSNH413BQYK7/meta.json: cannot get properties for Azure blob, address: 01G9ZTZFXTRMJJDSNH413BQYK7/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9ZTZFXTRMJJDSNH413BQYK7/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1e4221f2-59b7-48f7-461f-bb15bc701369]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1e4221f2-59b7-48f7-461f-bb15bc701369]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [5a8b8485-701e-000d-3168-dc11ba000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA4J1GHVF9GAJ5R007J87D8Z/meta.json: cannot get properties for Azure blob, address: 01GA4J1GHVF9GAJ5R007J87D8Z/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA4J1GHVF9GAJ5R007J87D8Z/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [04e5dc3b-b58c-492d-5eef-67863bea9326]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [04e5dc3b-b58c-492d-5eef-67863bea9326]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [91f606cc-101e-003f-3668-dc496a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA1HXB15191WQGBN25FAVAWZ/meta.json: cannot get properties for Azure blob, address: 01GA1HXB15191WQGBN25FAVAWZ/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA1HXB15191WQGBN25FAVAWZ/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [c6e9987f-d049-47cc-7326-5d223689a833]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [c6e9987f-d049-47cc-7326-5d223689a833]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d3a1b6fc-f01e-001c-4c68-dc26a1000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA5DGDHVPKREHMSH9S8BWP64/meta.json: cannot get properties for Azure blob, address: 01GA5DGDHVPKREHMSH9S8BWP64/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA5DGDHVPKREHMSH9S8BWP64/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6679eaf2-b152-4bb9-41d3-1970570e6d75]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6679eaf2-b152-4bb9-41d3-1970570e6d75]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a1a-b01e-0039-5968-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA56MP9TS4A39VFF5XY8Q6BR/meta.json: cannot get properties for Azure blob, address: 01GA56MP9TS4A39VFF5XY8Q6BR/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA56MP9TS4A39VFF5XY8Q6BR/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [326ff393-27a0-4bc9-6e0f-64616c01e3f5]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [326ff393-27a0-4bc9-6e0f-64616c01e3f5]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a43-101e-0059-0368-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA8DMJ5A2JXQPBZMQTYA26G1/meta.json: cannot get properties for Azure blob, address: 01GA8DMJ5A2JXQPBZMQTYA26G1/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA8DMJ5A2JXQPBZMQTYA26G1/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6b58cecc-85ce-40ff-6905-6caa77da2b71]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6b58cecc-85ce-40ff-6905-6caa77da2b71]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2971-d01e-0000-5b68-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA8VC10C1HRPXR441MXDZMC5/meta.json: cannot get properties for Azure blob, address: 01GA8VC10C1HRPXR441MXDZMC5/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA8VC10C1HRPXR441MXDZMC5/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7900eac2-5416-49ef-5e09-7ae531d034ea]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7900eac2-5416-49ef-5e09-7ae531d034ea]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbe07-701e-0060-1468-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA7ZX3NJCX11H3AFRAFKR8WA/meta.json: cannot get properties for Azure blob, address: 01GA7ZX3NJCX11H3AFRAFKR8WA/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA7ZX3NJCX11H3AFRAFKR8WA/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [ff74eb7c-199b-4757-4db5-d52b16e51d7d]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [ff74eb7c-199b-4757-4db5-d52b16e51d7d]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [5a8b8491-701e-000d-3b68-dc11ba000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA5V7W1TNC2D7FFC5Z03MZ8T/meta.json: cannot get properties for Azure blob, address: 01GA5V7W1TNC2D7FFC5Z03MZ8T/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA5V7W1TNC2D7FFC5Z03MZ8T/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [c9298403-3f0b-4d13-7707-bbd9d3dc8e6b]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [c9298403-3f0b-4d13-7707-bbd9d3dc8e6b]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [6228955b-f2022-10-10T05:23:11.668880179Z 01e-0058-4868-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA623K9VGKZ3NP5ME17P3BZD/meta.json: cannot get properties for Azure blob, address: 01GA623K9VGKZ3NP5ME17P3BZD/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA623K9VGKZ3NP5ME17P3BZD/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7426a617-037e-467f-533d-29171c89cdb5]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7426a617-037e-467f-533d-29171c89cdb5]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a5a87dde-d01e-000b-4c68-dce6c2000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA68Z9N8GA5QW506DTCS6W8G/meta.json: cannot get properties for Azure blob, address: 01GA68Z9N8GA5QW506DTCS6W8G/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA68Z9N8GA5QW506DTCS6W8G/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [51965525-95ef-43d4-6417-d78c33ab64d8]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [51965525-95ef-43d4-6417-d78c33ab64d8]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [11f2b2ba-e01e-0021-6d68-dc9387000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA86RTX97WSARZF3P0NJ8YNK/meta.json: cannot get properties for Azure blob, address: 01GA86RTX97WSARZF3P0NJ8YNK/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA86RTX97WSARZF3P0NJ8YNK/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [500b626d-d4ae-4ad9-69fe-787ce0d4b0fe]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [500b626d-d4ae-4ad9-69fe-787ce0d4b0fe]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [91f606d6-101e-003f-3e68-dc496a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA7S1CDSM1JSCPG3XQJKQTZE/meta.json: cannot get properties for Azure blob, address: 01GA7S1CDSM1JSCPG3XQJKQTZE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA7S1CDSM1JSCPG3XQJKQTZE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [2db949de-1f0f-48a4-611f-c413937aa1ba]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [2db949de-1f0f-48a4-611f-c413937aa1ba]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d3a1b70a-f01e-001c-5668-dc26a1000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA5MC4SVTCJZ3AFM5QC6DFJZ/meta.json: cannot get properties for Azure blob, address: 01GA5MC4SVTCJZ3AFM5QC6DFJZ/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA5MC4SVTCJZ3AFM5QC6DFJZ/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [ea97aa04-8c7a-4693-71b9-3b019cccc922]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [ea97aa04-8c7a-4693-71b9-3b019cccc922]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a296a-d01e-0000-5468-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA6PPR5BPT643GRXWD0H4PC5/meta.json: cannot get properties for Azure blob, address: 01GA6PPR5BPT643GRXWD0H4PC5/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA6PPR5BPT643GRXWD0H4PC5/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e6ae680d-8008-45d6-565b-66be53ead41c]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [e6ae680d-8008-45d6-565b-66be53ead41c]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [90bff56c-001e-002b-5068-dc8a0e000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA74E6N80EJBFKQC6R0Y3M6Z/meta.json: cannot get properties for Azure blob, address: 01GA74E6N80EJBFKQC6R0Y3M6Z/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA74E6N80EJBFKQC6R0Y3M6Z/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [afef6995-effd-40ec-5e79-2e4898303116]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [afef6995-effd-40ec-5e79-2e4898303116]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9f85c292-a01e-0004-6168-dc0b34000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA993FG9YK2P18EME6Y48D9Q/meta.json: cannot get properties for Azure blob, address: 01GA993FG9YK2P18EME6Y48D9Q/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA993FG9YK2P18EME6Y48D9Q/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [03b7ef4c-9ea8-41c1-5764-fa691e073ddd]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [03b7ef4c-9ea8-41c1-5764-fa691e073ddd]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a48-101e-0059-0868-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA1B1JMNX2SJ5NKAD4H125SE/meta.json: cannot get properties for Azure blob, address: 01GA1B1JMNX2SJ5NKAD4H125SE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA1B1JMNX2SJ5NKAD4H125SE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [a16bf96a-540f-4ca7-5121-5c842f152dd5]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [a16bf96a-540f-4ca7-5121-5c842f152dd5]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9e25deb7-f01e-003e-4d68-dc4897000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAA4JCGGJ06N5KSHWSQWN5H1/meta.json: cannot get properties for Azure blob, address: 01GAA4JCGGJ06N5KSHWSQWN5H1/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAA4JCGGJ06N5KSHWSQWN5H1/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [d78b0b24-4de5-468a-7bb8-5adf13d5457c]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [d78b0b24-4de5-468a-7bb8-5adf13d5457c]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2975-d01e-0000-5f68-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA9XPN89YERS6VEZYWA0T8DC/meta.json: cannot get properties for Azure blob, address: 01GA9XPN89YERS6VEZYWA0T8DC/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA9XPN89YERS6VEZYWA0T8DC/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [f0b29f9d-171a-48bd-467d-c6a4b44c0d30]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [f0b29f9d-171a-48bd-467d-c6a4b44c0d30]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a1c-b01e-0039-5b68-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA44A21VKSX96NPGFYS0KB62/meta.json: cannot get properties for Azure blob, address: 01GA44A21VKSX96NPGFYS0KB62/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA44A21VKSX96NPGFYS0KB62/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [864e811b-5eee-412a-4a88-a50a71e57c39]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [864e811b-5eee-412a-4a88-a50a71e57c39]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9e25deb8-f01e-003e-4e68-dc4897000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA9FZ6R9TSHNBG5SYFWBK8BE/meta.json: cannot get properties for Azure blob, address: 01GA9FZ6R9TSHNBG5SYFWBK8BE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA9FZ6R9TSHNBG5SYFWBK8BE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [89b0efe8-6c19-4359-585c-845715bfb37f]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [89b0efe8-6c19-4359-585c-845715bfb37f]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [91f606da-101e-003f-4268-dc42022-10-10T05:23:11.668924381Z 96a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAABE3RF318RTPYE3654TQSB/meta.json: cannot get properties for Azure blob, address: 01GAABE3RF318RTPYE3654TQSB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAABE3RF318RTPYE3654TQSB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1fdaead2-f92a-4c61-439f-e20f37c535ad]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1fdaead2-f92a-4c61-439f-e20f37c535ad]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [11f2b2c6-e01e-0021-7568-dc9387000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA927R89CBSZRBN3CWB4AXCZ/meta.json: cannot get properties for Azure blob, address: 01GA927R89CBSZRBN3CWB4AXCZ/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA927R89CBSZRBN3CWB4AXCZ/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [bdd5c08c-4ac1-4919-6b17-7e12bb977ae2]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [bdd5c08c-4ac1-4919-6b17-7e12bb977ae2]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [5a8b8493-701e-000d-3d68-dc11ba000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GABDRR0BSHGD8DXA7M76TGS6/meta.json: cannot get properties for Azure blob, address: 01GABDRR0BSHGD8DXA7M76TGS6/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GABDRR0BSHGD8DXA7M76TGS6/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [badc04fe-676d-4b4b-5f10-b3951b3df224]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [badc04fe-676d-4b4b-5f10-b3951b3df224]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [90bff56e-001e-002b-5268-dc8a0e000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA6XJFD8KK5GE9WNKJ5GVESF/meta.json: cannot get properties for Azure blob, address: 01GA6XJFD8KK5GE9WNKJ5GVESF/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA6XJFD8KK5GE9WNKJ5GVESF/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e179a718-bc15-4b4c-4aff-62a6a088ba24]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [e179a718-bc15-4b4c-4aff-62a6a088ba24]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a7842e4d-901e-0007-7168-dc0833000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA7J5N5AWRMR50J7A10M95TJ/meta.json: cannot get properties for Azure blob, address: 01GA7J5N5AWRMR50J7A10M95TJ/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA7J5N5AWRMR50J7A10M95TJ/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [c00a1c31-11a9-4500-5c14-1379732fd0dc]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [c00a1c31-11a9-4500-5c14-1379732fd0dc]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [222c6d7b-a01e-0069-7868-dca11a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA8MG9DA106CJYWN2Q58SX57/meta.json: cannot get properties for Azure blob, address: 01GA8MG9DA106CJYWN2Q58SX57/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA8MG9DA106CJYWN2Q58SX57/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [644ba597-0228-4b5c-4f02-8495bc68c795]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [644ba597-0228-4b5c-4f02-8495bc68c795]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [58d5b537-e01e-0047-2e68-dc21dd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAB019GAEPNJ45NZ8H9GQ4ER/meta.json: cannot get properties for Azure blob, address: 01GAB019GAEPNJ45NZ8H9GQ4ER/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAB019GAEPNJ45NZ8H9GQ4ER/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [730636c0-c933-4210-473a-6567d16acf03]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [730636c0-c933-4210-473a-6567d16acf03]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2978-d01e-0000-6268-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA7B9XX7J77BANAN003PF7S4/meta.json: cannot get properties for Azure blob, address: 01GA7B9XX7J77BANAN003PF7S4/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA7B9XX7J77BANAN003PF7S4/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8bc15378-2016-4ff9-5029-fd0059cf1c71]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8bc15378-2016-4ff9-5029-fd0059cf1c71]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [fdceb786-a01e-002d-1e68-dc7d76000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GABVG6G9EJDB6G49XYQV1YHD/meta.json: cannot get properties for Azure blob, address: 01GABVG6G9EJDB6G49XYQV1YHD/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GABVG6G9EJDB6G49XYQV1YHD/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [38139b4e-70fc-4aec-5223-34a36069be99]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [38139b4e-70fc-4aec-5223-34a36069be99]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a4a-101e-0059-0a68-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA9PTY0C8N879YTJNJSZ6AWX/meta.json: cannot get properties for Azure blob, address: 01GA9PTY0C8N879YTJNJSZ6AWX/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA9PTY0C8N879YTJNJSZ6AWX/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [45084407-a0a5-4d6b-7920-ff0c17e2aa4a]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [45084407-a0a5-4d6b-7920-ff0c17e2aa4a]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbe0d-701e-0060-1a68-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAAJ9V0AB5R8GYMBA22JP512/meta.json: cannot get properties for Azure blob, address: 01GAAJ9V0AB5R8GYMBA22JP512/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAAJ9V0AB5R8GYMBA22JP512/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [5b1ab4c8-be5e-4dba-4527-7be2c27e5520]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [5b1ab4c8-be5e-4dba-4527-7be2c27e5520]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [62289560-f01e-0058-4c68-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAAS5J8AKG8SPBTPVX00S7NF/meta.json: cannot get properties for Azure blob, address: 01GAAS5J8AKG8SPBTPVX00S7NF/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAAS5J8AKG8SPBTPVX00S7NF/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [01863ecf-2c10-4510-51ad-888ab63cc401]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [01863ecf-2c10-4510-51ad-888ab63cc401]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a5a87de9-d01e-000b-5768-dce6c2000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GACPZ3GANB5Z36PRNRZJD18G/meta.json: cannot get properties for Azure blob, address: 01GACPZ3GANB5Z36PRNRZJD18G/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GACPZ3GANB5Z36PRNRZJD18G/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1a30fc6f-d1a1-4b1d-7108-ccef1f1a8b90]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1a30fc6f-d1a1-4b1d-7108-ccef1f1a8b90]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9e25dec4-f01e-003e-5768-dc4897000000]\n   X-2022-10-10T05:23:11.668952782Z Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GABMMF89ERKX4XFX5HE5ETW7/meta.json: cannot get properties for Azure blob, address: 01GABMMF89ERKX4XFX5HE5ETW7/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GABMMF89ERKX4XFX5HE5ETW7/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [3da0860b-6d22-4163-5f18-0758604ab3cc]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [3da0860b-6d22-4163-5f18-0758604ab3cc]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9f85c29b-a01e-0004-6a68-dc0b34000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAE05F0SPPWHE6JQXQ0ZVBEH/meta.json: cannot get properties for Azure blob, address: 01GAE05F0SPPWHE6JQXQ0ZVBEH/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAE05F0SPPWHE6JQXQ0ZVBEH/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [bf4e4b46-3ed9-495b-6674-9cecac24d20e]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [bf4e4b46-3ed9-495b-6674-9cecac24d20e]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [222c6d7e-a01e-0069-7b68-dca11a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAB6X0RGFTKHJRT11RJAK4D5/meta.json: cannot get properties for Azure blob, address: 01GAB6X0RGFTKHJRT11RJAK4D5/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAB6X0RGFTKHJRT11RJAK4D5/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1983f5eb-bab9-4eaa-5dbd-1727ffe1563c]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1983f5eb-bab9-4eaa-5dbd-1727ffe1563c]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d3a1b70c-f01e-001c-5868-dc26a1000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GADS9QRAWNJ08JW925TW0QRR/meta.json: cannot get properties for Azure blob, address: 01GADS9QRAWNJ08JW925TW0QRR/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GADS9QRAWNJ08JW925TW0QRR/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [885ae365-88a6-4c9f-6dc7-d6a8cd136901]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [885ae365-88a6-4c9f-6dc7-d6a8cd136901]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a7842e55-901e-0007-7868-dc0833000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAEMRMRGSW8A6YN8R1Y0DKJ3/meta.json: cannot get properties for Azure blob, address: 01GAEMRMRGSW8A6YN8R1Y0DKJ3/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAEMRMRGSW8A6YN8R1Y0DKJ3/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [cc064de2-313f-4c21-7103-0537c40a9c76]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [cc064de2-313f-4c21-7103-0537c40a9c76]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [fdceb78b-a01e-002d-2368-dc7d76000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA3FPXC5HTYD1ZKXA5SBTZJG/meta.json: cannot get properties for Azure blob, address: 01GA3FPXC5HTYD1ZKXA5SBTZJG/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA3FPXC5HTYD1ZKXA5SBTZJG/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [ffa9120d-0f6f-4bdf-50b8-5110cff42d2a]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [ffa9120d-0f6f-4bdf-50b8-5110cff42d2a]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [4f3225b2-801e-0031-0268-dca561000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GADBJ989A89HVE6C5GECWC8R/meta.json: cannot get properties for Azure blob, address: 01GADBJ989A89HVE6C5GECWC8R/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GADBJ989A89HVE6C5GECWC8R/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1dfcf24b-defa-4720-450e-f114b429a3cf]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1dfcf24b-defa-4720-450e-f114b429a3cf]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [5a8b8497-701e-000d-4168-dc11ba000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAF2G25JTT4AH82T14VA8WPE/meta.json: cannot get properties for Azure blob, address: 01GAF2G25JTT4AH82T14VA8WPE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAF2G25JTT4AH82T14VA8WPE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [5192d9bf-def5-47d0-5016-900b1c6abbd4]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [5192d9bf-def5-47d0-5016-900b1c6abbd4]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbe16-701e-0060-2268-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAF9BSND4P2FCM0DGN9H4GWN/meta.json: cannot get properties for Azure blob, address: 01GAF9BSND4P2FCM0DGN9H4GWN/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAF9BSND4P2FCM0DGN9H4GWN/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7f54b131-90dc-4e7e-4e44-f21db2bb8fda]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7f54b131-90dc-4e7e-4e44-f21db2bb8fda]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [62289567-f01e-0058-5368-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GACXTTRADRNW8NYTB91K0MH5/meta.json: cannot get properties for Azure blob, address: 01GACXTTRADRNW8NYTB91K0MH5/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GACXTTRADRNW8NYTB91K0MH5/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [65fd5628-2a25-41ac-71eb-eab2724ec8a3]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [65fd5628-2a25-41ac-71eb-eab2724ec8a3]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [11f2b2c9-e01e-0021-7868-dc9387000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAC97N0AF0Y82AGX6AQ21141/meta.json: cannot get properties for Azure blob, address: 01GAC97N0AF0Y82AGX6AQ21141/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAC97N0AF0Y82AGX6AQ21141/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [94ce3cb5-f558-4c55-7613-0d120505993a]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [94ce3cb5-f558-4c55-7613-0d120505993a]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a297b-d01e-0000-6568-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAC2BXRESC989EQF2KS2Z5B6/meta.json: cannot get properties for Azure blob, address: 01GAC2BXRESC989EQF2KS2Z5B6/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAC2BXRESC989EQF2KS2Z5B6/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [c65238eb-6d71-4a12-50f1-f759dd25c424]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [c65238eb-6d71-4a12-50f1-f759dd25c424]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9e25dec3-f01e-003e-5668-dc4897000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAD4PJ0JCTWPS7EWSHW9PWYC/meta.json: cannot get properties for Azure blob, address: 01GAD4PJ0JCTWPS7EWSHW9PWYC/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAD4PJ0JCTWPS7EWSHW9PWYC/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [b203c407-59c4-4e80-5db3-1c0bcb18497b]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [b203c407-59c4-4e80-5db3-1c0bcb18497b]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [91f606dc-101e-003f-4368-dc496a000000]\n   X-Ms-Version: [20202022-10-10T05:23:11.668965383Z -04-08]\n\n\n; meta.json file exists: 01GAE7168F722ZD5ZWPJ59QKQ2/meta.json: cannot get properties for Azure blob, address: 01GAE7168F722ZD5ZWPJ59QKQ2/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAE7168F722ZD5ZWPJ59QKQ2/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8334818b-a7e0-40b3-5e79-996c13cec6e0]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8334818b-a7e0-40b3-5e79-996c13cec6e0]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a297f-d01e-0000-6968-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAEDWXGFSJK9XFGEW72AZJ5S/meta.json: cannot get properties for Azure blob, address: 01GAEDWXGFSJK9XFGEW72AZJ5S/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAEDWXGFSJK9XFGEW72AZJ5S/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [121325d5-9982-4e8f-4fbd-4e6239ad9bb8]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [121325d5-9982-4e8f-4fbd-4e6239ad9bb8]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [58d5b540-e01e-0047-3668-dc21dd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA3PJKHT936S1MY11ATDE4C0/meta.json: cannot get properties for Azure blob, address: 01GA3PJKHT936S1MY11ATDE4C0/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA3PJKHT936S1MY11ATDE4C0/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [b7b180b8-7101-4127-73c8-30a848122042]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [b7b180b8-7101-4127-73c8-30a848122042]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [fec64e63-201e-001e-2168-dc245b000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GACG3C8HK9SGSX2C2V93XCR5/meta.json: cannot get properties for Azure blob, address: 01GACG3C8HK9SGSX2C2V93XCR5/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GACG3C8HK9SGSX2C2V93XCR5/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7e0a6a14-4fb9-41fe-5d45-88014e9bcdb9]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7e0a6a14-4fb9-41fe-5d45-88014e9bcdb9]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a1d-b01e-0039-5c68-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAEVMAXHBS8CB2TBB1FFCN0S/meta.json: cannot get properties for Azure blob, address: 01GAEVMAXHBS8CB2TBB1FFCN0S/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAEVMAXHBS8CB2TBB1FFCN0S/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8b8ac36f-4aee-4bc2-5cb2-99f00b5c4c22]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8b8ac36f-4aee-4bc2-5cb2-99f00b5c4c22]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a4d-101e-0059-0d68-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GADJE0GF3HK0T9CT1P9APKDF/meta.json: cannot get properties for Azure blob, address: 01GADJE0GF3HK0T9CT1P9APKDF/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GADJE0GF3HK0T9CT1P9APKDF/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [9afaf48d-c590-4aad-631f-10721a5a20ed]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [9afaf48d-c590-4aad-631f-10721a5a20ed]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [90bff571-001e-002b-5568-dc8a0e000000]\n   X-Ms-Version: [2020-04-08]\n\n\n","level":"info","msg":"internal server is shutting down","service":"http/server","ts":"2022-10-10T05:23:11.666931699Z"}
{"caller":"intrumentation.go:81","level":"info","msg":"changing probe status","reason":"bucket store initial sync: sync block: incomplete view: 84 errors: meta.json file exists: 01G9XPA9M2RQ7B54SKHVM0NCJZ/meta.json: cannot get properties for Azure blob, address: 01G9XPA9M2RQ7B54SKHVM0NCJZ/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9XPA9M2RQ7B54SKHVM0NCJZ/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [f24a384e-aa1d-471f-450a-b6c4dfa6639b]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [f24a384e-aa1d-471f-450a-b6c4dfa6639b]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a1c-101e-0059-5e68-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9Z6CA5SE71KZ2CQYRJ2RCZ1/meta.json: cannot get properties for Azure blob, address: 01G9Z6CA5SE71KZ2CQYRJ2RCZ1/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9Z6CA5SE71KZ2CQYRJ2RCZ1/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [f6384e66-8987-41e1-6bf6-6b115c817c27]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [f6384e66-8987-41e1-6bf6-6b115c817c27]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a1e-101e-0059-6068-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9Y41NWGEFTBGZJ1DFKTYZP9/meta.json: cannot get properties for Azure blob, address: 01G9Y41NWGEFTBGZJ1DFKTYZP9/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9Y41NWGEFTBGZJ1DFKTYZP9/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [51d3aafe-b3cf-4e30-7fa6-617c4d82ae67]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [51d3aafe-b3cf-4e30-7fa6-617c4d82ae67]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a21-101e-0059-6368-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9YZGJXTA3W9D9490ZQNW3BT/meta.json: cannot get properties for Azure blob, address: 01G9YZGJXTA3W9D9490ZQNW3BT/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9YZGJXTA3W9D9490ZQNW3BT/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7705d09f-6558-4ca9-7488-b47bac6835eb]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7705d09f-6558-4ca9-7488-b47bac6835eb]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a2f-101e-0059-7168-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9YAXFC2SB69S33JJ7MTSF71/meta.json: cannot get properties for Azure blob, address: 01G9YAXFC2SB69S33JJ7MTSF71/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9YAXFC2SB69S33JJ7MTSF71/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8b8afa09-c148-4237-5755-a595dfa79b85]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8b8afa09-c148-4237-5755-a595dfa79b85]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a0b-b01e-0039-4c68-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA08PYDW283XJXDB5QMH4AXF/meta.json: cannot get properties for Azure blob, address: 01GA08PYDW283XJXDB5QMH4AXF/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA08PYDW283XJXDB5QMH4AXF/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [f39c15c8-e4ed-45af-68c2-8253424629f0]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [f39c15c8-e4ed-45af-68c2-8253424629f0]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [6228954c-f01e-0058-3b68-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA2M7Y4DD9Q3T2DEAC58M3VE/meta.json: cannot get properties for Azure blob, address: 01GA2M7Y4DD9Q3T2DEAC58M3VE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA2M7Y4DD9Q3T2DEAC58M3VE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e7d3ad1e-0e31-4170-4d72-896642ded771]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [e7d3ad1e-0e31-4170-4d72-896642ded771]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbde1-701e-0060-7368-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9XFEJC2VHT53WNR1DAYRMB9/meta.json: cannot get properties for Azure blob, address: 01G9XFEJC2VHT53WNR1DAYRMB9/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9XFEJC2VHT53WNR1DAYRMB9/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6068be36-cb9a-4945-6eb3-b29b93120917]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6068be36-cb9a-4945-6eb3-b29b93120917]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [62289554-f01e-0058-4168-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9ZM3RNTED3E3B0BXSV01F6P/meta.json: cannot get properties for Azure blob, address: 01G9ZM3RNTED3E3B0BXSV01F6P/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9ZM3RNTED3E3B0BXSV01F6P/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8d20d0a3-7ba8-4cc9-7fcd-b2a75243ca8e]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8d20d0a3-7ba8-4cc9-7fcd-b2a75243ca8e]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbdf8-701e-0060-0568-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9YRMVNSVH4BRCR5TXCRWPNC/meta.json: cannot get properties for Azure blob, address: 01G9YRMVNSVH4BRCR5TXCRWPNC/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9YRMVNSVH4BRCR5TXCRWPNC/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [adbeac35-7b2d-47df-5f4a-5b23723311c9]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [adbeac35-7b2d-47df-5f4a-5b23723311c9]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a17-b01e-0039-5668-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9XX5YMDFH453FNNPZ3CQ4NX/meta.json: cannot get properties for Azure blob, address: 01G9XX5YMDFH453FNNPZ3CQ4NX/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9XX5YMDFH453FNNPZ3CQ4NX/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [48764922-f654-4697-411e-2dd25f7a6b5f]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [48764922-f654-4697-411e-2dd25f7a6b5f]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a3c-101e-0059-7c68-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA38V643FP4ZFA3M0MG0SM0Y/meta.json: cannot get properties for Azure blob, address: 01GA38V643FP4ZFA3M0MG0SM0Y/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA38V643FP4ZFA3M0MG0SM0Y/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [ecc1bb44-5566-444e-4a65-1e56ec80dd08]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [ecc1bb44-5566-444e-4a65-1e56ec80dd08]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9f85c27a-a01e-0004-4d68-dc0b34000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA1RS14EWV9KE2SHAGQSETYC/meta.json: cannot get properties for Azure blob, address: 01GA1RS14EWV9KE2SHAGQSETYC/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA1RS14EWV9KE2SHAGQSETYC/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1c0f13e1-84b6-44e9-724d-e453e5559aae]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1c0f13e1-84b6-44e9-724d-e453e5559aae]\n   X-Ms-Error-Code: [BlobNot2022-10-10T05:23:11.673497970Z Found]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a7842e34-901e-0007-5c68-dc0833000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA2V3QEPZAFZJFF74FVWX1ZC/meta.json: cannot get properties for Azure blob, address: 01GA2V3QEPZAFZJFF74FVWX1ZC/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA2V3QEPZAFZJFF74FVWX1ZC/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [99c01cb6-0561-4174-7a4b-bd05e429bf9b]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [99c01cb6-0561-4174-7a4b-bd05e429bf9b]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2955-d01e-0000-4368-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA2DC6WGXZCZH6ZKN02AT5ZT/meta.json: cannot get properties for Azure blob, address: 01GA2DC6WGXZCZH6ZKN02AT5ZT/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA2DC6WGXZCZH6ZKN02AT5ZT/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [810271c5-0a85-4aba-5054-f64b1f12587d]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [810271c5-0a85-4aba-5054-f64b1f12587d]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [58d5b528-e01e-0047-2268-dc21dd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA01V75TAED4G99MBGBX6CC1/meta.json: cannot get properties for Azure blob, address: 01GA01V75TAED4G99MBGBX6CC1/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA01V75TAED4G99MBGBX6CC1/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [9dcacf22-cc21-4e20-6d9c-a914df4d38ca]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [9dcacf22-cc21-4e20-6d9c-a914df4d38ca]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [90bff55d-001e-002b-4568-dc8a0e000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA0PECWH35X9SKFC91AZ56T0/meta.json: cannot get properties for Azure blob, address: 01GA0PECWH35X9SKFC91AZ56T0/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA0PECWH35X9SKFC91AZ56T0/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [0996f21f-3797-4397-63b1-6ee3a5a3e9ab]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [0996f21f-3797-4397-63b1-6ee3a5a3e9ab]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2964-d01e-0000-4e68-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9YHS6M2TS082AG8P34W41TE/meta.json: cannot get properties for Azure blob, address: 01G9YHS6M2TS082AG8P34W41TE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9YHS6M2TS082AG8P34W41TE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [59ccd192-f0c6-4f69-6f6e-2681718f5472]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [59ccd192-f0c6-4f69-6f6e-2681718f5472]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a19-b01e-0039-5868-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA0XA455ATQTK8AZQ2P9RCG9/meta.json: cannot get properties for Azure blob, address: 01GA0XA455ATQTK8AZQ2P9RCG9/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA0XA455ATQTK8AZQ2P9RCG9/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6b588fe1-9d7a-4c11-5dda-5dcd80a21e5c]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6b588fe1-9d7a-4c11-5dda-5dcd80a21e5c]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a41-101e-0059-0168-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA3XEASTBFPY3CWQ80GRXKPW/meta.json: cannot get properties for Azure blob, address: 01GA3XEASTBFPY3CWQ80GRXKPW/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA3XEASTBFPY3CWQ80GRXKPW/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [99f4d2dd-e83e-4343-58bd-7f3e0d744b72]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [99f4d2dd-e83e-4343-58bd-7f3e0d744b72]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a5a87dd6-d01e-000b-4668-dce6c2000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA0FJNMP0YXE1V7RXGDS0ZPW/meta.json: cannot get properties for Azure blob, address: 01GA0FJNMP0YXE1V7RXGDS0ZPW/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA0FJNMP0YXE1V7RXGDS0ZPW/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [87299801-ce62-4fdd-594f-b419d9233dcf]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [87299801-ce62-4fdd-594f-b419d9233dcf]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbdfe-701e-0060-0b68-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9ZD81CS12222EQ6Z6A1GKZA/meta.json: cannot get properties for Azure blob, address: 01G9ZD81CS12222EQ6Z6A1GKZA/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9ZD81CS12222EQ6Z6A1GKZA/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [5565a2c7-a204-4fb2-5606-e82e971e7865]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [5565a2c7-a204-4fb2-5606-e82e971e7865]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [62289558-f01e-0058-4568-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA4ZRZ1VCGPNJDZTCWYF0HB3/meta.json: cannot get properties for Azure blob, address: 01GA4ZRZ1VCGPNJDZTCWYF0HB3/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA4ZRZ1VCGPNJDZTCWYF0HB3/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [64fbbfae-e2cd-484c-4789-1139bc5e727d]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [64fbbfae-e2cd-484c-4789-1139bc5e727d]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [90bff567-001e-002b-4b68-dc8a0e000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA26GHW3J2EPZMPBHDGNSMP6/meta.json: cannot get properties for Azure blob, address: 01GA26GHW3J2EPZMPBHDGNSMP6/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA26GHW3J2EPZMPBHDGNSMP6/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [2f9c2923-8e5d-4041-4076-b53256b24a97]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [2f9c2923-8e5d-4041-4076-b53256b24a97]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [222c6d74-a01e-0069-7368-dca11a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA4B5S9TTCDB7VBV8XQHXKH9/meta.json: cannot get properties for Azure blob, address: 01GA4B5S9TTCDB7VBV8XQHXKH9/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA4B5S9TTCDB7VBV8XQHXKH9/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [3864d7f7-dd88-49c2-483a-b794fbbca898]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [3864d7f7-dd88-49c2-483a-b794fbbca898]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [fdceb77c-a01e-002d-1768-dc7d76000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA1ZMT5W2PGA9CSX6JVBJ495/meta.json: cannot get properties for Azure blob, address: 01GA1ZMT5W2PGA9CSX6JVBJ495/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA1ZMT5W2PGA9CSX6JVBJ495/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [718cd10c-fcd8-4c8f-78a7-f45a2eaf2561]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [718cd10c-fcd8-4c8f-78a7-f45a2eaf2561]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-I2022-10-10T05:23:11.673526671Z s-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9f85c287-a01e-0004-5668-dc0b34000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA145VD5SF4AW3NGN0JNAHAT/meta.json: cannot get properties for Azure blob, address: 01GA145VD5SF4AW3NGN0JNAHAT/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA145VD5SF4AW3NGN0JNAHAT/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [88f6792e-aa42-4654-6edf-c4c796f9b249]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [88f6792e-aa42-4654-6edf-c4c796f9b249]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2969-d01e-0000-5368-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA4RX7T6FHZ6QC7JDT22NJ3X/meta.json: cannot get properties for Azure blob, address: 01GA4RX7T6FHZ6QC7JDT22NJ3X/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA4RX7T6FHZ6QC7JDT22NJ3X/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [933e5692-6c68-4b17-75da-c03cb760a21f]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [933e5692-6c68-4b17-75da-c03cb760a21f]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [58d5b532-e01e-0047-2968-dc21dd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA31ZEW2Z766F1QDRQ27152T/meta.json: cannot get properties for Azure blob, address: 01GA31ZEW2Z766F1QDRQ27152T/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA31ZEW2Z766F1QDRQ27152T/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [bbf3b06d-be36-4a9c-5170-381f2bcab735]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [bbf3b06d-be36-4a9c-5170-381f2bcab735]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a7842e43-901e-0007-6968-dc0833000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA6FV0XAF6H4K18CWJRSHS8X/meta.json: cannot get properties for Azure blob, address: 01GA6FV0XAF6H4K18CWJRSHS8X/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA6FV0XAF6H4K18CWJRSHS8X/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [2877312f-3245-43a1-6469-ac35a8e9ae36]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [2877312f-3245-43a1-6469-ac35a8e9ae36]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbe01-701e-0060-0e68-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01G9ZTZFXTRMJJDSNH413BQYK7/meta.json: cannot get properties for Azure blob, address: 01G9ZTZFXTRMJJDSNH413BQYK7/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01G9ZTZFXTRMJJDSNH413BQYK7/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1e4221f2-59b7-48f7-461f-bb15bc701369]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1e4221f2-59b7-48f7-461f-bb15bc701369]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [5a8b8485-701e-000d-3168-dc11ba000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA4J1GHVF9GAJ5R007J87D8Z/meta.json: cannot get properties for Azure blob, address: 01GA4J1GHVF9GAJ5R007J87D8Z/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA4J1GHVF9GAJ5R007J87D8Z/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [04e5dc3b-b58c-492d-5eef-67863bea9326]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [04e5dc3b-b58c-492d-5eef-67863bea9326]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [91f606cc-101e-003f-3668-dc496a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA1HXB15191WQGBN25FAVAWZ/meta.json: cannot get properties for Azure blob, address: 01GA1HXB15191WQGBN25FAVAWZ/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA1HXB15191WQGBN25FAVAWZ/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [c6e9987f-d049-47cc-7326-5d223689a833]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [c6e9987f-d049-47cc-7326-5d223689a833]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d3a1b6fc-f01e-001c-4c68-dc26a1000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA5DGDHVPKREHMSH9S8BWP64/meta.json: cannot get properties for Azure blob, address: 01GA5DGDHVPKREHMSH9S8BWP64/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA5DGDHVPKREHMSH9S8BWP64/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6679eaf2-b152-4bb9-41d3-1970570e6d75]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6679eaf2-b152-4bb9-41d3-1970570e6d75]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a1a-b01e-0039-5968-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA56MP9TS4A39VFF5XY8Q6BR/meta.json: cannot get properties for Azure blob, address: 01GA56MP9TS4A39VFF5XY8Q6BR/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA56MP9TS4A39VFF5XY8Q6BR/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [326ff393-27a0-4bc9-6e0f-64616c01e3f5]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [326ff393-27a0-4bc9-6e0f-64616c01e3f5]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a43-101e-0059-0368-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA8DMJ5A2JXQPBZMQTYA26G1/meta.json: cannot get properties for Azure blob, address: 01GA8DMJ5A2JXQPBZMQTYA26G1/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA8DMJ5A2JXQPBZMQTYA26G1/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [6b58cecc-85ce-40ff-6905-6caa77da2b71]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [6b58cecc-85ce-40ff-6905-6caa77da2b71]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2971-d01e-0000-5b68-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA8VC10C1HRPXR441MXDZMC5/meta.json: cannot get properties for Azure blob, address: 01GA8VC10C1HRPXR441MXDZMC5/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA8VC10C1HRPXR441MXDZMC5/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7900eac2-5416-49ef-5e09-7ae531d034ea]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7900eac2-5416-49ef-5e09-7ae531d034ea]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbe07-701e-0060-1468-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA7ZX3NJCX11H3AFRAFKR8WA/meta.json: cannot get properties for Azure blob, address: 01GA7ZX3NJCX11H3AFRAFKR8WA/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA7ZX3NJCX11H3AFRAFKR8WA/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [ff74eb7c-199b-4757-4db5-d52b16e51d7d]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [ff74eb7c-199b-4757-4db5-d52b16e51d7d]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [5a8b8491-701e-000d-3b68-dc11ba000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA5V7W1TNC2D7FFC5Z03MZ8T/meta.json: cannot get properties for Azure blob, address: 01GA5V7W1TNC2D7FFC5Z03MZ8T/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA5V7W1TNC2D7FFC5Z03MZ8T/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [c9298403-3f0b-4d13-7707-bbd9d3dc8e6b]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [c9298403-3f0b-4d13-7707-bbd9d3dc8e6b]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [2022-10-10T05:23:11.673563172Z true]\n   X-Ms-Request-Id: [6228955b-f01e-0058-4868-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA623K9VGKZ3NP5ME17P3BZD/meta.json: cannot get properties for Azure blob, address: 01GA623K9VGKZ3NP5ME17P3BZD/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA623K9VGKZ3NP5ME17P3BZD/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7426a617-037e-467f-533d-29171c89cdb5]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7426a617-037e-467f-533d-29171c89cdb5]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a5a87dde-d01e-000b-4c68-dce6c2000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA68Z9N8GA5QW506DTCS6W8G/meta.json: cannot get properties for Azure blob, address: 01GA68Z9N8GA5QW506DTCS6W8G/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA68Z9N8GA5QW506DTCS6W8G/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [51965525-95ef-43d4-6417-d78c33ab64d8]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [51965525-95ef-43d4-6417-d78c33ab64d8]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [11f2b2ba-e01e-0021-6d68-dc9387000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA86RTX97WSARZF3P0NJ8YNK/meta.json: cannot get properties for Azure blob, address: 01GA86RTX97WSARZF3P0NJ8YNK/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA86RTX97WSARZF3P0NJ8YNK/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [500b626d-d4ae-4ad9-69fe-787ce0d4b0fe]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [500b626d-d4ae-4ad9-69fe-787ce0d4b0fe]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [91f606d6-101e-003f-3e68-dc496a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA7S1CDSM1JSCPG3XQJKQTZE/meta.json: cannot get properties for Azure blob, address: 01GA7S1CDSM1JSCPG3XQJKQTZE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA7S1CDSM1JSCPG3XQJKQTZE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [2db949de-1f0f-48a4-611f-c413937aa1ba]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [2db949de-1f0f-48a4-611f-c413937aa1ba]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d3a1b70a-f01e-001c-5668-dc26a1000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA5MC4SVTCJZ3AFM5QC6DFJZ/meta.json: cannot get properties for Azure blob, address: 01GA5MC4SVTCJZ3AFM5QC6DFJZ/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA5MC4SVTCJZ3AFM5QC6DFJZ/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [ea97aa04-8c7a-4693-71b9-3b019cccc922]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [ea97aa04-8c7a-4693-71b9-3b019cccc922]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a296a-d01e-0000-5468-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA6PPR5BPT643GRXWD0H4PC5/meta.json: cannot get properties for Azure blob, address: 01GA6PPR5BPT643GRXWD0H4PC5/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA6PPR5BPT643GRXWD0H4PC5/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e6ae680d-8008-45d6-565b-66be53ead41c]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [e6ae680d-8008-45d6-565b-66be53ead41c]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [90bff56c-001e-002b-5068-dc8a0e000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA74E6N80EJBFKQC6R0Y3M6Z/meta.json: cannot get properties for Azure blob, address: 01GA74E6N80EJBFKQC6R0Y3M6Z/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA74E6N80EJBFKQC6R0Y3M6Z/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [afef6995-effd-40ec-5e79-2e4898303116]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [afef6995-effd-40ec-5e79-2e4898303116]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9f85c292-a01e-0004-6168-dc0b34000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA993FG9YK2P18EME6Y48D9Q/meta.json: cannot get properties for Azure blob, address: 01GA993FG9YK2P18EME6Y48D9Q/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA993FG9YK2P18EME6Y48D9Q/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [03b7ef4c-9ea8-41c1-5764-fa691e073ddd]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [03b7ef4c-9ea8-41c1-5764-fa691e073ddd]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a48-101e-0059-0868-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA1B1JMNX2SJ5NKAD4H125SE/meta.json: cannot get properties for Azure blob, address: 01GA1B1JMNX2SJ5NKAD4H125SE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA1B1JMNX2SJ5NKAD4H125SE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [a16bf96a-540f-4ca7-5121-5c842f152dd5]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [a16bf96a-540f-4ca7-5121-5c842f152dd5]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9e25deb7-f01e-003e-4d68-dc4897000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAA4JCGGJ06N5KSHWSQWN5H1/meta.json: cannot get properties for Azure blob, address: 01GAA4JCGGJ06N5KSHWSQWN5H1/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAA4JCGGJ06N5KSHWSQWN5H1/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [d78b0b24-4de5-468a-7bb8-5adf13d5457c]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [d78b0b24-4de5-468a-7bb8-5adf13d5457c]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2975-d01e-0000-5f68-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA9XPN89YERS6VEZYWA0T8DC/meta.json: cannot get properties for Azure blob, address: 01GA9XPN89YERS6VEZYWA0T8DC/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA9XPN89YERS6VEZYWA0T8DC/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [f0b29f9d-171a-48bd-467d-c6a4b44c0d30]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [f0b29f9d-171a-48bd-467d-c6a4b44c0d30]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a1c-b01e-0039-5b68-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA44A21VKSX96NPGFYS0KB62/meta.json: cannot get properties for Azure blob, address: 01GA44A21VKSX96NPGFYS0KB62/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA44A21VKSX96NPGFYS0KB62/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [864e811b-5eee-412a-4a88-a50a71e57c39]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [864e811b-5eee-412a-4a88-a50a71e57c39]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9e25deb8-f01e-003e-4e68-dc4897000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA9FZ6R9TSHNBG5SYFWBK8BE/meta.json: cannot get properties for Azure blob, address: 01GA9FZ6R9TSHNBG5SYFWBK8BE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA9FZ6R9TSHNBG5SYFWBK8BE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [89b0efe8-6c19-4359-585c-845715bfb37f]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [89b0efe8-6c19-4359-585c-845715bfb37f]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Re2022-10-10T05:23:11.673591873Z quest-Id: [91f606da-101e-003f-4268-dc496a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAABE3RF318RTPYE3654TQSB/meta.json: cannot get properties for Azure blob, address: 01GAABE3RF318RTPYE3654TQSB/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAABE3RF318RTPYE3654TQSB/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1fdaead2-f92a-4c61-439f-e20f37c535ad]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1fdaead2-f92a-4c61-439f-e20f37c535ad]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [11f2b2c6-e01e-0021-7568-dc9387000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA927R89CBSZRBN3CWB4AXCZ/meta.json: cannot get properties for Azure blob, address: 01GA927R89CBSZRBN3CWB4AXCZ/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA927R89CBSZRBN3CWB4AXCZ/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [bdd5c08c-4ac1-4919-6b17-7e12bb977ae2]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [bdd5c08c-4ac1-4919-6b17-7e12bb977ae2]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [5a8b8493-701e-000d-3d68-dc11ba000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GABDRR0BSHGD8DXA7M76TGS6/meta.json: cannot get properties for Azure blob, address: 01GABDRR0BSHGD8DXA7M76TGS6/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GABDRR0BSHGD8DXA7M76TGS6/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [badc04fe-676d-4b4b-5f10-b3951b3df224]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [badc04fe-676d-4b4b-5f10-b3951b3df224]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [90bff56e-001e-002b-5268-dc8a0e000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA6XJFD8KK5GE9WNKJ5GVESF/meta.json: cannot get properties for Azure blob, address: 01GA6XJFD8KK5GE9WNKJ5GVESF/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA6XJFD8KK5GE9WNKJ5GVESF/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e179a718-bc15-4b4c-4aff-62a6a088ba24]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [e179a718-bc15-4b4c-4aff-62a6a088ba24]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a7842e4d-901e-0007-7168-dc0833000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA7J5N5AWRMR50J7A10M95TJ/meta.json: cannot get properties for Azure blob, address: 01GA7J5N5AWRMR50J7A10M95TJ/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA7J5N5AWRMR50J7A10M95TJ/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [c00a1c31-11a9-4500-5c14-1379732fd0dc]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [c00a1c31-11a9-4500-5c14-1379732fd0dc]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [222c6d7b-a01e-0069-7868-dca11a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA8MG9DA106CJYWN2Q58SX57/meta.json: cannot get properties for Azure blob, address: 01GA8MG9DA106CJYWN2Q58SX57/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA8MG9DA106CJYWN2Q58SX57/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [644ba597-0228-4b5c-4f02-8495bc68c795]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [644ba597-0228-4b5c-4f02-8495bc68c795]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [58d5b537-e01e-0047-2e68-dc21dd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAB019GAEPNJ45NZ8H9GQ4ER/meta.json: cannot get properties for Azure blob, address: 01GAB019GAEPNJ45NZ8H9GQ4ER/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAB019GAEPNJ45NZ8H9GQ4ER/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [730636c0-c933-4210-473a-6567d16acf03]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [730636c0-c933-4210-473a-6567d16acf03]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a2978-d01e-0000-6268-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA7B9XX7J77BANAN003PF7S4/meta.json: cannot get properties for Azure blob, address: 01GA7B9XX7J77BANAN003PF7S4/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA7B9XX7J77BANAN003PF7S4/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8bc15378-2016-4ff9-5029-fd0059cf1c71]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8bc15378-2016-4ff9-5029-fd0059cf1c71]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [fdceb786-a01e-002d-1e68-dc7d76000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GABVG6G9EJDB6G49XYQV1YHD/meta.json: cannot get properties for Azure blob, address: 01GABVG6G9EJDB6G49XYQV1YHD/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GABVG6G9EJDB6G49XYQV1YHD/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [38139b4e-70fc-4aec-5223-34a36069be99]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [38139b4e-70fc-4aec-5223-34a36069be99]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a4a-101e-0059-0a68-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA9PTY0C8N879YTJNJSZ6AWX/meta.json: cannot get properties for Azure blob, address: 01GA9PTY0C8N879YTJNJSZ6AWX/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA9PTY0C8N879YTJNJSZ6AWX/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [45084407-a0a5-4d6b-7920-ff0c17e2aa4a]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [45084407-a0a5-4d6b-7920-ff0c17e2aa4a]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbe0d-701e-0060-1a68-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAAJ9V0AB5R8GYMBA22JP512/meta.json: cannot get properties for Azure blob, address: 01GAAJ9V0AB5R8GYMBA22JP512/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAAJ9V0AB5R8GYMBA22JP512/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [5b1ab4c8-be5e-4dba-4527-7be2c27e5520]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [5b1ab4c8-be5e-4dba-4527-7be2c27e5520]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [62289560-f01e-0058-4c68-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAAS5J8AKG8SPBTPVX00S7NF/meta.json: cannot get properties for Azure blob, address: 01GAAS5J8AKG8SPBTPVX00S7NF/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAAS5J8AKG8SPBTPVX00S7NF/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [01863ecf-2c10-4510-51ad-888ab63cc401]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [01863ecf-2c10-4510-51ad-888ab63cc401]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a5a87de9-d01e-000b-5768-dce6c2000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GACPZ3GANB5Z36PRNRZJD18G/meta.json: cannot get properties for Azure blob, address: 01GACPZ3GANB5Z36PRNRZJD18G/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GACPZ3GANB5Z36PRNRZJD18G/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1a30fc6f-d1a1-4b1d-7108-ccef1f1a8b90]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1a30fc6f-d1a1-4b1d-7108-ccef1f1a8b90]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9e25de2022-10-10T05:23:11.673624575Z c4-f01e-003e-5768-dc4897000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GABMMF89ERKX4XFX5HE5ETW7/meta.json: cannot get properties for Azure blob, address: 01GABMMF89ERKX4XFX5HE5ETW7/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GABMMF89ERKX4XFX5HE5ETW7/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [3da0860b-6d22-4163-5f18-0758604ab3cc]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [3da0860b-6d22-4163-5f18-0758604ab3cc]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9f85c29b-a01e-0004-6a68-dc0b34000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAE05F0SPPWHE6JQXQ0ZVBEH/meta.json: cannot get properties for Azure blob, address: 01GAE05F0SPPWHE6JQXQ0ZVBEH/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAE05F0SPPWHE6JQXQ0ZVBEH/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [bf4e4b46-3ed9-495b-6674-9cecac24d20e]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [bf4e4b46-3ed9-495b-6674-9cecac24d20e]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [222c6d7e-a01e-0069-7b68-dca11a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAB6X0RGFTKHJRT11RJAK4D5/meta.json: cannot get properties for Azure blob, address: 01GAB6X0RGFTKHJRT11RJAK4D5/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAB6X0RGFTKHJRT11RJAK4D5/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1983f5eb-bab9-4eaa-5dbd-1727ffe1563c]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1983f5eb-bab9-4eaa-5dbd-1727ffe1563c]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [d3a1b70c-f01e-001c-5868-dc26a1000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GADS9QRAWNJ08JW925TW0QRR/meta.json: cannot get properties for Azure blob, address: 01GADS9QRAWNJ08JW925TW0QRR/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GADS9QRAWNJ08JW925TW0QRR/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [885ae365-88a6-4c9f-6dc7-d6a8cd136901]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [885ae365-88a6-4c9f-6dc7-d6a8cd136901]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a7842e55-901e-0007-7868-dc0833000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAEMRMRGSW8A6YN8R1Y0DKJ3/meta.json: cannot get properties for Azure blob, address: 01GAEMRMRGSW8A6YN8R1Y0DKJ3/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAEMRMRGSW8A6YN8R1Y0DKJ3/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [cc064de2-313f-4c21-7103-0537c40a9c76]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [cc064de2-313f-4c21-7103-0537c40a9c76]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [fdceb78b-a01e-002d-2368-dc7d76000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA3FPXC5HTYD1ZKXA5SBTZJG/meta.json: cannot get properties for Azure blob, address: 01GA3FPXC5HTYD1ZKXA5SBTZJG/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA3FPXC5HTYD1ZKXA5SBTZJG/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [ffa9120d-0f6f-4bdf-50b8-5110cff42d2a]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [ffa9120d-0f6f-4bdf-50b8-5110cff42d2a]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [4f3225b2-801e-0031-0268-dca561000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GADBJ989A89HVE6C5GECWC8R/meta.json: cannot get properties for Azure blob, address: 01GADBJ989A89HVE6C5GECWC8R/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GADBJ989A89HVE6C5GECWC8R/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [1dfcf24b-defa-4720-450e-f114b429a3cf]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [1dfcf24b-defa-4720-450e-f114b429a3cf]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [5a8b8497-701e-000d-4168-dc11ba000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAF2G25JTT4AH82T14VA8WPE/meta.json: cannot get properties for Azure blob, address: 01GAF2G25JTT4AH82T14VA8WPE/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAF2G25JTT4AH82T14VA8WPE/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [5192d9bf-def5-47d0-5016-900b1c6abbd4]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [5192d9bf-def5-47d0-5016-900b1c6abbd4]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ad8dbe16-701e-0060-2268-dcbb94000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAF9BSND4P2FCM0DGN9H4GWN/meta.json: cannot get properties for Azure blob, address: 01GAF9BSND4P2FCM0DGN9H4GWN/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAF9BSND4P2FCM0DGN9H4GWN/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7f54b131-90dc-4e7e-4e44-f21db2bb8fda]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7f54b131-90dc-4e7e-4e44-f21db2bb8fda]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [62289567-f01e-0058-5368-dcfacd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GACXTTRADRNW8NYTB91K0MH5/meta.json: cannot get properties for Azure blob, address: 01GACXTTRADRNW8NYTB91K0MH5/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GACXTTRADRNW8NYTB91K0MH5/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [65fd5628-2a25-41ac-71eb-eab2724ec8a3]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [65fd5628-2a25-41ac-71eb-eab2724ec8a3]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [11f2b2c9-e01e-0021-7868-dc9387000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAC97N0AF0Y82AGX6AQ21141/meta.json: cannot get properties for Azure blob, address: 01GAC97N0AF0Y82AGX6AQ21141/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAC97N0AF0Y82AGX6AQ21141/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [94ce3cb5-f558-4c55-7613-0d120505993a]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [94ce3cb5-f558-4c55-7613-0d120505993a]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a297b-d01e-0000-6568-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAC2BXRESC989EQF2KS2Z5B6/meta.json: cannot get properties for Azure blob, address: 01GAC2BXRESC989EQF2KS2Z5B6/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAC2BXRESC989EQF2KS2Z5B6/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [c65238eb-6d71-4a12-50f1-f759dd25c424]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [c65238eb-6d71-4a12-50f1-f759dd25c424]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [9e25dec3-f01e-003e-5668-dc4897000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAD4PJ0JCTWPS7EWSHW9PWYC/meta.json: cannot get properties for Azure blob, address: 01GAD4PJ0JCTWPS7EWSHW9PWYC/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAD4PJ0JCTWPS7EWSHW9PWYC/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [b203c407-59c4-4e80-5db3-1c0bcb18497b]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [b203c407-59c4-4e80-5db3-1c0bcb18497b]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [91f606dc-101e-003f-43682022-10-10T05:23:11.673642375Z -dc496a000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAE7168F722ZD5ZWPJ59QKQ2/meta.json: cannot get properties for Azure blob, address: 01GAE7168F722ZD5ZWPJ59QKQ2/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAE7168F722ZD5ZWPJ59QKQ2/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8334818b-a7e0-40b3-5e79-996c13cec6e0]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8334818b-a7e0-40b3-5e79-996c13cec6e0]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [a44a297f-d01e-0000-6968-dcfeb6000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAEDWXGFSJK9XFGEW72AZJ5S/meta.json: cannot get properties for Azure blob, address: 01GAEDWXGFSJK9XFGEW72AZJ5S/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAEDWXGFSJK9XFGEW72AZJ5S/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [121325d5-9982-4e8f-4fbd-4e6239ad9bb8]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [121325d5-9982-4e8f-4fbd-4e6239ad9bb8]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [58d5b540-e01e-0047-3668-dc21dd000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GA3PJKHT936S1MY11ATDE4C0/meta.json: cannot get properties for Azure blob, address: 01GA3PJKHT936S1MY11ATDE4C0/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GA3PJKHT936S1MY11ATDE4C0/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [b7b180b8-7101-4127-73c8-30a848122042]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [b7b180b8-7101-4127-73c8-30a848122042]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [fec64e63-201e-001e-2168-dc245b000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GACG3C8HK9SGSX2C2V93XCR5/meta.json: cannot get properties for Azure blob, address: 01GACG3C8HK9SGSX2C2V93XCR5/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GACG3C8HK9SGSX2C2V93XCR5/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [7e0a6a14-4fb9-41fe-5d45-88014e9bcdb9]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [7e0a6a14-4fb9-41fe-5d45-88014e9bcdb9]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [ed161a1d-b01e-0039-5c68-dcbe12000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GAEVMAXHBS8CB2TBB1FFCN0S/meta.json: cannot get properties for Azure blob, address: 01GAEVMAXHBS8CB2TBB1FFCN0S/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GAEVMAXHBS8CB2TBB1FFCN0S/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [8b8ac36f-4aee-4bc2-5cb2-99f00b5c4c22]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [8b8ac36f-4aee-4bc2-5cb2-99f00b5c4c22]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [8b9a3a4d-101e-0059-0d68-dcfb30000000]\n   X-Ms-Version: [2020-04-08]\n\n\n; meta.json file exists: 01GADJE0GF3HK0T9CT1P9APKDF/meta.json: cannot get properties for Azure blob, address: 01GADJE0GF3HK0T9CT1P9APKDF/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://mystorageaccount.blob.core.windows.net/thanos/01GADJE0GF3HK0T9CT1P9APKDF/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [9afaf48d-c590-4aad-631f-10721a5a20ed]\n   X-Ms-Date: [Mon, 10 Oct 2022 05:23:10 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.\n   Date: [Mon, 10 Oct 2022 05:23:09 GMT]\n   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n   X-Ms-Client-Request-Id: [9afaf48d-c590-4aad-631f-10721a5a20ed]\n   X-Ms-Error-Code: [BlobNotFound]\n   X-Ms-Is-Soft-Deleted: [true]\n   X-Ms-Request-Id: [90bff571-001e-002b-5568-dc8a0e000000]\n   X-Ms-Version: [2020-04-08]\n\n\n","status":"not-healthy","ts":"2022-10-10T05:23:11.671617792Z"}

index and the chunk files are in place though.

There is no problem on v0.27.0

Oct 10 '22 08:10 PowerSurj

we also had quite a lot of azure issues since 0.28:

level=warn ts=2022-09-06T13:14:07.754975677Z caller=intrumentation.go:67 msg="changing probe status" status=not-ready reason="could not sync metas: incomplete view: 8 errors: meta.json file exists: 01GC2BJQ3GCAQCA6BCKR3M4BBT/meta.json: cannot get properties for Azure blob, address: 01GC2BJQ3GCAQCA6BCKR3M4BBT/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://1aargoststne.privatelink.blob.core.windows.net/platform-argos-amadeus-com/01GC2BJQ3GCAQCA6BCKR3M4BBT/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e984428b-03d2-4993-5584-81b8255d5cc3]\n   X-Ms-Date: [Tue, 06 Sep 2022 13:14:05 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.
[...]

We have also the same issue. Thanos version is v0.28 The meta.json file gets deleted but we have soft delete enabled so we are able to restore it. Then the compactor is able to find the blob and delete it.

Oct 31 '22 08:10 nicklev

I see the same issue using Azure Storage (RA-GRS). I am running v0.28. I tried serveral settings like --consistency-delay=30m and 10m but now I have --consistency-delay=5s running for 5 days without any error messages! Obviously, this is not the solution.

Oct 31 '22 12:10 paltryeffort

I downgraded Thanos to version 0.27 (Helm chart 11.3.0) as suggested to previous comment but I still get the same error.

level=error ts=2022-11-03T10:41:09.933669867Z caller=main.go:158 err="incomplete view: meta.json file exists: 01GGJAZ7YMG6HRX3GE1RPFXEDD/meta.json: cannot get properties for Azure blob, address: 01GGJAZ7YMG6HRX3GE1RPFXEDD/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., ...

Nov 03 '22 10:11 nicklev

we also had quite a lot of azure issues since 0.28:

level=warn ts=2022-09-06T13:14:07.754975677Z caller=intrumentation.go:67 msg="changing probe status" status=not-ready reason="could not sync metas: incomplete view: 8 errors: meta.json file exists: 01GC2BJQ3GCAQCA6BCKR3M4BBT/meta.json: cannot get properties for Azure blob, address: 01GC2BJQ3GCAQCA6BCKR3M4BBT/meta.json: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=404 The specified blob does not exist., Details: (none)\n   HEAD https://1aargoststne.privatelink.blob.core.windows.net/platform-argos-amadeus-com/01GC2BJQ3GCAQCA6BCKR3M4BBT/meta.json?timeout=61\n   Authorization: REDACTED\n   User-Agent: [Thanos Azure-Storage/0.14 (go1.18.5; linux)]\n   X-Ms-Client-Request-Id: [e984428b-03d2-4993-5584-81b8255d5cc3]\n   X-Ms-Date: [Tue, 06 Sep 2022 13:14:05 GMT]\n   X-Ms-Version: [2020-04-08]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 404 The specified blob does not exist.
[...]

We have the same issue and we keep performing the same "fix" for every blob that the compactor complains about then crashes on until it stops crashing.

Nov 11 '22 22:11 erewok

Bitnami has published a new chart version (v11.6.4) with Thanos 0.29.0, which seems to have fixed my issue. I'm going to test it on some more environments, but results so far have been promising.

Can anyone still reproduce the error with Thanos 0.29?

Nov 30 '22 15:11 maartengo

@maartengo, we upgraded to 0.29.0 3 weeks ago following your comment and haven't seen this error reappear since then. Thanks for the suggestion.

Dec 20 '22 20:12 erewok

Good to hear. If others still have this issue then I suggest that a new issue is created.

Jan 02 '23 07:01 maartengo

thanos thanos copied to clipboard

gateway/compact: Upgrade from 0.26 to 0.28 results in Azure BlobNotFound errors

thanos
thanos copied to clipboard